diff --git a/.claude/launch.json b/.claude/launch.json new file mode 100644 index 00000000..8f541789 --- /dev/null +++ b/.claude/launch.json @@ -0,0 +1,11 @@ +{ + "version": "0.0.1", + "configurations": [ + { + "name": "astro-dev", + "runtimeExecutable": "npm", + "runtimeArgs": ["run", "dev"], + "port": 4321 + } + ] +} diff --git a/AGENTS.md b/AGENTS.md index 97eedcf8..84167378 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,23 +13,26 @@ The single most important thing you can do here is **write documentation that so ## 2. Writing Style & Voice (READ THIS BEFORE WRITING CONTENT) -Good documentation reads like a knowledgeable colleague explaining something at your desk — not like a manual, and definitely not like ad copy. Picture the reader as someone who is in the middle of a task, slightly frustrated, and scanning for the answer. Everything below serves that reader. +Good documentation reads like a knowledgeable colleague explaining something at your desk — not like a manual, and definitely not like ad copy. Picture the reader as someone who is in the middle of a task, slightly frustrated, and scanning for the answer. Everything below serves that reader, and it follows the same playbook as the Stripe, Google developer docs: talk to the reader directly, lead with the task, and cut anything that isn't pulling weight. ### Write to the reader, about their task -- Use second person ("you"). Address the reader directly: "You'll need an API key before you start," not "Users must first obtain an API key." -- Make the reader the subject, not the product. Write "you land on the dashboard," not "KloudMate lands you on the dashboard." Naming the product once is fine and sometimes useful, but when it's the grammatical subject of sentence after sentence, the page reads like a feature announcement instead of a guide. The reader is the one doing the task — keep them at the center. When there's no natural "you," describe the system state instead ("the **Setup checklist** drawer opens automatically"). +- Use second person. Address the reader as "you," the way the Google, Stripe, and Datadog docs do: "you" for explanations, conditions, and outcomes ("you can group alerts by service," "if you're behind a proxy," "you'll land on the dashboard"), and the imperative for steps ("Open the **Add** menu," "Run the install script"). Prefer "you" over "the user" or "they." +- Make the reader the subject, not the product. Write "you land on the dashboard," not "KloudMate lands you on the dashboard." Naming the product once is fine and sometimes useful, but when it's the grammatical subject of sentence after sentence, the page reads like a feature announcement instead of a guide. When there's no natural "you," describe the system state instead ("the **Setup checklist** drawer opens automatically"). +- Use the imperative for procedures ("Open **Settings**," "Run the migration"), the serial (Oxford) comma, and American English spelling. This matches the Google, Stripe, and Datadog docs. - Lead with the goal, then the steps. Open a page by saying what the reader will accomplish and when they'd use it — not with history or background they didn't ask for. - Prefer active voice and present tense. "The alert fires within seconds" beats "An alert will be sent." - Cut the throat-clearing. Phrases like "It's worth noting that…" or "In this section, we will discuss…" add nothing. Just say the thing. ### Sound like a person, not a model +- Say it plainly, and don't dramatize. Don't set up a point with a rhetorical line before delivering it, and don't dress a plain fact in drama. Lines like "Autonomous mode is the answer," "That doesn't fit here," "Only the delivery vehicle differs," or a trailing tag like "which it is" read as AI voiceover. State the fact instead: "Autonomous mode removes the per-instance UI step." Cut figurative labels ("delivery vehicle," "zero-touch level") and tacked-on editorial asides ("the standard immutable-infrastructure model"). If a sentence sounds like a product pitch or a trailer voiceover, keep the fact and drop the performance. - Vary your sentence length. A run of identical medium-length sentences is the clearest sign a machine wrote the page. Mix short, direct lines with longer explanatory ones. - Watch for repeated sentence openers, too. Three sentences in a row that start with the same word — especially the product name — turn into a drumbeat. Recast some of them so the reader, an action, or the result leads instead. -- Contractions are good and usually better — "don't," "you'll," "it's." They match how people actually talk. +- Contractions are good and usually better: "don't," "you'll," "it's." They match how people actually talk. - Be concrete. Show a real endpoint, a real value, a real error message. "Set `retention_days` to `30`" is more useful than "configure the retention setting appropriately." - Don't pad. If a sentence can be deleted without losing meaning, delete it. +- Go easy on em dashes. A period, comma, or colon almost always reads cleaner, and a page peppered with them reads as machine-written. ### Words and phrases to avoid @@ -40,6 +43,7 @@ These read as filler or AI boilerplate. Strike them on sight: - **Overused verbs:** *leverage* → use "use"; *utilize* → use "use"; *delve into* → use "cover" or "explain"; *facilitate* → use "let" or "help." - **Condescending qualifiers:** simply, just, easy, obviously, of course. If a step were genuinely simple, the reader wouldn't be on this page. Telling a stuck user that something is "easy" only makes them feel worse. - **Vague intensifiers:** very, really, quite, "a variety of," "a wide range of," "a number of." +- **Padding and Latinate shortcuts:** drop "please" (just give the step); *e.g.* → "for example"; *i.e.* → "that is"; *via* → "through" or "with"; "in order to" → "to"; "once you" → "after you." (This mirrors the Datadog docs linter.) ### Be honest and helpful about the hard parts @@ -65,7 +69,7 @@ These read as filler or AI boilerplate. Strike them on sight: > "KloudMate offers a powerful and seamless way to effortlessly leverage your observability data. In today's complex cloud environments, it's worth noting that users can simply configure a wide variety of robust alerting options to suit their needs." > > **Prefer:** -> "Alerts tell you when something needs attention — an error-rate spike, a service going quiet, a budget threshold crossed. This page walks you through creating your first alert and routing it to Slack." +> "Alerts tell you when something needs attention: an error-rate spike, a service going quiet, a budget threshold crossed. This page walks you through creating your first alert and routing it to Slack." The "prefer" version is shorter, says what the reader gets, names real scenarios, and keeps the reader (not the product) at the center. Aim for that on every page. @@ -140,7 +144,7 @@ Run through this on any page you wrote or edited. It's the human-quality equival - [ ] Does the page open by telling the reader what they'll accomplish? - [ ] Did I read it out loud (or in my head)? Does it sound like a person, or like a template? -- [ ] Is the reader ("you") the subject of most sentences, rather than the product? Does any product name appear as the subject several sentences in a row? +- [ ] Is the reader ("you") the subject of the explanations, with steps in the imperative, rather than the product? Does any product name appear as the subject several sentences in a row? - [ ] Are there any banned words from Section 2 (powerful, seamless, leverage, simply, just…)? - [ ] Do the sentences vary in length, or do they all march at the same pace? - [ ] Will every code block actually run if pasted? diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..65a0e981 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,26 @@ +# CLAUDE.md + +Guidance for Claude Code when working in this repository (KloudMate documentation). + +## Read AGENTS.md first + +[AGENTS.md](AGENTS.md) is the authoritative guide for this repo: project context, the writing style and voice rules (Section 2, read it before writing any content), the tech stack (Astro + Starlight), and the directory and routing structure. Follow it. The rules below are additions specific to the KloudMate agent docs (`src/content/docs/docs/kloudmate-agent/**`), not replacements. + +## Agent-docs terminology: plain words, no product jargon + +- Do not use **"baseline."** It means nothing to a reader. Use **"eBPF monitoring"** or **"ETW monitoring"** for the eBPF or ETW layer, and **"automatic monitoring"** (or "what the agent collects automatically") for the general idea. Drop the redundant article too: write "eBPF monitoring needs kernel 4.14," not "the eBPF monitoring needs…". It reads as a mass noun, and the docs already start sentences with a bare "eBPF." +- Do not describe PHP container instrumentation as happening **"in place."** Say **"without a redeploy"** or **"directly"** — the point is that no rebuild or redeploy is needed; "default" would misdescribe it. Exception: the ASG page's "in-place upgrade" is the correct standard term for that deployment strategy, so leave it. + +## Phrasing: describe the system, do not assert what the reader owns + +AGENTS.md already says to keep the reader (not the product) as the subject, and to describe system state when there is no natural "you." One specific case to watch: do not make presumptuous ownership or management claims about the reader's setup, such as "You own the container instances" or "You own the raw YAML." Rephrase to describe the thing: + +- "You own the container instances" becomes "The EC2 instances are self-managed." +- "You own the raw YAML" becomes "The raw YAML is edited directly." +- "You manage the agent from the web interface" becomes "The agent is managed from the web interface." + +Give step instructions in the imperative: "Turn on the toggle," "Run the install script." The objection is only to declarative claims about the reader's environment, not to instructional voice. + +## Verify before you finish + +Run `npm run build` after content changes. Its postbuild step (`scripts/check-links.mjs`) fails on broken internal links, which catches renamed heading anchors and bad relative paths. A clean run ends with `check-links: no broken internal links found`. diff --git a/scripts/validate-frontmatter.mjs b/scripts/validate-frontmatter.mjs index b577956a..5932d79f 100644 --- a/scripts/validate-frontmatter.mjs +++ b/scripts/validate-frontmatter.mjs @@ -8,8 +8,9 @@ import { readFileSync, readdirSync, statSync } from 'fs'; import { join, relative } from 'path'; +import { fileURLToPath } from 'url'; -const DOCS_DIR = new URL('../src/content/docs', import.meta.url).pathname; +const DOCS_DIR = fileURLToPath(new URL('../src/content/docs', import.meta.url)); const REQUIRED_FIELDS = ['title', 'description']; const FRONTMATTER_RE = /^---\r?\n([\s\S]*?)\r?\n---/; diff --git a/src/content/docs/docs/alerts/create-alerts.mdx b/src/content/docs/docs/alerts/create-alerts.mdx index 12a06fc2..4bc9f18a 100644 --- a/src/content/docs/docs/alerts/create-alerts.mdx +++ b/src/content/docs/docs/alerts/create-alerts.mdx @@ -16,10 +16,10 @@ The **Alerts** screen displays a list of all existing alert rules along with the From the **more options (⋯)** icon on any rule, you can: -- **View** the rule details -- **View State History** for the rule - **Edit** the rule configuration - **Duplicate** the rule +- **Move to folder** to move the rule into a different [folder](../folders/) +- **View state history** for the rule - **Pause Evaluation** or **Pause Notifications** - **Delete** the rule @@ -37,7 +37,7 @@ Click the **Create Alert** button at the top-right corner of the **Alerts** scre - **From Template** — Start with a pre-configured alert for a common monitoring scenario. - **From Scratch** — Create a custom alert from an empty configuration. -- **Using AI** — Describe the alert you want in plain English and let KloudMate build it for you. +- **Using AI** — Open **KloudMate Builder** and describe the alert you want in plain English. ### From Template @@ -59,6 +59,9 @@ KloudMate's assistant can automatically generate queries and thresholds based on 1. In the **Create Alert** dialog, select **Using AI**. 2. A text box appears. Describe the alert you want to create. + +![Using AI prompt](./images/using-ai-prompt.png) + 3. Click **Create Alert**. KloudMate generates the alert configuration based on your description. 4. To review or adjust the settings, click the menu next to the alert and select **Edit**. @@ -176,7 +179,7 @@ Two top-level fields and one section: The Notifications step covers how this alert flows into the grouping engine. KloudMate replaced free-form notification tags with **labels** that routing rules match on: -- **Labels** are auto-derived from query dimensions, the alert rule's folder, and a small set of system keys. You don't enter them by hand. +- **Labels** — Key/value pairs attached to every signal this alert emits. Add them with **Add label**; the list can also be left empty. Routing rules match on these labels, plus the reserved `alarm_id`, `alarm_rule_folder_id`, and per-instance query labels, to decide which channels notify and how signals are grouped. - **Routing** is decided by [Routing Rules](../routing-rules/) in **Alerts → Routing rules**, which match alerts by label and send them to one or more notification channels. - **Severity** flows through the reserved `severity` annotation from the previous step — downstream tools use it to prioritize. diff --git a/src/content/docs/docs/alerts/images/advanced-mode-options.png b/src/content/docs/docs/alerts/images/advanced-mode-options.png index 208da9f4..c21c6d55 100644 Binary files a/src/content/docs/docs/alerts/images/advanced-mode-options.png and b/src/content/docs/docs/alerts/images/advanced-mode-options.png differ diff --git a/src/content/docs/docs/alerts/images/alarm-detail-overview.png b/src/content/docs/docs/alerts/images/alarm-detail-overview.png index 2a621824..0c0e89d3 100644 Binary files a/src/content/docs/docs/alerts/images/alarm-detail-overview.png and b/src/content/docs/docs/alerts/images/alarm-detail-overview.png differ diff --git a/src/content/docs/docs/alerts/images/create-alarm-dialog.png b/src/content/docs/docs/alerts/images/create-alarm-dialog.png index 95d3b6b4..781082a2 100644 Binary files a/src/content/docs/docs/alerts/images/create-alarm-dialog.png and b/src/content/docs/docs/alerts/images/create-alarm-dialog.png differ diff --git a/src/content/docs/docs/alerts/images/from-template-dropdown.png b/src/content/docs/docs/alerts/images/from-template-dropdown.png index e84b9a8a..e7a0fe1b 100644 Binary files a/src/content/docs/docs/alerts/images/from-template-dropdown.png and b/src/content/docs/docs/alerts/images/from-template-dropdown.png differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.jpeg b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.jpeg index 3e23d4fe..e3b77572 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.jpeg and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.jpeg differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.png b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.png index dad11693..f7d3b5b2 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.png and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-1.png differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-2.png b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-2.png index dd65310d..7fcb331c 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-2.png and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-2.png differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-3.png b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-3.png index 45263afb..a1986f76 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-3.png and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-3.png differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-4.png b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-4.png index 3f4c8d67..6cb0f520 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-4.png and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-4.png differ diff --git a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-5.png b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-5.png index 31b22fcc..14350afa 100644 Binary files a/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-5.png and b/src/content/docs/docs/alerts/images/setting-up-kloudmate-alarms-5.png differ diff --git a/src/content/docs/docs/alerts/images/using-ai-prompt.png b/src/content/docs/docs/alerts/images/using-ai-prompt.png new file mode 100644 index 00000000..63b58162 Binary files /dev/null and b/src/content/docs/docs/alerts/images/using-ai-prompt.png differ diff --git a/src/content/docs/docs/apm-and-tracing/apm-views/api-monitoring.mdx b/src/content/docs/docs/apm-and-tracing/apm-views/api-monitoring.mdx index a42c099e..c5346d43 100644 --- a/src/content/docs/docs/apm-and-tracing/apm-views/api-monitoring.mdx +++ b/src/content/docs/docs/apm-and-tracing/apm-views/api-monitoring.mdx @@ -13,9 +13,7 @@ Where the [Service Map](../service-map/) shows caller-to-callee dependencies and In the left navigation, open **APM & Tracing → API Monitoring**. -:::note[Screenshot placeholder] -Add a screenshot of the API Monitoring page here (`./images/api-monitoring-1.png`): the page header, the filter bar, and the endpoints table with protocol badges, the Status column, and throughput sparklines visible. -::: +![API Monitoring page, showing the filter bar and the endpoints table with protocol badges, the Status column, and throughput trend sparklines.](./images/api-monitoring-1.png) Use the **time range** picker in the top-right to set the observation window. The view defaults to the last **24 hours** so the per-endpoint trend sparklines have enough history to draw. @@ -64,9 +62,7 @@ To find failing endpoints quickly, sort by **Error rate**, or set the **Status** Click any row to open the endpoint drill-down in a slide-over panel. The panel is shareable — its address encodes the selected endpoint, so you can send a teammate a direct link, and closing it returns you to the list with your filters, sort, and time range intact. -:::note[Screenshot placeholder] -Add a screenshot of the endpoint drill-down here (`./images/api-monitoring-2.png`): the drawer header with protocol and method badges, the Requests / Error rate / p95 summary tiles, and the time-series charts. -::: +![Endpoint drill-down panel, showing the protocol and method badges, the Requests / Error rate / p95 latency summary tiles, and the throughput, error rate, latency, and status class charts.](./images/api-monitoring-2.png) At the top, summary tiles show **Requests**, **Error rate**, and **p95 latency** for the endpoint over the selected window. Below them are time-series charts: diff --git a/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-1.png b/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-1.png new file mode 100644 index 00000000..2cc5ec23 Binary files /dev/null and b/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-1.png differ diff --git a/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-2.png b/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-2.png new file mode 100644 index 00000000..ae57cc41 Binary files /dev/null and b/src/content/docs/docs/apm-and-tracing/apm-views/images/api-monitoring-2.png differ diff --git a/src/content/docs/docs/apm-and-tracing/apm-views/service-map.mdx b/src/content/docs/docs/apm-and-tracing/apm-views/service-map.mdx index 3f483ae6..a66e350b 100644 --- a/src/content/docs/docs/apm-and-tracing/apm-views/service-map.mdx +++ b/src/content/docs/docs/apm-and-tracing/apm-views/service-map.mdx @@ -4,10 +4,19 @@ description: "Use the Service Map to understand service dependencies, traffic fl sidebar: order: 3 --- -The **Service Map** is a visual representation of the relationships and interdependencies between services in your workspace. It is generated from tracing data and gives you a topology-level view of your architecture. +The **Service Map** is a visual representation of the relationships and interdependencies between services in your workspace. It is built from your services' distributed traces and gives you a topology-level view of your architecture. You can use it to understand service dependencies, traffic flow, and operational health at a glance. +## How the map is built + +The map is built from distributed traces, so both sources of tracing feed it: + +- **[eBPF monitoring](../../../kloudmate-agent/ebpf-observability/)** captures traces at the kernel level with no code change, so an eBPF-covered service appears on the map on its own — on VMs, hosts, and containers. (On [Kubernetes](../../../kloudmate-agent/auto-instrumentation/kubernetes/), eBPF provides the network topology instead, and traces come from the SDK.) +- **[Auto-instrumentation](../../../kloudmate-agent/auto-instrumentation/overview/)** and the OpenTelemetry SDK produce traces from inside the application. + +Each service becomes a node under its `service.name`, and an edge is drawn wherever one service's trace calls another. A service needs a distinct `service.name` to appear as its own node — the agent gives an eBPF-monitored service the same name it carries once auto-instrumented, so it keeps one identity whether eBPF or the SDK is tracing it. + ## Overview Click **Service Map** in the left navigation to open the map. diff --git a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ebpf.mdx b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ebpf.mdx index cb561ac1..b7345c69 100644 --- a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ebpf.mdx +++ b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ebpf.mdx @@ -1,66 +1,60 @@ --- title: "Zero-Downtime Observability (eBPF)" -description: "Observe your infrastructure seamlessly with the KloudMate eBPF Agent—no code changes or application restarts required." +description: "Get APM metrics, traces, and a service map from the Linux kernel with eBPF, with no code changes and no application restarts." sidebar: label: "eBPF (Zero-Downtime)" --- -Mission-critical environments managed by forward-thinking SRE teams and SOCs face a fundamental paradox: you need deep observability to maintain stability, but deploying traditional observability agents often requires application restarts or code changes that compromise that stability. +Traditional APM agents usually need a code change or an application restart before they collect anything. KloudMate's eBPF approach avoids both: it runs in the Linux kernel and observes the system's behavior without altering your user-level applications. -KloudMate's eBPF (Extended Berkeley Packet Filter) approach solves this paradox. It provides a drop-in APM component that lives in the Linux kernel—observing the behavior of the entire system dynamically without altering user-level applications. +Enable the eBPF receiver on the KloudMate Agent and you get metrics, traces, and a service map for your infrastructure with no code changes, no per-service configuration, and no restarts. -By deploying the KloudMate Agent with the eBPF receiver enabled, you unlock instant, out-of-the-box observability for your entire infrastructure—**without requiring code changes, manual configurations, or application restarts.** +## What you get -## Out-of-the-Box Value +The eBPF receiver reads activity from the kernel and turns it into telemetry. As soon as it's running: -The eBPF receiver extracts vast amounts of actionable metrics securely from the kernel layer. From day one, your teams receive: +### Universal RED metrics +Request Rate, Error Rate, and Duration (latency) for every observed service. +- **Protocol-aware:** HTTP, HTTP/2, gRPC, MySQL, PostgreSQL, Redis, MongoDB, Kafka, and Elasticsearch. +- **Operational context:** each metric carries markers such as HTTP status codes, gRPC status flags, and database operation type. -### 1. Universal RED Metrics -Automatically generates and exports comprehensive Request Rate, Error Rate, and Duration (Latency) metrics across all observed services. -- **Protocol Agnostic:** Automatically supports HTTP/HTTP2, gRPC, MySQL, PostgreSQL, Redis, MongoDB, Kafka, and Elasticsearch. -- **Operational Context:** Every metric includes critical operational markers such as exact HTTP status codes, gRPC status flags, and Database query behaviors. +### Distributed tracing +- Links an incoming request to the outbound calls it makes, for example an HTTP handler querying a database. +- Produces OpenTelemetry-compliant spans, which build the service map. -### 2. Auto-Distributed Tracing -- Intelligently links incoming network requests to outgoing dependency calls instantaneously (e.g., an HTTP handler querying a database instance). -- Generates strictly compliant OpenTelemetry trace spans, producing a seamless Service Map representing your living architecture. +### Service inventory and metadata +- Detects each process's language (`km.apm.runtime.language`) without touching the binary. +- Adds host IDs, process IDs, and cloud-provider metadata to traces. +- On Kubernetes, correlates telemetry with `namespace`, `pod_name`, `deployment`, and `node_name`. -### 3. Dynamic Service Inventory & Metadata -- Automatically identifies the process language (`km.apm.runtime.language`) of your running applications without touching the binary. -- Enriches traces with dense Host IDs, Process IDs, and Cloud Provider metadata tags. -- If running under Kubernetes, it correlates metrics securely with K8s attributes (`namespace`, `pod_name`, `deployment`, `node_name`) dynamically. +### Network observability +- Captures L3/L4 flow metrics: bytes transferred, TCP retransmits, connection-state changes, and packet drops. +- Gives visibility into service-to-service communication for dependency mapping and security review. -### 4. Granular Network Observability -- Captures L3/L4 network flow metrics transparently—including full bytes transferred, TCP retransmits, state alterations, and packet drops. -- Empowers security teams with definitive visibility into service-to-service communication dependencies and anomalous traffic mapping. +## eBPF compared with SDK instrumentation -## The eBPF Edge vs. Traditional SDK Telemetry +eBPF and language SDKs solve different problems. eBPF gives broad, no-touch coverage at network boundaries; SDKs give deep, in-process detail. They work well together. -While traditional OpenTelemetry heavily relies on language-specific SDKs (manual instrumentation) or runtime dependencies (auto-instrumentation/zero-code), the KloudMate eBPF approach is superior for foundational infrastructure visibility. - -| Feature | KloudMate eBPF Receiver | SDK / Auto-Instrumentation | +| Feature | KloudMate eBPF receiver | SDK / auto-instrumentation | | :--- | :--- | :--- | -| **Code Changes** | **None.** Deploy the agent to the Linux node. | Requires SDK dependencies or attaching agents via env vars. | -| **Application Restarts** | **No restarts required.** Immediate visibility. | Requires rolling restarts to inject instrumentation agents. | -| **Setup Complexity** | **Low.** Single DaemonSet or VM process per host. | **High.** Per-service configuration, library updates. | -| **Language Support** | **Universal.** Go, Rust, C++, Python, Java, Node.js, Ruby. | Requires per-language SDKs; compiled languages are difficult. | -| **Performance Overhead**| **Extremely low.** Runs securely in kernel space. | Higher, especially with rich library-level auto-instrumentation. | -| **Missing Services** | Impossible — the kernel sees every network packet. | Un-instrumented services remain blind spots. | - -### Limitations of eBPF - -While eBPF provides unparalleled breadth, it is important to understand its limitations compared to manual SDK instrumentation: -- **Business Logic Context:** eBPF cannot inherently understand custom business logic (e.g., `user_id`, `cart_value`, or specific function execution paths within your application code). -- **In-Process Tracing:** eBPF excels at network boundaries and system calls. It does not provide deep stack traces or internal function execution timing like an SDK would. +| **Code changes** | None. Deploy the agent to the Linux node. | SDK dependencies, or agents attached via env vars. | +| **Application restarts** | None. | Rolling restarts to inject the instrumentation. | +| **Setup** | One DaemonSet or VM process per host. | Per-service configuration and library updates. | +| **Language support** | Any language, including Go, Rust, and C++. | Per-language SDKs; compiled languages are harder. | +| **Overhead** | Low; runs in kernel space. | Higher, especially with rich library instrumentation. | +| **Coverage** | Every service the kernel sees on the network. | Un-instrumented services are blind spots. | -## Recommended Strategic Deployment +### What eBPF cannot do -1. **Start with eBPF:** The eBPF Receiver serves as the ultimate foundational layer. Deploy the KloudMate Agent to grab immediate Service Maps, rigorous network insights, and universal APM metrics across every language and stack. -2. **Enrich Where Necessary:** Once eBPF has illuminated your architecture, strategically deploy [Manual Instrumentation](../../manual-instrumentation/) SDKs into the specific handful of applications requiring bespoke business logic mapping (such as tracing a specific User ID or capturing unique transaction states). +eBPF works at network boundaries and system calls, so it does not see inside your application: +- **Business logic:** it cannot read custom values like `user_id` or `cart_value`, or trace specific function paths. +- **In-process detail:** it does not produce deep stack traces or internal function timing the way an SDK does. -The KloudMate platform seamlessly merges eBPF kernel intelligence with your application telemetry data, giving your SRE teams a unified view of reality. +## Recommended approach -## Configuration & Setup +1. **Start with eBPF.** Deploy the KloudMate Agent to get a service map, network metrics, and RED metrics across every language with no per-service work. +2. **Add SDKs where you need depth.** For the services where you need business-logic context, such as a specific user ID or transaction state, add [Manual Instrumentation](../../manual-instrumentation/). The eBPF and SDK data join into one view. -To install the agent and enable eBPF observability features, please refer to the main configuration guide. +## Configuration and setup -[**View eBPF Agent Installation & Configuration Docs →**](../../../kloudmate-agent/ebpf-observability/) +To install the agent and enable eBPF, see the [eBPF observability guide](../../../kloudmate-agent/ebpf-observability/). diff --git a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/index.mdx b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/index.mdx index 95078b7a..ed9ed728 100644 --- a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/index.mdx +++ b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/index.mdx @@ -18,13 +18,13 @@ If your applications run on Linux (VMs, bare metal, or Kubernetes), the preferre ## Kubernetes Workloads -If your applications run on Kubernetes, you can leverage the KloudMate Agent to easily deploy APM and patching. +If your applications run on Kubernetes, you can use the KloudMate Agent to deploy APM. - [Kubernetes APM with KloudMate Agent](./kubernetes-apm/) ## OpenTelemetry Auto-Instrumentation -For richer framework-specific attributes or environments where eBPF is not an option, KloudMate natively leverages standard OpenTelemetry auto-instrumentation agents. This approach requires attaching an agent to your runtime and **restarting your application**. +For richer framework-specific attributes, or environments where eBPF is not an option, KloudMate uses standard OpenTelemetry auto-instrumentation agents. This approach requires attaching an agent to your runtime and **restarting your application**. Select your language to get started: diff --git a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ruby.mdx b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ruby.mdx index a80d9872..c4c53d47 100644 --- a/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ruby.mdx +++ b/src/content/docs/docs/apm-and-tracing/auto-instrumentation/ruby.mdx @@ -1,17 +1,15 @@ --- title: "Ruby Auto Instrumentation" -description: "Send auto-instrumented Ruby traces to KloudMate." +description: "Auto-instrument a Ruby or Rails app with OpenTelemetry and send the traces to KloudMate — what gets captured out of the box, and where the SQL comes from." sidebar: label: "Ruby" --- -KloudMate natively supports OpenTelemetry (OTel) for auto-instrumenting Ruby applications. Because OpenTelemetry actively maintains robust auto-instrumentation agents for Ruby, we rely on their standard process to ensure you always have the most up-to-date and compatible tracing capabilities. +KloudMate uses OpenTelemetry to auto-instrument Ruby applications. The instrumentation gems are maintained by the OpenTelemetry project; KloudMate ingests whatever they emit over OTLP, so your coverage tracks the upstream gems. -## Routing Data to KloudMate +## Route data to KloudMate -To send auto-instrumented data to KloudMate, you only need to configure the OpenTelemetry agent with KloudMate's OTLP endpoint and your API key using environment variables. - -When running your Ruby application, provide the following environment variables: +Point the OpenTelemetry exporter at KloudMate's OTLP endpoint and your API key through environment variables. When you run your app, set: ```bash export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.kloudmate.com:4318" @@ -19,10 +17,53 @@ export OTEL_EXPORTER_OTLP_HEADERS="Authorization=YOUR_API_KEY" export OTEL_SERVICE_NAME="your-ruby-service-name" ``` -*Replace `YOUR_API_KEY` with your actual KloudMate API key.* +Replace `YOUR_API_KEY` with your KloudMate API key. + +## Install the gems + +Add the SDK, the all-in-one instrumentation bundle, and the OTLP exporter to your `Gemfile`: + +```ruby +gem 'opentelemetry-sdk' +gem 'opentelemetry-instrumentation-all' +gem 'opentelemetry-exporter-otlp' +``` + +Enable them once at boot. For Rails, use an initializer: + +```ruby +# config/initializers/opentelemetry.rb +require 'opentelemetry/sdk' +require 'opentelemetry/instrumentation/all' +require 'opentelemetry/exporter/otlp' + +OpenTelemetry::SDK.configure do |c| + c.use_all # turn on every instrumentation whose library is loaded +end +``` + +The `opentelemetry-exporter-otlp` gem is required. Without it the SDK has no exporter, logs `otlp exporter cannot be configured` at startup, and drops every span. + +## What gets instrumented automatically + +`use_all` turns on each instrumentation whose underlying library is loaded. For a Rails app it covers the whole request path with no extra code: + +| Layer | Gem | What you get | +|---|---|---| +| HTTP server | `rack`, `action_pack` | One span per request — the root of the trace. `action_pack` adds the matched route, controller, and action, and names the span `GET /users/:id` (Rails 7.1+). | +| Database (SQL) | `pg`, `mysql2`, `trilogy` | One span per query, carrying the SQL as `db.statement`. This is the span that shows the query text. | +| Database (ORM) | `active_record` | Timing spans for model operations — `User#save`, `User.create`, `User query`. They show the Rails call, not the SQL. | +| View rendering | `action_view` | Spans for template, partial, collection, and layout renders, tagged with the template path. | +| Background jobs | `active_job`, plus `sidekiq`, `delayed_job`, `resque`, `que` | An enqueue span and a perform span per job, linked across the two processes. | +| Outbound HTTP | `net_http`, `faraday`, `http`, `httpx`, `excon` | A client span for each call your app makes to another service. | +| Caches and stores | `redis`, `dalli`, `mongo` | A span per cache or store operation. | + +The bundle also covers GraphQL, gRPC, Sinatra, Grape, the AWS SDK, and Kafka and RabbitMQ clients when your app uses them. -## Setup Instructions +:::note +The SQL query text (`db.statement`) comes from the database driver instrumentation (`pg`, `mysql2`), not from `active_record`. SQLite has no such instrumentation, so a SQLite app shows the model operation but no SQL — PostgreSQL and MySQL show the query. +::: -For the actual installation and usage instructions of the OpenTelemetry Ruby auto-instrumentation gems, please refer to the official OpenTelemetry documentation. +## Reference -[**View Official OpenTelemetry Ruby Zero-code Instrumentation Docs →**](https://opentelemetry.io/docs/zero-code/ruby/) +For setup, configuration options, and the full list of instrumentations, see the [OpenTelemetry Ruby zero-code instrumentation docs](https://opentelemetry.io/docs/zero-code/ruby/). diff --git a/src/content/docs/docs/apm-and-tracing/index.mdx b/src/content/docs/docs/apm-and-tracing/index.mdx index 51e30f4f..f478672c 100644 --- a/src/content/docs/docs/apm-and-tracing/index.mdx +++ b/src/content/docs/docs/apm-and-tracing/index.mdx @@ -23,6 +23,7 @@ Start in **APM Views** when you need higher-level service health and dependency KloudMate supports multiple ways to generate APM and tracing data: - [KloudMate Agent eBPF Observability](../kloudmate-agent/ebpf-observability/) for kernel-level tracing and RED metrics without code changes or restarts +- [KloudMate Agent Application APM](../kloudmate-agent/auto-instrumentation/overview/) to add full distributed tracing to a service, one at a time, and view the results here - [Auto Instrumentation](./auto-instrumentation/) for the preferred Kubernetes APM flow through the KloudMate Agent - [Manual Instrumentation](./manual-instrumentation/) for SDK-based tracing, application metrics, and custom metrics with the most control. diff --git a/src/content/docs/docs/apm-and-tracing/trace-explorer.mdx b/src/content/docs/docs/apm-and-tracing/trace-explorer.mdx index f7527d3f..03c2abc8 100644 --- a/src/content/docs/docs/apm-and-tracing/trace-explorer.mdx +++ b/src/content/docs/docs/apm-and-tracing/trace-explorer.mdx @@ -50,7 +50,7 @@ Click **Saved Queries** to view and apply previously saved searches. To save a n ## Traces Result Views -Switch between views using the toggle above the results. They all read the same trace data through the active time range and filters — each gives you a different level of detail. +Switch between views using the toggle above the results. They all read the same trace data, subject to the active time range and filters — each gives you a different level of detail. ### Trace Summary @@ -88,7 +88,7 @@ This view is best when you need the most granular look at tracing activity. To i ![image](./images/traces-6.jpeg) -## Open a trace +## Open a Trace Select a trace or span from any result view to open it in [Trace Detail](../trace-detail/), where you inspect the timeline waterfall, per-span attributes, and the Service List, Request Flow, and AI Flow views. diff --git a/src/content/docs/docs/aws-integration/index.mdx b/src/content/docs/docs/aws-integration/index.mdx index 2ec721e3..ab0696b7 100644 --- a/src/content/docs/docs/aws-integration/index.mdx +++ b/src/content/docs/docs/aws-integration/index.mdx @@ -1,6 +1,6 @@ --- title: "AWS Integration" -description: "Connect AWS accounts to KloudMate and automatically ingest telemetry using our native integrations." +description: "Connect an AWS account to KloudMate and automatically ingest metrics, logs, and traces from your AWS services." sidebar: label: "Overview" order: 1 diff --git a/src/content/docs/docs/azure-integration/auto-integration-with-azure-monitor.mdx b/src/content/docs/docs/azure-integration/auto-integration-with-azure-monitor.mdx index ffde8467..e7b6f902 100644 --- a/src/content/docs/docs/azure-integration/auto-integration-with-azure-monitor.mdx +++ b/src/content/docs/docs/azure-integration/auto-integration-with-azure-monitor.mdx @@ -1,10 +1,10 @@ --- title: "Auto-Integration with Azure Monitor" -description: "Documentation for Auto-Integration with Azure Monitor (Azure CLI Access required)" +description: "Integrate Azure Monitor with KloudMate automatically using an Azure CLI script." sidebar: order: 1 --- -This document provides an automated approach to integrating Azure Monitor with KloudMate using the OpenTelemetry Receiver. The Azure Monitor OpenTelemetry Receiver allows you to scrape telemetry data from Azure Monitor resources and send them to KloudMate for centralized monitoring and analysis. +Integrate Azure Monitor with KloudMate automatically using the OpenTelemetry Azure Monitor receiver. The receiver scrapes telemetry from Azure Monitor resources and sends it to KloudMate for centralized monitoring and analysis. ## Prerequisites diff --git a/src/content/docs/docs/azure-integration/azure-application-gateway-log-monitoring.mdx b/src/content/docs/docs/azure-integration/azure-application-gateway-log-monitoring.mdx index 52f4e2b8..9238b699 100644 --- a/src/content/docs/docs/azure-integration/azure-application-gateway-log-monitoring.mdx +++ b/src/content/docs/docs/azure-integration/azure-application-gateway-log-monitoring.mdx @@ -1,6 +1,6 @@ --- title: "Azure Application Gateway Log Monitoring using Event Hub" -description: "Documentation for Azure Application Gateway Log Monitoring using Event Hub" +description: "Stream Azure Application Gateway logs to KloudMate through Azure Event Hub." sidebar: order: 2 --- diff --git a/src/content/docs/docs/azure-integration/enable-service-principal-azure.mdx b/src/content/docs/docs/azure-integration/enable-service-principal-azure.mdx index b952c8cc..ed7214ba 100644 --- a/src/content/docs/docs/azure-integration/enable-service-principal-azure.mdx +++ b/src/content/docs/docs/azure-integration/enable-service-principal-azure.mdx @@ -1,10 +1,10 @@ --- title: "Enable Service Principal Azure" -description: "Documentation for Enable Service Principal Azure" +description: "Create an Azure AD service principal so KloudMate can authenticate to Azure Monitor." sidebar: order: 5 --- -This document demonstrates how to enable service principals on Azure to integrate Azure monitor with KloudMate +Create a service principal on Azure so KloudMate can authenticate to Azure Monitor. Set it up once, then use it in the auto or manual integration. 1\. Navigate to Microsoft Entra ID and register a new app. diff --git a/src/content/docs/docs/azure-integration/manual-integration-of-azure-monitor.mdx b/src/content/docs/docs/azure-integration/manual-integration-of-azure-monitor.mdx index c6e18035..dd760952 100644 --- a/src/content/docs/docs/azure-integration/manual-integration-of-azure-monitor.mdx +++ b/src/content/docs/docs/azure-integration/manual-integration-of-azure-monitor.mdx @@ -1,6 +1,6 @@ --- title: "Manual Integration of Azure Monitor" -description: "Documentation for Manual Integration of Azure Monitor" +description: "Manually configure a KloudMate Agent to scrape Azure Monitor metrics." sidebar: order: 3 --- diff --git a/src/content/docs/docs/database-monitoring/database-activity-monitoring-dam/ml-classifier-deployment-guide.mdx b/src/content/docs/docs/database-monitoring/database-activity-monitoring-dam/ml-classifier-deployment-guide.mdx index b6f60db5..990cf219 100644 --- a/src/content/docs/docs/database-monitoring/database-activity-monitoring-dam/ml-classifier-deployment-guide.mdx +++ b/src/content/docs/docs/database-monitoring/database-activity-monitoring-dam/ml-classifier-deployment-guide.mdx @@ -5,7 +5,7 @@ sidebar: order: 2 --- -The **KloudMate ML Classifier** is an AI-powered data security module that natively integrates with your telemetry pipeline. By leveraging advanced Machine Learning engines alongside pattern-matching rules, it automatically inspects database queries in real-time to meticulously detect sensitive information, such as PII (Personally Identifiable Information) and PHI (Protected Health Information). It then dynamically enriches your traces with risk fingerprints before your telemetry ever leaves your infrastructure. +The **KloudMate ML Classifier** inspects database queries in your telemetry pipeline to detect sensitive data, such as Personally Identifiable Information (PII) and Protected Health Information (PHI). It combines a machine learning model with pattern-matching rules, and tags your traces with risk fingerprints before the telemetry leaves your infrastructure. This guide covers two strategies for deploying the **KloudMate ML Classifier**: diff --git a/src/content/docs/docs/database-monitoring/database-integrations/mongodb-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/mongodb-monitoring.mdx index 09860a8c..5c1c7fda 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/mongodb-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/mongodb-monitoring.mdx @@ -1,10 +1,8 @@ --- title: "MongoDB" -description: "Documentation for MongoDB Monitoring" +description: "Monitor MongoDB health, performance, and logs with the KloudMate Agent and OpenTelemetry." --- -MongoDB is a widely used NoSQL database designed for scalability and flexibility. It stores data in a JSON-like format (BSON), enabling dynamic schema design and efficient handling of unstructured data. MongoDB is known for its high performance in read- and write-heavy environments and is commonly used for real-time analytics, content management, and IoT systems. It supports horizontal scaling through sharding and provides built-in replication for high availability. - -MongoDB Monitoring in KloudMate helps you observe the health, performance, and behavior of your MongoDB servers by collecting metrics and logs using the **KloudMate Agent** powered by OpenTelemetry. This allows you to monitor MongoDB instances running on AWS EC2, Azure Virtual Machines, or on-premise servers from a centralized view. +This guide sets up MongoDB monitoring with the **KloudMate Agent** and OpenTelemetry. It collects metrics and logs so you can watch the health, performance, and behavior of MongoDB instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. ### **What This Integration Provides** diff --git a/src/content/docs/docs/database-monitoring/database-integrations/mssql-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/mssql-monitoring.mdx index 18410d7a..08070f47 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/mssql-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/mssql-monitoring.mdx @@ -1,11 +1,9 @@ --- title: "MSSQL" -description: "Documentation for MSSQL Monitoring" +description: "Monitor Microsoft SQL Server health, performance, queries, and logs with the KloudMate Agent and OpenTelemetry." --- -Microsoft SQL Server (MSSQL) is a robust relational database management system from Microsoft, renowned for its enterprise-grade performance, high availability features, and advanced analytics capabilities. It powers mission-critical applications, data warehousing, and business intelligence workloads. - -MSSQL Monitoring in KloudMate helps you observe the health, performance, and behavior of your Microsoft SQL Server instances by collecting metrics and logs using the KloudMate Agent powered by OpenTelemetry. This enables centralized monitoring of MSSQL instances running on Windows hosts, AWS EC2, Azure Virtual Machines, or on-premise servers. +This guide sets up Microsoft SQL Server (MSSQL) monitoring with the KloudMate Agent and OpenTelemetry. It collects metrics and logs so you can watch the health, performance, and query behavior of MSSQL instances running on Windows hosts, AWS EC2, Azure Virtual Machines, or on-premise servers. # MSSQL Integration using KloudMate Agents diff --git a/src/content/docs/docs/database-monitoring/database-integrations/mysql-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/mysql-monitoring.mdx index 61d202dc..6fce644e 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/mysql-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/mysql-monitoring.mdx @@ -1,12 +1,10 @@ --- -title: "MYSQL" -description: "Documentation for MYSQL Monitoring" +title: "MySQL" +description: "Monitor MySQL health, performance, and logs with the KloudMate Agent and OpenTelemetry." --- ![image](./images/mysql-monitoring-1.png) -The MySQL integration delivers near real-time health and performance metrics for your MySQL databases. These metrics are visualized using a built-in dashboard, and you can configure alerts to notify your team about important MySQL conditions. - -MySQL Monitoring in KloudMate helps you observe the health, performance, and behavior of your MySQL databases by collecting metrics and logs using the **KloudMate Agent** powered by OpenTelemetry. This enables centralized monitoring of MySQL instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. +This guide sets up MySQL monitoring with the **KloudMate Agent** and OpenTelemetry. It collects metrics and logs so you can watch the health, performance, and query behavior of MySQL instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. A built-in dashboard visualizes the metrics, and you can set alerts on key MySQL conditions. # MySQL Integration using KloudMate Agents diff --git a/src/content/docs/docs/database-monitoring/database-integrations/oracledb-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/oracledb-monitoring.mdx index 0bf72fe5..e0ceab27 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/oracledb-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/oracledb-monitoring.mdx @@ -1,11 +1,9 @@ --- title: "OracleDB" -description: "Documentation for OracleDB Monitoring" +description: "Monitor Oracle Database sessions, performance, tablespaces, and logs with the KloudMate Agent and OpenTelemetry." --- -Oracle Database (OracleDB) is foundational for many enterprise applications, and robust monitoring is essential for maintaining performance and reliability. KloudMate provides comprehensive visibility into OracleDB by delivering real-time insights through logs and metrics using the KloudMate Agent powered by OpenTelemetry. - -OracleDB Monitoring in KloudMate enables centralized monitoring of Oracle Database instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. +This guide sets up Oracle Database (OracleDB) monitoring with the KloudMate Agent and OpenTelemetry. It collects logs and metrics so you can watch sessions, system statistics, tablespace usage, and performance across OracleDB instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. # OracleDB Integration using KloudMate Agents @@ -63,7 +61,7 @@ Two named receiver instances are used to logically separate SQL performance quer - Log in to the KloudMate platform - Go to **Settings → Agents** -- Select the agent running on the PostgreSQL host +- Select the agent running on the OracleDB host - Click **Actions → Collector Configuration** - YAML editor opens for configuration diff --git a/src/content/docs/docs/database-monitoring/database-integrations/postgresql-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/postgresql-monitoring.mdx index 03961180..58766ff0 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/postgresql-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/postgresql-monitoring.mdx @@ -1,12 +1,10 @@ --- title: "PostgreSQL" -description: "Documentation for PostgreSQL Monitoring" +description: "Monitor PostgreSQL health, performance, queries, and logs with the KloudMate Agent and OpenTelemetry." --- ![image](./images/postgresql-monitoring-1.png) -PostgreSQL is a powerful open-source relational database known for reliability, extensibility, and strong compliance with SQL standards. It is widely used for transactional applications, analytics workloads, and enterprise systems that require data integrity and performance. - -PostgreSQL Monitoring in KloudMate helps you observe the health, performance, and behavior of your PostgreSQL databases by collecting metrics and logs using the **KloudMate Agent** powered by OpenTelemetry. This enables centralized monitoring of PostgreSQL instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. +This guide sets up PostgreSQL monitoring with the **KloudMate Agent** and OpenTelemetry. It collects metrics and logs so you can watch the health, performance, and query behavior of PostgreSQL instances running on AWS EC2, Azure Virtual Machines, or on-premise servers. # PostgreSQL Integration using KloudMate Agents diff --git a/src/content/docs/docs/database-monitoring/database-integrations/redis-monitoring.mdx b/src/content/docs/docs/database-monitoring/database-integrations/redis-monitoring.mdx index 47c47636..f544845b 100644 --- a/src/content/docs/docs/database-monitoring/database-integrations/redis-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/database-integrations/redis-monitoring.mdx @@ -1,10 +1,8 @@ --- title: "Redis" -description: "Documentation for Redis Monitoring" +description: "Monitor Redis health and performance with the KloudMate Agent and OpenTelemetry." --- -Redis is an in-memory data structure store commonly used as a database, cache, and message broker. It is designed for high performance and low latency, making it suitable for real-time applications such as caching, session management, analytics, messaging queues, and leaderboards. Redis supports multiple data structures and offers features such as persistence, replication, and clustering for scalability and high availability. - -Redis Monitoring in KloudMate helps you observe the health, performance, and behavior of your Redis servers by collecting metrics using the **KloudMate Agent** powered by OpenTelemetry. This enables centralized monitoring of Redis instances running on cloud virtual machines or on-premise servers. +This guide sets up Redis monitoring with the **KloudMate Agent** and OpenTelemetry. It collects metrics so you can watch the health and performance of Redis instances running on cloud virtual machines or on-premise servers. ### What This Integration Provides diff --git a/src/content/docs/docs/database-monitoring/direct-database-monitoring.mdx b/src/content/docs/docs/database-monitoring/direct-database-monitoring.mdx index 295c072b..9d2c5a5b 100644 --- a/src/content/docs/docs/database-monitoring/direct-database-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/direct-database-monitoring.mdx @@ -1,16 +1,16 @@ --- title: "Direct Database Monitoring" -description: "Documentation for Direct Database Monitoring" +description: "Connect the KloudMate Agent directly to a database with a read-only user for query metrics, execution plans, and schema metadata." sidebar: order: 4 --- -The KloudMate Agent can connect directly to your database using a dedicated monitoring user to collect deep query metrics, execution plans, and schema metadata. This approach provides the richest level of database insight — including per-query latency breakdowns, explain plans, and wait event analysis — across both self-hosted and managed database services. +The KloudMate Agent can connect directly to your database using a dedicated monitoring user to collect deep query metrics, execution plans, and schema metadata. It collects the most detailed query-level data, including per-query latency breakdowns, explain plans, and wait event analysis, across self-hosted and managed database services. :::note When to Use - Use Direct Database Monitoring when you need deep query-level analytics, including execution plans, query digest metrics, wait event analysis, and schema discovery. This is the most comprehensive monitoring approach and works on any environment — self-hosted, RDS, Aurora, Cloud SQL, Azure, or AlloyDB. + Use Direct Database Monitoring when you need deep query-level analytics, including execution plans, query digest metrics, wait event analysis, and schema discovery. This is the most complete monitoring approach and works on any environment: self-hosted, RDS, Aurora, Cloud SQL, Azure, or AlloyDB. ::: ## How It Works diff --git a/src/content/docs/docs/database-monitoring/index.mdx b/src/content/docs/docs/database-monitoring/index.mdx index 926013fb..c2728d8a 100644 --- a/src/content/docs/docs/database-monitoring/index.mdx +++ b/src/content/docs/docs/database-monitoring/index.mdx @@ -8,6 +8,10 @@ sidebar: The KloudMate Agent provides three approaches to database monitoring, from zero-configuration kernel-level profiling to deep SQL-level query analytics, so you can choose the right level of visibility for your environment. +:::tip +To collect server-side database metrics and logs, the KloudMate Agent can wire up database receivers for you through a guided wizard. See [Agent database monitoring](../kloudmate-agent/database-monitoring/overview/). +::: + :::note Which approach should I use? diff --git a/src/content/docs/docs/database-monitoring/opentelemetry-database-monitoring.mdx b/src/content/docs/docs/database-monitoring/opentelemetry-database-monitoring.mdx index 052d639b..7f9c20ed 100644 --- a/src/content/docs/docs/database-monitoring/opentelemetry-database-monitoring.mdx +++ b/src/content/docs/docs/database-monitoring/opentelemetry-database-monitoring.mdx @@ -1,6 +1,6 @@ --- title: "OpenTelemetry Database Monitoring" -description: "Documentation for OpenTelemetry Database Monitoring" +description: "Capture database query spans with OpenTelemetry SDK auto-instrumentation and send them to KloudMate." sidebar: order: 3 --- @@ -43,7 +43,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318" OTEL_SERVICE_NAME="your-service-name" ``` -Database spans will automatically be captured when using supported auto-instrumentation libraries. +Database spans are captured automatically when you use supported auto-instrumentation libraries. :::note For zero-configuration kernel-level profiling, see [Database Access Monitoring (DAM)](../database-activity-monitoring-dam/). For deep query analytics with execution plans and schema discovery, see [Direct Database Monitoring](../direct-database-monitoring/). diff --git a/src/content/docs/docs/getting-started/core-concepts.mdx b/src/content/docs/docs/getting-started/core-concepts.mdx index b7c1526b..593aa91c 100644 --- a/src/content/docs/docs/getting-started/core-concepts.mdx +++ b/src/content/docs/docs/getting-started/core-concepts.mdx @@ -5,7 +5,7 @@ sidebar: order: 2 --- -KloudMate relies on industry-standard observability frameworks to seamlessly collect and analyze data from your infrastructure and applications. Before integrating your systems, it helps to understand three fundamental concepts: **Signals** , **OpenTelemetry** , and **eBPF**. +KloudMate collects and analyzes data from your infrastructure and applications through open observability standards. Before you integrate your systems, it helps to understand three concepts: **Signals**, **OpenTelemetry**, and **eBPF**. This 2-minute primer provides the essential context you need to get started. @@ -23,7 +23,7 @@ Observability is built upon three primary pillars of data, often referred to as KloudMate uses OTel as its primary data integration layer. This offers several advantages: - **No Vendor Lock-in:** You instrument your code once using OTel standards. You don't need proprietary KloudMate code in your applications. -- **Standardized Data:** OTel ensures that the data sent to KloudMate is uniformly structured, making correlation and analysis much more powerful. +- **Standardized Data:** OTel keeps the data sent to KloudMate uniformly structured, so correlation and analysis across sources stay consistent. - **The Collector:** OTel provides a "Collector" component that can receive, process, and export data. When you set up KloudMate integrations, you are often configuring an OTel Collector to forward data to the KloudMate platform. *For more details, see our dedicated [OpenTelemetry Guide](../../opentelemetry/).* @@ -36,10 +36,10 @@ A **service-level objective (SLO)** turns the question *"is this service reliabl [eBPF](https://ebpf.io/) is a technology that allows programs to run directly within the operating system kernel safely and efficiently, without requiring kernel source code changes or loading kernel modules. -In the context of observability and the KloudMate Agent (KloudMate Agent): -- **Zero-Code Instrumentation:** eBPF allows the agent to observe network traffic, application performance, and system calls automatically. You do not need to modify your application code to get deep insights. -- **Low Overhead:** Because eBPF runs in the kernel, it can collect high-fidelity data (like detailed network metrics or process executions) with extremely low performance overhead. -- **Immediate Visibility:** By deploying the KloudMate Agent (which leverages eBPF), you gain instant visibility into your infrastructure without manual configuration. +For the KloudMate Agent, eBPF provides: +- **Zero-Code Instrumentation:** the agent observes network traffic, application performance, and system calls automatically, with no changes to your application code. +- **Low Overhead:** because eBPF runs in the kernel, it collects high-fidelity data, such as detailed network metrics or process executions, with low performance overhead. +- **Immediate Visibility:** deploying the KloudMate Agent, which uses eBPF, gives you visibility into your infrastructure without manual configuration. ## Next Steps diff --git a/src/content/docs/docs/getting-started/get-help.mdx b/src/content/docs/docs/getting-started/get-help.mdx index 028ef060..d9567f13 100644 --- a/src/content/docs/docs/getting-started/get-help.mdx +++ b/src/content/docs/docs/getting-started/get-help.mdx @@ -1,14 +1,12 @@ ---- -title: "Get Help" -description: "Find support channels, community links, and ways to contact the KloudMate team." -sidebar: - order: 5 --- -**For questions, feedback, bug reports, or feature requests, we are always available.** Our growing community and support teams are available **round-the-clock**, to help you with anything and everything. You can reach our support teams by: - -- Using the live chat widget in the bottom right corner of [the website](https://www.kloudmate.com) -- Writing to us at [hello@kloudmate.com](mailto\:hello@kloudmate.com) +title: "Get Help" +description: "Find support channels, community links, and ways to contact the KloudMate team." +sidebar: + order: 5 +--- -We respond to queries via email and have a [Slack Community](https://join.slack.com/t/kloudmate/shared_invite/zt-zvzq985i-Khwnq2jsm5uLTdBe87gCgw) of developers and customers who interact, share experiences, and support each other. The community offers faster responses to issues or queries that you might have. +Have a question, a bug to report, or a feature request? Here is how to reach the KloudMate team and community. -[Join us on Slack](https://join.slack.com/t/kloudmate/shared_invite/zt-zvzq985i-Khwnq2jsm5uLTdBe87gCgw) +- **Live chat.** Use the chat widget in the bottom-right corner of [the website](https://www.kloudmate.com). +- **Email.** Write to [hello@kloudmate.com](mailto:hello@kloudmate.com), and the support team replies by email. +- **Slack community.** Join the [KloudMate Slack](https://join.slack.com/t/kloudmate/shared_invite/zt-zvzq985i-Khwnq2jsm5uLTdBe87gCgw) to ask questions and compare notes with other developers and customers. It is usually the fastest way to get an answer. diff --git a/src/content/docs/docs/getting-started/sending-data-to-kloudmate.mdx b/src/content/docs/docs/getting-started/sending-data-to-kloudmate.mdx index d206233d..a43ac258 100644 --- a/src/content/docs/docs/getting-started/sending-data-to-kloudmate.mdx +++ b/src/content/docs/docs/getting-started/sending-data-to-kloudmate.mdx @@ -6,9 +6,9 @@ sidebar: --- import { LinkCard } from 'starlight-theme-nova/components' -KloudMate provides multiple, flexible methods for ingesting your observability data. Because KloudMate is built natively on OpenTelemetry standards, you are never locked into a proprietary collection method. +KloudMate can ingest your telemetry several ways. Because it is built on OpenTelemetry, you are not locked into a proprietary collection method. -Choose the ingestion path below that best matches your existing architecture, your environment (cloud vs. on-premise), and how much control you want over the telemetry collection process. +Choose the ingestion path below that best matches your architecture, your environment (cloud or on-premise), and how much control you want over telemetry collection. ## Recommended Starting Points @@ -63,9 +63,9 @@ If you are unsure where to begin, use this table to map your immediate goal to t ## Common Rollout Pattern -For teams starting from scratch, the following phased approach is recommended: +For a team starting from scratch, a phased rollout works well: -1. **Infrastructure First:** Start with the [KloudMate Agent](../../kloudmate-agent/) to get baseline telemetry, host metrics, and zero-code visibility. +1. **Infrastructure First:** Start with the [KloudMate Agent](../../kloudmate-agent/) to get host metrics, logs, and zero-code visibility. 2. **Application Tracing:** Add [OpenTelemetry](../../opentelemetry/) instrumentation to your applications where you need richer distributed tracing or custom metrics. 3. **Cloud Integrations:** Layer in cloud-specific integrations for [AWS](../../aws-integration/), [Azure](../../azure-integration/), or [Prometheus](../../infrastructure/integrations/prometheus-integration/) to centralize your remaining data sources. diff --git a/src/content/docs/docs/getting-started/what-is-kloudmate.mdx b/src/content/docs/docs/getting-started/what-is-kloudmate.mdx index e221f526..b1ac2eec 100644 --- a/src/content/docs/docs/getting-started/what-is-kloudmate.mdx +++ b/src/content/docs/docs/getting-started/what-is-kloudmate.mdx @@ -4,26 +4,21 @@ description: "Learn what KloudMate is and how it brings logs, metrics, traces, a sidebar: order: 1 --- -[KloudMate](https://www.kloudmate.com) is a comprehensive **observability platform** that helps developers identify and troubleshoot issues and optimize application performance across distributed environments. -KloudMate offers **end-to-end visibility** into application infrastructures, distributed systems, and development environments. It includes **OpenTelemetry integration,** a real-time alerting system, powerful dashboards, and more — all designed to provide comprehensive observability and monitoring. **KloudMate offers a Single-Source-of-Truth approach** for observability by enabling developers to gather and consolidate telemetry data from multiple distributed systems using **OpenTelemetry.** With KloudMate, you can diagnose your application infrastructure, pinpoint and troubleshoot errors, and drill down to associated logs, metrics, or traces. +[KloudMate](https://www.kloudmate.com) is an observability platform for finding and fixing problems across distributed systems. You collect logs, metrics, and traces from your applications and infrastructure in one place, then follow a single request across those signals to diagnose an issue instead of switching between tools. -And to keep things neat and organized, KloudMate lets you create multiple **Workspaces**. You can use workspaces to group multiple teams, accounts, or orgs. For example, you can create separate workspaces for development, production, etc. +It builds on [OpenTelemetry](https://opentelemetry.io/), so you instrument once with an open standard instead of a proprietary agent. On top of that data you get dashboards, real-time alerting, and correlation across signals, so a spike in one chart links to the logs and traces behind it. -### Observability +## Workspaces -Observability is the practice of using tools to aggregate, correlate, and analyze streams of data from applications and the underlying infrastructure. +A **workspace** groups related environments, teams, or accounts and keeps their data and access separate. A common setup is one workspace each for development, staging, and production. Add or rename workspaces later under **Settings → Workspaces**. -Observability essentially depends upon the ability to log, monitor, and trace different servers, applications, systems, and data processes across the lifecycle of the environment. +## What observability means here -KloudMate's observability tools give you visibility into distributed application resources, faster insights, and automated problem identification and resolution. +Observability is the ability to understand what a system is doing from the signals it emits, by collecting and correlating logs, metrics, and traces across every service and host. When something breaks, that correlated view is what lets you find the cause instead of guessing at it. -The pages in this section will help you get started with KloudMate. +## Next steps -Related Resources - -- [Setting Up KloudMate](../setting-up-kloudmate/) -- [Sending Data to KloudMate](../sending-data-to-kloudmate/) -- [Get Help](../get-help/) - -# +- [Setting up KloudMate](../setting-up-kloudmate/) — sign up and send your first event. +- [Sending data to KloudMate](../sending-data-to-kloudmate/) — pick an ingestion path. +- [Get help](../get-help/) — support and community. diff --git a/src/content/docs/docs/infrastructure/hosts.mdx b/src/content/docs/docs/infrastructure/hosts.mdx index 52c18666..34484fb5 100644 --- a/src/content/docs/docs/infrastructure/hosts.mdx +++ b/src/content/docs/docs/infrastructure/hosts.mdx @@ -8,7 +8,7 @@ sidebar: The **Hosts** module provides a centralized view for monitoring the health, performance, and resource utilization of your infrastructure. This view automatically populates with data once the KloudMate Agent is installed on your Linux or Windows servers. -From this view, you can seamlessly navigate from a fleet-wide overview down to the metrics and logs of an individual host or its associated containers. +From this view, you can navigate from a fleet-wide overview down to the metrics and logs of an individual host or its associated containers. ## Overview @@ -61,14 +61,14 @@ Provides immediate access to the logs generated by the selected resource. - You can search, filter, and analyze logs specific to that host or container directly within the drawer. ### Containers Tab (Hosts Only) -When inspecting a Host, this tab lists all the active containers running on that specific machine, displaying their individual CPU, Memory, Image, and Runtime metrics. Clicking a container in this list will update the drawer to show the details for that specific container. +When inspecting a Host, this tab lists all the active containers running on that specific machine, displaying their individual CPU, Memory, Image, and Runtime metrics. Clicking a container in this list updates the drawer to show the details for that container. ## Getting Started -Host metrics are collected **automatically** with zero configuration required once you deploy the agent. +Host metrics are collected **automatically**, with no configuration, once you deploy the agent. 1. Install the [KloudMate Agent](../../kloudmate-agent/) on your servers. -2. The agent will begin forwarding core metrics (CPU, Memory, Filesystem, Uptime) and system logs to KloudMate. -3. Your servers will automatically appear in the Hosts view within a few minutes. +2. The agent forwards core metrics (CPU, Memory, Filesystem, Uptime) and system logs to KloudMate. +3. Your servers appear in the Hosts view within a few minutes. For instructions on adding custom cloud metadata (like AWS EC2 or Azure tags) or capturing advanced process-level metrics, see the [Advanced Host Metrics Collection](../server-metrics-to-kloudmate/) guide. diff --git a/src/content/docs/docs/infrastructure/index.mdx b/src/content/docs/docs/infrastructure/index.mdx index b6eafeb8..70252f43 100644 --- a/src/content/docs/docs/infrastructure/index.mdx +++ b/src/content/docs/docs/infrastructure/index.mdx @@ -9,6 +9,10 @@ Infrastructure monitoring in KloudMate gives you visibility into the health, per This section covers common infrastructure ingestion paths, whether you are monitoring a single server, a Kubernetes cluster, a Prometheus deployment, or a virtualized environment. +:::tip +The fastest way to start collecting host and Kubernetes metrics is the KloudMate Agent, which sends them automatically after install. See [Install the agent](../kloudmate-agent/) and [Host metrics and logs](../kloudmate-agent/baseline/host-metrics-and-logs/). +::: + ## What You Can Monitor - Hosts and virtual machines diff --git a/src/content/docs/docs/infrastructure/server-metrics-to-kloudmate.mdx b/src/content/docs/docs/infrastructure/server-metrics-to-kloudmate.mdx index 6d909eab..4d6c3359 100644 --- a/src/content/docs/docs/infrastructure/server-metrics-to-kloudmate.mdx +++ b/src/content/docs/docs/infrastructure/server-metrics-to-kloudmate.mdx @@ -5,7 +5,7 @@ sidebar: label: "Advanced Host Configuration" order: 4 --- -The KloudMate Agent automatically collects essential host metrics (CPU, Memory, Filesystem, Uptime) from Linux and Windows systems out-of-the-box using the OpenTelemetry `hostmetrics` receiver. You can view these metrics immediately in the [Hosts View](../hosts/). +The KloudMate Agent automatically collects essential host metrics (CPU, Memory, Filesystem, Uptime) from Linux and Windows systems using the OpenTelemetry `hostmetrics` receiver. You can view these metrics immediately in the [Hosts View](../hosts/). If you need to collect extended metrics (like detailed process-level data, disk I/O, or network metrics) or attach cloud provider tags, you can modify the agent's configuration following this guide. diff --git a/src/content/docs/docs/kloudmate-agent/advanced-configuration.mdx b/src/content/docs/docs/kloudmate-agent/advanced-configuration.mdx new file mode 100644 index 00000000..c95b4cee --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/advanced-configuration.mdx @@ -0,0 +1,46 @@ +--- +title: "Advanced configuration" +description: "Manual YAML, custom receivers, sampling, overhead tuning, autonomous instrumentation, self-telemetry, and credential rotation for the KloudMate agent." +sidebar: + order: 13 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Most agents run well in managed mode with the defaults. This page collects the advanced controls for cases that need hand-tuning. For the everyday model, see the [configuration model](../concepts/config-model/). + +## Manual YAML and custom receivers + +Switching an agent to **manual mode** gives you the raw collector YAML in the editor. Use it to hand-tune the configuration or add a component the managed integrations do not cover. This is for non-standard setups. The trade-off: you edit the YAML by hand, and the feature toggles are disabled. On Kubernetes, the agent's configuration lives in the collectors' ConfigMaps. Do not edit those directly, because the agent controls them in managed mode. See the [configuration model](../concepts/config-model/) and [Kubernetes platform notes](../platform-notes/kubernetes/). + +If you only need to add one component on top of the managed configuration, the [custom config override](../custom-config-override/) is lighter. It layers your YAML on top of managed mode and keeps the feature toggles working, so you do not have to hand-edit the whole configuration. + +The agent's own settings (such as the API key, host name override, and health reporting) are set through `agent.yaml`, environment variables, or command-line flags on the host, and through Helm values on Kubernetes. See the [configuration reference](../reference/agent-config/) for every setting, its environment variable, and its default. + +## Sampling + +Set a global trace sampling rate and override it per service to balance trace fidelity against ingestion cost. See [Sampling](../auto-instrumentation/sampling/). + +## eBPF and overhead tuning + +eBPF monitoring runs within a CPU and memory budget and reduces its overhead automatically on small instances. You can tune context propagation, request-header capture, database detection, and buffer sizes further. See [eBPF observability](../ebpf-observability/). + +## Collector self-telemetry + +Turn on the collector's in-band `otelcol_*` self-telemetry when you want detail on exporter queues and send failures. It is off by default because it adds metric volume. See [Collector self-telemetry](../self-observability/self-telemetry/). + +## Autonomous instrumentation + +The default model is to instrument per service. For automated rollouts, the agent supports an **autonomous mode** (`auto-instrument: true`) that instruments every eligible service on a host without per-service toggles, and auto-wires reachable web servers. Use it when you want blanket coverage and are willing to give up the per-service opt-in. See [Application APM overview](../auto-instrumentation/overview/). + +## Credential rotation + +By default, the agent stores only a reference to the environment variable, not the password itself. To rotate a database password, update the environment variable or Kubernetes Secret. You do not need to change the agent's configuration. See [Database credentials](../database-monitoring/credentials/). + +## Related + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/_meta.json b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/_meta.json new file mode 100644 index 00000000..18fd0551 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/_meta.json @@ -0,0 +1,5 @@ +{ + "label": "Application APM", + "order": 6, + "collapsed": true +} diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/go-ebpf.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/go-ebpf.mdx new file mode 100644 index 00000000..8b45b201 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/go-ebpf.mdx @@ -0,0 +1,39 @@ +--- +title: "Go (eBPF)" +description: "Why the KloudMate agent covers Go services through eBPF uprobes instead of code injection, and how Go traces connect to instrumented neighbors." +sidebar: + order: 6 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Go is the one runtime the agent does not inject. It compiles to a static binary with no runtime to attach an agent to. So instead of code injection, the agent covers Go services through extended Berkeley Packet Filter (eBPF) monitoring. There is no **Instrument** toggle to turn on for a Go service. It is covered automatically wherever eBPF monitoring is active: on **Linux, Kubernetes, and Docker**, but not on **Windows**, which has no eBPF. + +## Rich eBPF coverage for Go + +Go gets more from eBPF than other runtimes do. The agent's eBPF tracer follows Go programs closely enough to produce traces across many protocols: + +- HTTP +- gRPC +- Kafka +- Redis +- SQL +- MongoDB + +The result is close to what a language SDK would produce, without changing your code or your build. + +## How Go traces connect to other services + +eBPF monitoring and the language agents both propagate W3C trace context (`traceparent`). So when an instrumented service calls a Go service, the Go service's eBPF span joins the caller's trace. You see one connected trace with the correct parent-child links, not two separate views. + +:::note +In the Discovered Services list, a Go service is marked as covered by eBPF rather than as one you can instrument, and its Instrument toggle is disabled. This is expected: eBPF monitoring is already tracing it. +::: + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/images/kubernetes-apm-list.png b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/images/kubernetes-apm-list.png new file mode 100644 index 00000000..8dd41ac4 Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/images/kubernetes-apm-list.png differ diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/kubernetes.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/kubernetes.mdx new file mode 100644 index 00000000..a4968ecb --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/kubernetes.mdx @@ -0,0 +1,84 @@ +--- +title: "Kubernetes" +description: "How the KloudMate agent traces Kubernetes workloads one at a time — Off, eBPF, or SDK — with the OpenTelemetry Operator handling SDK injection." +sidebar: + order: 3 +--- + +import { Tabs, TabItem, LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On Kubernetes, application performance monitoring (APM) works the same opt-in, per-workload way as on other platforms: for each workload you choose **Off**, **eBPF**, or **SDK**. eBPF tracing runs in the DaemonSet collector and needs no code change and no restart. SDK tracing uses the OpenTelemetry (OTel) Operator — the agent adds an annotation, the Operator injects the matching language agent, and that workload's pods restart with tracing on. Off is the default, so nothing is traced until you turn it on. For the shared flow, consent, and status behavior, see the [Application APM overview](../overview/). + +:::note[Which modes a workload offers] +Workloads with a supported SDK language — Java, Node.js, Python, or .NET — offer all three modes. Go, Ruby, and unrecognized runtimes such as Hasura have no SDK injector, so they offer **Off** or **eBPF** only. eBPF traces those at the kernel level with no injector and no restart, which is the only way to get traces from a workload no language agent can attach to. +::: + +## Prerequisites + +Auto-instrumentation on Kubernetes uses the OpenTelemetry Operator, which the KloudMate agent installs. Enable APM at install time, or turn it on afterward. See the [Kubernetes installation guide](../../installation/kubernetes-agent/), including the prerequisite Cert Manager and the `featuresEnabled.apm` value. + +## Enable APM + +There are two ways to instrument workloads. + +### From the dashboard + +After the agent is installed, open the cluster's APM configuration. The agent lists the workloads it discovered, grouped by namespace, with each workload's detected language. For each one, choose **Off**, **eBPF**, or **SDK**, then apply. In a large cluster, use the namespace filter or the search box to find a workload. + +![The Kubernetes APM list grouped by namespace, with per-workload Off / eBPF / SDK controls. SDK-capable workloads show all three modes; Go and unrecognized runtimes such as Hasura show only Off and eBPF.](./images/kubernetes-apm-list.png) + +### With deployment annotations + +If a workload isn't detected automatically, you can turn on **SDK** tracing by hand: add the OpenTelemetry Operator annotation for its language and restart the workload. This is the same annotation the SDK control applies. eBPF tracing has no annotation — the DaemonSet drives it, so you select it from the dashboard. + + + + +```bash +kubectl patch deployment -n -p '{"spec":{"template":{"metadata":{"annotations":{"instrumentation.opentelemetry.io/inject-java":"km-agent/km-agent-instrumentation-crd"}}}}}' +``` + + + + +```bash +kubectl patch deployment -n -p '{"spec":{"template":{"metadata":{"annotations":{"instrumentation.opentelemetry.io/inject-nodejs":"km-agent/km-agent-instrumentation-crd"}}}}}' +``` + + + + +```bash +kubectl patch deployment -n -p '{"spec":{"template":{"metadata":{"annotations":{"instrumentation.opentelemetry.io/inject-python":"km-agent/km-agent-instrumentation-crd"}}}}}' +``` + + + + +```bash +kubectl patch deployment -n -p '{"spec":{"template":{"metadata":{"annotations":{"instrumentation.opentelemetry.io/inject-dotnet":"km-agent/km-agent-instrumentation-crd"}}}}}' +``` + + + + +After adding the annotation, restart the deployment so the Operator injects the agent into new pods. + +## How each mode applies + +The opt-in flow is the same as on [Linux](../linux-systemd/): discover a workload, pick Off, eBPF, or SDK, and let traces flow. What differs is how Kubernetes applies each mode: + +- **eBPF** changes take effect on the next agent check-in with **no pod restart**. The DaemonSet reloads its configuration in place and starts tracing the workloads you selected, so switching a workload on or off doesn't roll any pods. +- **SDK** changes go through the OTel Operator's annotation, so **only that workload's pods** roll to pick up the injected agent. Linux, by contrast, applies SDK through systemd config files and a service restart. + +## Monitored namespaces + +APM discovery applies to the namespaces you monitor, which you set at install time with `monitoredNamespaces`. Leave it empty to discover workloads in every namespace except the system ones: `kube-system`, `kube-public`, `kube-node-lease`, `cert-manager`, and anything starting with `kube-` are excluded by default. To also skip your own cluster-infrastructure namespaces — ingress controllers, `vpa`, `external-secrets`, and the like — list them in the `excludedNamespaces` Helm value. See the [Kubernetes installation guide](../../installation/kubernetes-agent/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/language-support.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/language-support.mdx new file mode 100644 index 00000000..7e60a8c1 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/language-support.mdx @@ -0,0 +1,139 @@ +--- +title: "Languages and process managers" +description: "Which languages the KloudMate agent auto-instruments on Linux, how the method changes with systemd, PM2, or a standalone process, and the known limitations." +sidebar: + label: "Languages & process managers" + order: 2.5 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On a Linux host, how the agent adds tracing to a service depends on two things: the language it is written in, and how it is started (as a systemd service, under PM2, or on its own). The agent picks the right method for each combination. This page lists what is supported and what to watch out for. + +Every method follows the same steps to turn tracing on, ask before it restarts anything, and roll back if needed. See the [Application APM overview](../overview/). + +## Support matrix + +| Language | systemd service | PM2 app | Standalone process | +|---|---|---|---| +| Java | Startup config, then restart | Attaches live, no restart | Attaches live, no restart | +| Node.js | Startup config, then restart | Restart with tracing on | Run under systemd or PM2 | +| Python | Startup config, then restart | Restart with tracing on | Run under systemd or PM2 | +| .NET | Startup config, then restart | Run under systemd | Run under systemd | +| PHP | PHP config + graceful reload | PHP config + graceful reload | PHP config + graceful reload | +| Go | eBPF, no code change | eBPF, no code change | eBPF, no code change | +| Ruby | eBPF, no code change | eBPF, no code change | eBPF, no code change | + +With any method, the traces go to the agent's local collector, so an instrumented service shares a single trace with the [eBPF monitoring](../../ebpf-observability/) around it. + +## systemd services + +This is the main method on Linux. When you instrument a systemd service, the agent adds a small startup config file for it and, after you confirm, restarts the service so the change takes effect: + +- **Java** loads the OpenTelemetry Java agent through `JAVA_TOOL_OPTIONS`. +- **Node.js** loads the instrumentation through `NODE_OPTIONS`. +- **Python** loads it through `PYTHONPATH`. +- **.NET** loads the CLR profiler through the `CORECLR_*` and `DOTNET_STARTUP_HOOKS` variables. + +Because the service restarts, the agent can set everything up front: a fixed `service.name` (the name shown in the Discovered Services list), the sampling rate, and where to send the traces. Turning instrumentation off removes the config file and restarts the service, so it goes back to how it was. Running the same action twice changes nothing, so it is safe to repeat. + +## PM2 apps + +PM2 apps are not systemd services, so the agent uses a different method. For **Node.js and Python** apps under PM2, it restarts each selected app with tracing switched on (`pm2 restart --update-env`). Each app keeps its PM2 process name as its `service.name`, so apps under the same PM2 manager stay separate. + +The agent covers PM2 for every user on the host, not just root. If an app does not come back up after the change, the agent undoes it and restarts the app, so a failed attempt never leaves your app down. + +Java under PM2 uses the runtime attach method described below. .NET under PM2 is uncommon and is not handled automatically; run a .NET service under systemd to instrument it. + +## Java without systemd (runtime attach) + +A Java process that is not a systemd service can still be instrumented, whether it runs as a plain `java -jar` process, under PM2, or from a script. The agent attaches the OpenTelemetry Java agent to the **already-running** process, with no restart needed. This is tested against common frameworks, including Spring Boot and Tomcat. + +The agent sets `service.name` automatically, trying these in order: + +1. The jar name for a `java -jar app.jar` launch (so `app.jar` reports as `app`). +2. `spring.application.name`, when the app sets it. +3. Otherwise `unknown_service:java`. + +Attaching to a running process has two consequences: + +- **You cannot force a specific `service.name`.** If you need a fixed name, set `spring.application.name` in the app, or run it as a systemd service. +- **It cannot be undone while the app runs.** Once attached, the agent stays until the process restarts. Turning instrumentation off does not remove it from a running process; it takes effect the next time the process restarts. + +## Standalone Node.js and Python + +The agent can instrument Node.js and Python only when they run under systemd or PM2. A process started on its own, under neither, cannot be instrumented automatically. Run it under systemd or PM2 first. The Discovered Services list flags these processes and tells you to do this. + +## Python versions + +SDK injection supports **Python 3.10 through 3.13**. Other versions, such as 3.14, are covered by [eBPF monitoring](../../ebpf-observability/) instead. + +## PHP + +PHP is instrumented through its own configuration, not a restart. The agent adds a config file for the target's PHP build, then reloads PHP gracefully (a php-fpm or Apache reload) so no requests are dropped. It uses the native OpenTelemetry extension on **PHP 8**, and falls back to a tracer that also covers **PHP 7**. See [PHP instrumentation](../php/). + +## Go + +Go does not use code injection. It is covered by [eBPF monitoring](../go-ebpf/), which gives Go services RED metrics (request rate, errors, and duration), the service map, and traces from the kernel, with no change to the app. + +## Ruby + +A Ruby app has two levels of coverage: + +- **eBPF monitoring, with no code change.** [eBPF](../../ebpf-observability/) gives a Ruby service HTTP server traces, RED metrics (request rate, errors, and duration), and the service map from the kernel, with no gem and on any Ruby version. This is the same as Go. +- **Deep Ruby tracing** needs the OpenTelemetry Ruby gems added to the app's `Gemfile`. The agent cannot add them from outside, because Ruby only loads gems listed in the app's `Gemfile`. With them, `use_all` traces the whole request path — HTTP handling, database queries, view rendering, background jobs, and outbound HTTP calls. + + ```ruby + # Gemfile + gem 'opentelemetry-sdk' + gem 'opentelemetry-instrumentation-all' + gem 'opentelemetry-exporter-otlp' + ``` + + ```ruby + # config/initializers/opentelemetry.rb + require 'opentelemetry/sdk' + require 'opentelemetry/instrumentation/all' + require 'opentelemetry/exporter/otlp' + OpenTelemetry::SDK.configure { |c| c.use_all } + ``` + + Set the service name and where to send traces through the app's environment: + + ```text + OTEL_SERVICE_NAME=my-rails-app + OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 + ``` + + The SQL query text on a database span comes from the driver instrumentation (`pg`, `mysql2`), not from ActiveRecord. A database with no OpenTelemetry driver — SQLite — shows the model operation (`User#save`) but no SQL. For the full breakdown of what these gems capture, see [Ruby auto-instrumentation](../../../apm-and-tracing/auto-instrumentation/ruby/). + + The current OpenTelemetry Ruby gems need **Ruby 3.3 or newer**. On Ruby 3.0 to 3.2 you have to pin older gem versions, and on Ruby 2.x they will not install at all. On those versions, use eBPF monitoring, which does not depend on the Ruby version. + +## How discovered services are named and filtered + +The agent keeps the Discovered Services list focused on real applications: + +- **Infrastructure and the agent's own processes are left out.** The agent never lists or instruments itself, cloud-provider agents (such as the AWS SSM agent, the Google guest agent, or the Azure agent), or container runtimes (such as `containerd`, `dockerd`, and `kubelet`). Left in, these would clutter the list and could be traced as if they were your services. +- **A plain Java process is listed by its jar name.** A `java -jar app.jar` process shows as `app` (matching its `service.name`), not the generic `java`. + +For the full discovery model, see [Discovery](../../concepts/discovery/). + +## Limitations + +Keep these in mind, especially for Java runtime attach: + +- **Java runtime attach does not cover the JDK's built-in `com.sun.net.httpserver`.** That server sets up its handler before the agent attaches, and it cannot be changed afterward. Common frameworks like Spring Boot and Tomcat are fully covered; only apps that serve directly from the JDK's built-in server are affected. +- **JDK 21 and newer print a warning about dynamic agent loading.** Runtime attach triggers the JVM warning `A future release of the JVM may disallow dynamic loading of agents by default` ([JEP 451](https://openjdk.org/jeps/451)). It is harmless today. A future JVM may need you to start the app with `-XX:+EnableDynamicAgentLoading` for runtime attach to keep working. +- **Standalone Node.js and Python need systemd or PM2.** Without one of them, there is no automatic way to instrument these apps. +- **Python SDK injection covers Python 3.10 through 3.13.** Other versions, such as 3.14, are covered by eBPF monitoring. +- **Go has no code injection.** Its coverage comes from eBPF monitoring. +- **Ruby has no code injection.** For no-code coverage it uses eBPF. Deep Ruby tracing (database queries, view rendering, background jobs, outbound HTTP) needs the OpenTelemetry Ruby gems added to the app, which require Ruby 3.3 or newer. + +## Related + + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/linux-systemd.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/linux-systemd.mdx new file mode 100644 index 00000000..bcca015a --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/linux-systemd.mdx @@ -0,0 +1,49 @@ +--- +title: "Linux (systemd)" +description: "How the KloudMate agent instruments Java, Node.js, Python, and .NET services on Linux, per service, without changing your application code." +sidebar: + order: 2 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On Linux, the agent instruments a service by attaching the matching language agent to it and restarting it. It changes only the service you chose, not the whole system. For the flow, consent, and rollback behavior, see the [Application APM overview](../overview/). + +## How it works + +When you instrument a service, the agent adds the tracing settings to that service and, with your consent, restarts it so the settings take effect. Turning instrumentation off reverses the change and restarts the service, leaving it as it was. Applying the same choice twice does nothing, so it is safe to re-run. + +## Per-runtime support + +The agent uses the standard tracing mechanism for each runtime, so the behavior matches what each language's OpenTelemetry (OTel) agent documents. + +| Runtime | How it is attached | +|---|---| +| Java | The Java agent, through the standard `JAVA_TOOL_OPTIONS` startup option | +| Node.js | The Node.js instrumentation, loaded at startup with `NODE_OPTIONS` | +| Python | The Python auto-instrumentation | +| .NET | The .NET (CLR) profiler (less common on Linux than on Windows) | + +:::note +PHP and Go work differently. PHP is reloaded gracefully instead of restarted; see [PHP instrumentation](../php/). Go is not attached at all; it is covered by [eBPF monitoring](../go-ebpf/). +::: + +## Bundled with the agent + +The language agents come inside the KloudMate agent package and are versioned with it, so they upgrade when the agent does and there is no separate download to manage. + +## Restart behavior + +The agent restarts one service at a time, only with your consent, and can wait for the service to come back up before reporting success. If a restart fails, the agent undoes the change and restarts the service, returning it to its previous state. See the [overview](../overview/) for details. + +## Other process managers + +systemd is the main method on Linux, but not the only one. Node.js and Python apps under **PM2** are instrumented by restarting them with the tracing environment applied. A **Java** process that is not a systemd service is instrumented by runtime attach, with no restart, whether it runs as a bare `java -jar` process or under another supervisor. A bare Node.js or Python process still has to run under systemd or PM2 first. See [Languages and process managers](../language-support/) for the full matrix and the limits of each method. + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/overview.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/overview.mdx new file mode 100644 index 00000000..95d1ecb0 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/overview.mdx @@ -0,0 +1,81 @@ +--- +title: "Application APM overview" +description: "How the KloudMate agent adds distributed tracing to your applications one service at a time, with restart consent and automatic rollback." +sidebar: + order: 1 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Application performance monitoring (APM) is opt-in and per service. For each service the agent discovers, you choose how it is traced: + +- **Off** — not traced. This is the default, so nothing is traced (and nothing costs) until you turn it on. +- **eBPF** — Rate, Errors, and Duration (RED) metrics and trace spans, captured out of process with no code change and no restart. +- **SDK** — OpenTelemetry (OTel) auto-instrumentation attached in-process for full distributed traces, with spans for your frameworks, database calls, and outbound requests. It needs a one-time process restart, so you turn it on one service at a time. + +## The per-service flow + +The flow is the same across every deployment mode: + +1. **Discover.** The agent finds your services and lists them under **Discovered Services**. See [Discovery](../../concepts/discovery/). +2. **Choose a mode.** For each service, pick Off, eBPF, or SDK, then apply. eBPF takes effect on the next agent check-in with no restart. +3. **Confirm the restart (SDK only).** SDK instrumentation takes effect when the process restarts, so the agent asks before applying. After you confirm, the service shows **Pending** while the agent applies the change and restarts the app. +4. **Traces flow.** The service's chip flips to **Instrumented** and it sends application traces to the agent's local collector, correlated with its eBPF spans. + +## Status chips + +Each service selected for SDK instrumentation shows a status chip so you always know where it stands: + +- **Pending:** the service is selected for SDK and the agent is wiring it up — applying the change and restarting the app. It clears to **Instrumented** once the agent confirms, usually within a minute. +- **Instrumented:** the SDK is applied and the service is sending application traces. +- **Failed:** the restart failed, so the agent rolled back the change (see rollback below) and the service reverts to its previous mode. +- **Unsupported:** the runtime version has no bundled instrumentation — for example a Python minor release with no bundled cell. The service stays covered by eBPF; move to a supported version to add SDK traces. + +Services covered by eBPF (Go, Ruby, or anything without an SDK injector) don't restart and carry no chip — eBPF traces them as soon as you turn them on. A separate **Restart pending** marker on the *agents list* is unrelated: it means the agent's own collector configuration is queued for a restart, not a per-service instrumentation change. + +TODO(screenshot): the Discovered Services list with per-service Off / eBPF / SDK controls and status chips. + +## Restart consent and health gating + +The agent never restarts your services silently. + +- Turning SDK instrumentation on or off needs a restart, so the agent asks for **explicit confirmation** first, and the prompt lists exactly which apps restart. eBPF and Off changes apply with no restart and no prompt. +- The agent restarts **one service at a time**. +- It can apply an optional **health gate**. After the restart, the agent waits for the service to become active. Where the listen port is known, it also waits for the service to accept connections before reporting success. + +## Automatic rollback + +If an instrumented service fails to restart or falls into a crash loop, the agent **rolls back automatically**. It removes the change, reloads, and restarts the service, so the service returns to its previous mode, and its chip shows **Failed**. Because you instrument one service at a time, a problem with one never affects the others. + +## Supported runtimes + +| Runtime | How the agent instruments it | +|---|---| +| Java | OpenTelemetry Java agent, attached per service | +| Node.js | OpenTelemetry Node.js instrumentation, attached per service | +| Python | OpenTelemetry Python instrumentation, attached per service | +| .NET | OpenTelemetry .NET auto-instrumentation (CLR profiler) | +| PHP | Attached per service, covering both PHP 7 and PHP 8 | +| Go | Covered by eBPF monitoring, not injected | + +The language agents ship inside the KloudMate agent package and are versioned with it, so there is no separate download to manage. + +## Clean traces without duplicates + +When you instrument a service, the agent tells eBPF monitoring to stop tracing that service's own requests. You get the richer application spans from the language agent, while eBPF keeps tracing the services around it, all on one connected trace. + +## Instrument everything (advanced) + +Instrumenting per service is the default and the recommended approach. For automated rollouts, the agent also has an autonomous mode that instruments every eligible service without per-service toggles. This is an advanced option. See [Advanced configuration](../../advanced-configuration/). + +## Per-mode guides + + + + + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/php.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/php.mdx new file mode 100644 index 00000000..ef16f2d3 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/php.mdx @@ -0,0 +1,32 @@ +--- +title: "PHP" +description: "How the KloudMate agent adds route-level tracing to PHP 7 and 8 applications on Linux and Docker." +sidebar: + order: 5 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +The agent traces PHP **7.0 through 8.x** applications on **Linux** (host PHP-FPM) and **Docker**. It does **not** apply on **Windows**. + +## How it works + +When you instrument a PHP service, the agent attaches tracing to it automatically, with **no application code changes**. It reloads the PHP process manager gracefully, so requests keep being served. If tracing fails to load, it rolls back. If a PHP build is not supported, the agent skips it and tells you why. + +## What you get + +The agent produces route-level server spans, database spans (for example, PDO calls), and error spans. It propagates W3C trace context (`traceparent`), so a PHP span shares its trace identifier with eBPF monitoring and with any upstream or downstream service. + +The overhead is small: a fixed cost of roughly 1.6 milliseconds per request, which is a low single-digit percentage on realistic requests. + +## PHP in containers + +In Docker, the agent instruments PHP containers without a redeploy, and matches the container's PHP build automatically. The instrumented application sends its traces to the agent through the container's host gateway. See [Docker platform notes](../../platform-notes/docker/) for the container details. + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/sampling.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/sampling.mdx new file mode 100644 index 00000000..1fc179f8 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/sampling.mdx @@ -0,0 +1,35 @@ +--- +title: "Sampling" +description: "Set a global sampling default and per-service overrides for the KloudMate agent to balance trace fidelity against ingestion cost." +sidebar: + order: 7 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Sampling controls how many traces an instrumented service keeps. Keeping every request gives you the most detail but costs the most to ingest. Keeping a fraction lowers the cost but drops some detail. The agent lets you set one global default and override it for individual services. Sampling applies to the opt-in application traces. eBPF monitoring metrics are collected separately. + +## Global default and per-service overrides + +- **Global default:** a single sampling rate applies to every service you instrument, so you set the trade-off once. +- **Per-service override:** raise or lower the rate for one service. For example, keep every trace on a low-traffic, high-value service, and sample a noisy one more aggressively. + +The agent applies the rate to each instrumented service through the standard OpenTelemetry (OTel) sampler settings, so the behavior matches what the language agents document. + +## Parent-based sampling keeps traces whole + +The agent uses parent-based ratio sampling. When a service continues a trace that an upstream service already started, it follows the upstream decision instead of making its own. This keeps a sampled trace whole from end to end, so you do not get traces that are kept on one hop and dropped on the next. + +## Choosing a rate + +- Start with a moderate default and adjust based on your traffic volume and ingestion budget. +- Increase the rate on services where you need complete traces for debugging or service-level objectives. +- Decrease the rate on very high-volume services where a representative sample is enough. + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/auto-instrumentation/windows-dotnet.mdx b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/windows-dotnet.mdx new file mode 100644 index 00000000..3e023bd1 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/auto-instrumentation/windows-dotnet.mdx @@ -0,0 +1,75 @@ +--- +title: "Windows and .NET" +description: "How the KloudMate agent instruments .NET on IIS and Windows services with the CLR profiler, plus Java, Node.js, and Python through per-service registry environment." +sidebar: + order: 4 +--- + +import { Tabs, TabItem, LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Windows has no `LD_PRELOAD` and no production eBPF. But its main workload, .NET, can be traced with no code changes through the Common Language Runtime (CLR) profiler. The agent uses the profiler for .NET. For Java, Node.js, and Python it sets per-service environment variables, and it monitors all IIS traffic through Event Tracing for Windows (ETW). The opt-in flow, consent, and rollback are the same as in the [overview](../overview/). + +## .NET applications + +For .NET, the agent uses the OpenTelemetry .NET auto-instrumentation CLR profiler. It sets the profiler environment (`CORECLR_ENABLE_PROFILING=1` for .NET Core and modern .NET, `COR_ENABLE_PROFILING=1` for .NET Framework, plus the profiler identifier and path), then recycles the workload. It writes these entries straight to the Windows registry or to the Internet Information Services (IIS) `applicationHost.config`, so it does not rely on PowerShell or an administrator running commands by hand. + +### Supported runtimes + +Coverage depends on the **installed CLR runtime**, not the framework your application targets. So most older applications are covered, as long as they run on a supported runtime: + +| Installed runtime | Result | +|---|---| +| .NET Framework 4.6.2 and newer (CLR 4.x) | Full application traces through the CLR profiler | +| .NET Core and modern .NET | Full application traces through the CLR profiler | +| .NET Framework 3.5 (CLR 2.0) | No profiler; covered by ETW monitoring below | + +An application built for .NET 4.0 or 4.5 but running on a 4.6.2 or newer runtime is instrumented, because the profiler goes by the runtime, not the build target. + +### IIS and Windows services + + + + +Instrumentation is **per application pool**. The agent writes the profiler environment into that pool's `` in `applicationHost.config`, then recycles the pool. Other pools are unaffected. + + + + +IIS 8.5 has no per-pool environment, so the agent writes to the global registry. The change is host-wide and affects all application pools rather than a single one. + + + + +For a .NET Windows service, the agent writes the profiler environment to the service's registry environment at `HKLM\SYSTEM\CurrentControlSet\Services\\Environment`, then restarts the service. + + + + +:::caution +An ASP.NET Core application pool must be set to "No Managed Code" for the profiler to attach, and applying instrumentation requires an IIS recycle. +::: + +## Java, Node.js, and Python on Windows + +These runtimes use the same language agents as Linux. The agent injects them through the per-service registry environment at `HKLM\SYSTEM\CurrentControlSet\Services\\Environment`, which is the Windows equivalent of a systemd startup config. It sets the standard variable for the runtime (`JAVA_TOOL_OPTIONS`, `NODE_OPTIONS`, or `PYTHONPATH`), then restarts the service to apply it. + +## ETW monitoring + +Because Windows has no eBPF, the agent monitors without any code changes through a custom ETW receiver. It taps the HTTP.sys provider to produce: + +- **Server spans** for every request, at the request-path level. +- **RED metrics** (Rate, Errors, and Duration) grouped by application pool. + +This works for any IIS traffic without injection. That includes workloads that cannot use the profiler, such as .NET Framework 3.5 (CLR 2.0) pools and non-.NET applications hosted on IIS. The ETW receiver only reads events. It needs no credentials and changes nothing in your applications, and it runs under the Local System account the agent uses. When a pool is also instrumented with the profiler, the agent stops generating ETW spans for that pool but keeps its RED metrics, so you do not get duplicate spans. + +## Discovery on Windows + +To build the Discovered Services list, the agent reads the Service Control Manager, IIS sites and application pools, and listening ports through Windows Management Instrumentation (WMI). Services appear with a kind of `iis` or `windows_service`. See [Discovery](../../concepts/discovery/) and [Windows platform notes](../../platform-notes/windows/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/baseline/_meta.json b/src/content/docs/docs/kloudmate-agent/baseline/_meta.json new file mode 100644 index 00000000..8203b3b9 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/baseline/_meta.json @@ -0,0 +1,5 @@ +{ + "label": "Automatic monitoring", + "order": 4, + "collapsed": true +} diff --git a/src/content/docs/docs/kloudmate-agent/baseline/host-metrics-and-logs.mdx b/src/content/docs/docs/kloudmate-agent/baseline/host-metrics-and-logs.mdx new file mode 100644 index 00000000..f10a2b23 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/baseline/host-metrics-and-logs.mdx @@ -0,0 +1,75 @@ +--- +title: "Host metrics and logs" +description: "What the KloudMate agent collects automatically after install: host metrics on Linux, Windows, Kubernetes, and Docker, plus the system, container, or pod logs each platform exposes — with no configuration." +sidebar: + order: 1 +--- + +import { Tabs, TabItem, LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Host metrics and logs are what the agent collects automatically, with no setup. As soon as the agent is installed, it starts collecting infrastructure telemetry from the host and sending it to KloudMate. You do not enable anything or write any configuration. + +## What flows automatically + +What the agent collects depends on the deployment mode, because each platform exposes host telemetry differently. + + + + +- **Host metrics:** CPU, memory, disk, network, and process metrics. +- **Logs:** none are collected from the host automatically. Add application log files with [log monitoring](../../log-monitoring/), or the systemd journal with a [custom config override](../../custom-config-override/). + + + + +- **Host metrics:** CPU, memory, disk, and network usage, collected from Windows performance counters. +- **Windows Event Log:** the System, Application, and Security channels. +- **Internet Information Services (IIS) metrics** and **Windows service status**, where those roles are present. + + + + +- **Per-node metrics:** CPU, memory, disk, and network for each node. +- **Pod logs:** container stdout and stderr. +- **Cluster metrics and events:** workload, namespace, and node state, plus cluster events. + + + + +- **Container metrics:** CPU, memory, network, and disk usage for each running container, plus container lifecycle events. +- **Container logs:** stdout and stderr from each container. +- **Host metrics:** CPU, memory, disk, and network for the Docker host. + + + + +:::note +Host telemetry needs no configuration. eBPF monitoring and application tracing add to it when you enable them. +::: + +## Where to view it + +Host metrics and logs feed the infrastructure and log views in KloudMate: + +- **Infrastructure monitoring** shows host and container metrics, availability, and trends. See [Infrastructure](../../../infrastructure/). +- **Kubernetes monitoring** shows cluster, node, namespace, pod, and workload metrics. See [Kubernetes](../../../infrastructure/kubernetes/). +- **Log Explorer** lets you search and filter logs and correlate them with metrics. See [Log Explorer](../../../logs/log-explorer/). + +## Collecting custom application logs + +The agent does not collect your application log **files** automatically. To add them, use [log monitoring](../../log-monitoring/). You point the agent at a folder and file-name pattern, choose how to parse each line, then preview the result before you save. Multiline grouping, for stack traces or pretty-printed JSON, is a modifier you apply on top of any format. For a hand-written receiver in manual mode, see [Collect file logs](../../../logs/collect-file-logs/). + +## What else the agent collects automatically + +Alongside host metrics and logs, two more capabilities turn on without configuration: + +- **Discovery** lists what runs on the host, so you can instrument it. See [Discovery](../../concepts/discovery/). +- **eBPF monitoring** adds Rate, Errors, and Duration (RED) metrics, a service map, and Database Activity Monitoring, with no code changes. See [eBPF observability](../../ebpf-observability/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/concepts/_meta.json b/src/content/docs/docs/kloudmate-agent/concepts/_meta.json new file mode 100644 index 00000000..3b8d4ea2 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/concepts/_meta.json @@ -0,0 +1,5 @@ +{ + "label": "Concepts", + "order": 2, + "collapsed": true +} diff --git a/src/content/docs/docs/kloudmate-agent/concepts/architecture.mdx b/src/content/docs/docs/kloudmate-agent/concepts/architecture.mdx new file mode 100644 index 00000000..50d213c0 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/concepts/architecture.mdx @@ -0,0 +1,46 @@ +--- +title: "Architecture" +description: "How the KloudMate agent is managed from the web interface, and how the same model works across Linux, Windows, Kubernetes, and Docker." +sidebar: + order: 1 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +With the KloudMate agent, you decide what to collect in the KloudMate web interface, and the agent on your host does the collecting. The two sides work the same way across every deployment mode. + +## Two sides + +- **KloudMate (the web interface):** where you choose what each agent collects, and where you see the discovered inventory and the telemetry. +- **The agent (on your host):** finds your services, collects host metrics and logs, applies instrumentation when you ask for it, and sends the telemetry to KloudMate. + +The agent checks in with KloudMate regularly to report what it found and pick up its settings. When you change a setting in the web interface, the agent applies it on its next check-in. You never edit files on the host or use Secure Shell (SSH). + +## Principles + +A few principles shape how the agent behaves: + +- **Standard monitoring is automatic; depth is opt-in.** Anything safe and low-risk (host metrics, logs, and eBPF monitoring) turns on by itself. Anything that needs a process restart is something you turn on per service. +- **You do not write configuration.** You turn features on and off in the web interface, and KloudMate produces the settings for you. A manual editor is available for advanced setups. +- **The experience is the same everywhere.** You find a service, turn on what you want, and the agent applies it, whether you run on a virtual machine, Windows, Kubernetes, or Docker. + +## How the model maps onto each deployment mode + +The two-sided model is the same in every mode. What changes is how the agent is packaged and how it applies instrumentation. + +| Mode | How the agent runs | How it applies instrumentation | +|---|---|---| +| Linux / VM | A single agent as a systemd service. | Per service, using systemd. | +| Windows | A single agent as a Windows service. | Per service or application pool, using the registry and Internet Information Services (IIS). | +| Kubernetes | A fleet-manager pod plus collector pods. One agent represents one cluster. | Per workload, using the OpenTelemetry (OTel) Operator. | +| Docker | A containerized collector. | PHP containers without a redeploy; Java, Node.js, Python, and .NET through eBPF monitoring. | + +On Kubernetes, one cluster appears as one agent rather than one per node. See [Kubernetes platform notes](../../platform-notes/kubernetes/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/concepts/config-model.mdx b/src/content/docs/docs/kloudmate-agent/concepts/config-model.mdx new file mode 100644 index 00000000..8aa09917 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/concepts/config-model.mdx @@ -0,0 +1,62 @@ +--- +title: "Configuration model" +description: "How managed and manual configuration modes work for the KloudMate agent, and how to provide database credentials safely." +sidebar: + order: 3 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +In **managed mode**, you turn features on and off from the KloudMate web interface and the agent is configured for you, with no hand-edited YAML. This is how new agents run by default, and it covers host metrics, eBPF monitoring, application performance monitoring (APM), database monitoring, and everything else you toggle in the interface. + +A feature only appears where it applies. For example, eBPF monitoring is offered only on hosts with a supported Linux kernel. + +## Managed and manual modes + +Every agent runs in one of two configuration modes. + +| | Managed mode | Manual mode | +|---|---|---| +| Who owns the configuration | KloudMate, from the features you enable | The operator, in the raw YAML | +| YAML editor | Read-only preview | Editable | +| Feature toggles | Available | Disabled | +| Default for | New agents | Existing agents after upgrade | + +- In **managed mode**, the configuration is generated from your enabled integrations every time. The YAML editor shows a read-only preview of what was generated. Toggling a feature regenerates the configuration. + +![Managed mode: the Config mode toggle set to Managed, with a read-only preview of the generated collector configuration.](./images/config-model-managed-mode.png) + +- In **manual mode**, you own the YAML blob. The integration toggles are disabled, and the YAML editor is writable. This is for advanced setups that need hand-tuned configuration. + +![Manual mode: the Config mode toggle set to Manual, with an editable collector configuration and a warning that KloudMate no longer manages it.](./images/config-model-manual-mode.png) + +## Undo a change + +To undo a managed change, turn the toggle back. There is no separate history or rollback button. + +If you switch an agent from manual to managed mode, KloudMate saves a copy of your hand-edited YAML first, so you never lose your manual edits. + +## Credentials + +Some integrations, such as database receivers, need credentials. You can provide them two ways: + +- **Environment reference (default):** the value stays on the host as an environment variable and never reaches KloudMate or the stored configuration. This is recommended, and it makes rotation easier: you update the secret on the host without touching the agent. +- **Plaintext inline (opt-in):** the value is stored in the configuration, and the interface warns you that it is stored unencrypted. Use this only when you accept that trade-off. + +See [Database credentials](../../database-monitoring/credentials/). + +## Migrating existing agents + +When an agent is upgraded, it stays in **manual** mode so nothing changes unexpectedly. To adopt managed mode, switch it in the interface. It starts from a sensible default: host metrics, logs, and eBPF monitoring, plus anything discovery found. New agents already start in managed mode. + +:::caution +Switching an agent from manual to managed mode replaces its hand-edited YAML with generated configuration. KloudMate saves the previous manual configuration first, so it is not lost. Still, review the change before you switch a production agent. +::: + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/concepts/discovery.mdx b/src/content/docs/docs/kloudmate-agent/concepts/discovery.mdx new file mode 100644 index 00000000..55118120 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/concepts/discovery.mdx @@ -0,0 +1,62 @@ +--- +title: "Discovery" +description: "How the KloudMate agent inventories the services, runtimes, and databases on each host, and how that inventory powers instrumentation and monitoring." +sidebar: + order: 2 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Discovery builds a live inventory of what runs on each host or cluster. The web interface uses it to offer to instrument a service or monitor a database, instead of asking you to configure it by hand. It runs on all deployment modes. + +## What discovery finds + +The agent finds two kinds of things: + +- **Application services**, classified by runtime (Java, .NET, Node.js, Python, PHP, Go, and Ruby), with framework and web-server hints (nginx, Apache, and php-fpm) where they apply. +- **Databases and middleware** running locally, such as PostgreSQL, MySQL, Redis, and MongoDB. + +For each service, discovery records how it is managed and which ports it listens on. + +## The Discovered Services list + +Discovery shows its inventory as a **Discovered Services** list on the host detail page in the web interface. Each entry shows: + +- **Name** and **runtime** (for example, `checkout` running Java). +- **Runtime version**, where it affects a decision (for example, the PHP major and minor version, or whether a .NET app runs on .NET Framework or .NET Core). +- **Kind**: how the service is managed, such as a systemd unit, a bare process, a container, a Windows service, or an Internet Information Services (IIS) application pool. +- **Listen ports**. +- **State**: `running` or `idle`. Discovery also reports installed-but-not-running services as `idle`, so you can choose to instrument one on its next start. +- **Instrumentable** flag: whether the agent has a way to attach application tracing to that runtime. + +:::note +The `instrumentable` flag shows whether the agent can inject tracing into that runtime. Java, .NET, Node.js, and Python are instrumentable. PHP is instrumentable when a tracer is available for the detected version. Go is not injected: it is covered by eBPF monitoring instead, so it appears as covered rather than instrumentable. +::: + +![The Application instrumentation panel on a host, showing the instrumentable difference: .NET, Java, Node.js, Python, and PHP services offer Off / eBPF / SDK, while Go services and other system services offer only Off / eBPF.](./images/discovery-services-list.png) + +## Stable service identity + +Each discovered service gets a **stable identity** that survives restarts and process-ID changes. This is what lets an **Instrument** choice keep applying after a service restarts: you instrument `checkout`, not a process that will be gone after the next deploy. + +## Privacy and safety + +Discovery keeps sensitive data on the host: + +- It **never sends raw environment variables or full command lines** to KloudMate, because these can contain secrets. It extracts only what it needs to classify a service (runtime, version, and listen ports) and discards the rest. +- It skips the agent itself and developer tools, so they are not listed or instrumented. + +## How discovery powers the rest of the agent + +The Discovered Services list is the entry point for the opt-in features: + +- **Application APM:** each instrumentable service has an **Instrument** toggle that starts tracing for that service. See [Application APM](../../auto-instrumentation/overview/). +- **Database monitoring:** a discovered database starts the monitoring wizard, which prefills the engine and endpoint. See [Database monitoring](../../database-monitoring/overview/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-managed-mode.png b/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-managed-mode.png new file mode 100644 index 00000000..39dc428e Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-managed-mode.png differ diff --git a/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-manual-mode.png b/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-manual-mode.png new file mode 100644 index 00000000..79987fe1 Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/concepts/images/config-model-manual-mode.png differ diff --git a/src/content/docs/docs/kloudmate-agent/concepts/images/discovery-services-list.png b/src/content/docs/docs/kloudmate-agent/concepts/images/discovery-services-list.png new file mode 100644 index 00000000..9334e067 Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/concepts/images/discovery-services-list.png differ diff --git a/src/content/docs/docs/kloudmate-agent/custom-config-override.mdx b/src/content/docs/docs/kloudmate-agent/custom-config-override.mdx new file mode 100644 index 00000000..691f5dab --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/custom-config-override.mdx @@ -0,0 +1,81 @@ +--- +title: "Custom config override" +description: "Layer your own collector receivers, processors, or pipelines on top of a managed KloudMate agent without switching to manual mode." +sidebar: + label: "Custom config override" + order: 13.5 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +The **custom config override** lets you add your own collector configuration on top of a managed agent, without giving up managed mode. Use it when the managed integrations cover almost everything you need and you want to add one processor, adjust an exporter, or wire in an extra pipeline. + +This is the lighter alternative to [manual mode](../advanced-configuration/). Manual mode hands you the whole collector YAML and turns the feature toggles off. The override keeps you in managed mode, keeps every toggle working, and applies only the small piece of YAML you add. + +## Where the override sits + +Your override is the **final layer** of the agent's configuration. The agent merges it on top of everything else, in this order: + +1. The base configuration. +2. Whatever your managed integrations generate. +3. Every receiver and processor the agent adds automatically (eBPF, PHP, PM2, database monitoring, and so on). +4. **Your override, last.** + +Because it merges last, the override wins any conflict. It is also kept when your integrations change, so you do not have to reapply it after toggling a feature. + +## Set an override + +1. Open the agent's **Configuration** page. +2. Find the **Custom config override** section. +3. Enter your collector YAML in the editor. It is validated as you type. +4. Choose **Save override**. + +The agent picks up the change on its next check-in. + +## How the merge works + +The override is deep-merged with the configuration underneath it: + +- **Maps merge.** Adding a new processor under `processors:` leaves the existing processors untouched. +- **Scalars and lists are replaced, not combined.** If your override sets a key that already has a value, or a list such as a pipeline's `receivers:`, your value replaces what was there. A list is never appended to, so include every element you want when you override one. + +## Example: stamp an attribute on all metrics + +This adds a `resource` processor and wires it into the metrics pipeline, so every metric carries a `team` attribute: + +```yaml +processors: + resource/team: + attributes: + - key: team + value: platform + action: upsert +service: + pipelines: + metrics: + processors: [resource/team] +``` + +Because a list is replaced rather than merged, the `processors:` list you set here becomes the metrics pipeline's processor list. Include the other processors that pipeline needs alongside `resource/team`. + +## Example: raise exporter verbosity while debugging + +To make the debug exporter log full telemetry while you diagnose a pipeline: + +```yaml +exporters: + debug: + verbosity: detailed +``` + +## Safety + +Before it applies a new configuration, the agent validates it. If your override makes the configuration invalid (most often from a wrong top-level key), the agent **rejects it and keeps the last working configuration** instead of restarting the collector into a broken state. The reason is reported back to your workspace, so you can see why an override was not applied without logging in to the host. + +## Related + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/_meta.json b/src/content/docs/docs/kloudmate-agent/database-monitoring/_meta.json new file mode 100644 index 00000000..f1a1a068 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/_meta.json @@ -0,0 +1,5 @@ +{ + "label": "Database monitoring", + "order": 7, + "collapsed": true +} diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/credentials.mdx b/src/content/docs/docs/kloudmate-agent/database-monitoring/credentials.mdx new file mode 100644 index 00000000..c84c81d6 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/credentials.mdx @@ -0,0 +1,49 @@ +--- +title: "Database credentials" +description: "The three credential modes for KloudMate agent database monitoring: environment reference, plaintext inline, and operator-managed secrets, and how to rotate them." +sidebar: + order: 5 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Database monitoring needs a monitoring user's credentials to read from the database. The agent supports three ways to provide them, so you can keep secrets out of your configuration. The automatic eBPF view of your queries needs no credentials at all. For the general flow, see the [Database monitoring overview](../overview/). + +## The three credential modes + +### Environment reference (default) + +The credential is stored as a reference, not a value. The agent reads it from an environment variable on the host at runtime, so the plaintext value never reaches KloudMate or the stored configuration. This is the default and the recommended mode. + +The agent also handles escaping for you, so a password with special characters works in a connection string without you having to encode it. + +### Plaintext inline + +You can enter the value directly. It is then stored in the configuration. The interface shows a clear "stored unencrypted" warning, because the value lives in the configuration and on disk. Use this only when you accept that trade-off. + +### Operator-managed + +The credential is still an environment reference, but you supply the environment variable yourself rather than through the wizard. This suits secrets that come from an external secret manager. How you supply the variable depends on the platform: + +- **On a virtual machine:** set `KM_SECRET_` for the agent, for example through the systemd `EnvironmentFile`. +- **On Kubernetes:** put `KM_SECRET_` in a Kubernetes Secret and point the Helm release at it with the `dbMonitoring.secretName` value. See [Database monitoring on Kubernetes](../on-kubernetes/). + +## Rotation + +The reference modes store only a reference. So you rotate a database password by updating the environment variable or Kubernetes Secret on the host. The agent's configuration does not change. + +:::note +On Kubernetes, the collector reads the Secret when its pod starts, so a rotated password takes effect on the next pod restart. See the ordering note in [Database monitoring on Kubernetes](../on-kubernetes/). +::: + +## Grants + +Whichever mode you use, the monitoring user needs read access to the engine's statistics. Some coverages need specific grants too, for example `GRANT pg_monitor` on PostgreSQL. The wizard shows the exact setup steps for the coverages you selected. See [Engines and coverages](../engines-and-coverages/). + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/engines-and-coverages.mdx b/src/content/docs/docs/kloudmate-agent/database-monitoring/engines-and-coverages.mdx new file mode 100644 index 00000000..006dd668 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/engines-and-coverages.mdx @@ -0,0 +1,60 @@ +--- +title: "Engines and coverages" +description: "The database engines the KloudMate agent monitors and the metrics, query performance, locks, and logs available for each engine." +sidebar: + order: 2 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +You choose what to monitor for a database as a set of **coverages**. This page lists the engines the agent monitors and the coverages available for each. + +## Coverage types + +- **Standard metrics:** the database's health and capacity metrics, such as connections, cache hit ratio, replication lag, buffers, and table and index statistics. You can collect a basic set or a full set. +- **Query performance:** statistics for the most expensive queries across all clients, so you can find slow and heavy queries. +- **Locks:** lock and blocking information. +- **Query logs:** query activity captured as log records. +- **Server and slow logs:** the database's own log entries, including slow queries. + +## Engine and coverage matrix + +| Engine | Standard metrics | Query performance | Locks | Query logs | Logs | +|---|---|---|---|---|---| +| PostgreSQL | Yes | Yes | Yes | No | Server logs | +| MySQL | Yes | Yes | No | No | Slow log | +| SQL Server | Yes | Yes | No | Yes | No | +| Oracle | Yes | Yes | No | Yes | No | +| MongoDB | Yes | No | No | No | No | +| MongoDB Atlas | Yes | No | No | No | No | +| Redis | Yes | No | No | No | No | +| Elasticsearch | Yes | No | No | No | No | +| ClickHouse | Yes | No | No | No | No | +| SAP HANA | Yes | No | No | No | No | +| Snowflake | Yes | Yes | No | No | No | + +:::note +Database log coverages are available on a virtual machine. On Kubernetes, the agent collects database metrics and query coverages. See [Database monitoring on Kubernetes](../on-kubernetes/). +::: + +## Setup steps + +Some coverages need a one-time setup step on the database. The wizard shows you the step when you select that coverage. For example: + +- PostgreSQL query performance needs the `pg_stat_statements` extension enabled, which needs a PostgreSQL restart. +- Query performance and query-log coverages need the monitoring user to have the right grants, such as `GRANT pg_monitor` on PostgreSQL or the equivalent view permissions on SQL Server and Oracle. +- Log coverages may need a log setting turned on, such as the MySQL slow query log. + +If a prerequisite is missing, the agent reports it and the other coverages keep working. + +## Filtering database logs + +Database logs carry a `km.db.coverage` attribute set to the coverage they came from, such as `query-performance`, `server-logs`, or `slow-log`. You can filter database logs by this attribute in Log Explorer. + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/images/wizard-add-database.png b/src/content/docs/docs/kloudmate-agent/database-monitoring/images/wizard-add-database.png new file mode 100644 index 00000000..2d6d8c16 Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/database-monitoring/images/wizard-add-database.png differ diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/on-kubernetes.mdx b/src/content/docs/docs/kloudmate-agent/database-monitoring/on-kubernetes.mdx new file mode 100644 index 00000000..33280cba --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/on-kubernetes.mdx @@ -0,0 +1,86 @@ +--- +title: "Database monitoring on Kubernetes" +description: "How the KloudMate agent monitors databases running in your Kubernetes cluster, using a Kubernetes Secret for credentials and managed mode." +sidebar: + order: 4 +--- + +import { Steps, LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On Kubernetes, the agent monitors the databases running in your cluster. The flow is the same as on a virtual machine, with two Kubernetes differences. Credentials come from a Kubernetes Secret, and database monitoring needs managed mode. For the general model, see the [Database monitoring overview](../overview/). + +## One monitor per cluster + +On Kubernetes, one collector in the cluster monitors your databases. Each database is monitored once for the whole cluster, not once per node. + +## Discovery prefills the endpoint + +The agent finds databases running in your cluster. It prefills the wizard with their in-cluster address, `..svc.cluster.local:`. Use this address. It is the name your applications connect to, so the database's metrics and traces line up. Databases outside the cluster are not discovered, so add them manually. + +## Managed mode is required + +Database monitoring needs the agent in **managed mode**. On Kubernetes, use the "Configure databases" action on the agent. If the agent is still in manual mode, it prompts you to switch. See the [configuration model](../../concepts/config-model/). + +## Available coverages + +On Kubernetes, the agent collects database metrics and query coverages: standard metrics, query performance, locks, and query logs. Database log coverages are available on a virtual machine, not on Kubernetes. See [Engines and coverages](../engines-and-coverages/). + +## Provide credentials with a Kubernetes Secret + +Credentials use an environment reference backed by a Kubernetes Secret you provide. Set it up before you configure any database, because the collector reads the Secret when its pod starts. + + +1. **Create a Kubernetes Secret** with a key of the form `KM_SECRET_` for each database credential. For example, a key `KM_SECRET_postgresql` holding the monitoring user's password. + + ```bash + kubectl create secret generic km-db-creds \ + --namespace km-agent \ + --from-literal=KM_SECRET_postgresql='' + ``` + +2. **Point the Helm release at the Secret** with the `dbMonitoring.secretName` value, and upgrade the release. The cluster collector then reads the Secret through `envFrom`. + + ```bash + helm upgrade kloudmate-release kloudmate/km-kube-agent \ + --namespace km-agent --reuse-values \ + --set dbMonitoring.secretName=km-db-creds + ``` + +3. **Configure the database** in the wizard, referencing the key by name. The password stays in the cluster and never reaches KloudMate. + + +:::caution +Because the collector reads the Secret at pod startup, create the Secret and run the Helm upgrade before you configure a database. If you rotate a password without changing the configuration, the collector keeps the old value until its pod restarts. +::: + +## Worked example: PostgreSQL + + +1. **Create a monitoring role** in PostgreSQL and grant it the built-in `pg_monitor` role: + + ```sql + CREATE ROLE kmmonitor LOGIN PASSWORD ''; + GRANT pg_monitor TO kmmonitor; + ``` + +2. **Store the password** in a Kubernetes Secret under `KM_SECRET_postgresql` and set `dbMonitoring.secretName`, as above. + +3. **Configure PostgreSQL** in the wizard with the Service DNS endpoint (prefilled from discovery), the `kmmonitor` user, and an environment reference to the secret. Choose the coverages you want, such as standard metrics, locks, and query performance. + +4. **Enable query performance prerequisites, if used.** Query performance reads `pg_stat_statements`. Add it to `shared_preload_libraries` and create the extension, which needs a PostgreSQL restart: + + ```sql + -- postgresql.conf: shared_preload_libraries = 'pg_stat_statements' (then restart) + CREATE EXTENSION pg_stat_statements; + ``` + + +After you apply the configuration, the cluster collector starts collecting from PostgreSQL. Its metrics carry the same database identity as the application spans that hit it. + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/on-vm.mdx b/src/content/docs/docs/kloudmate-agent/database-monitoring/on-vm.mdx new file mode 100644 index 00000000..8b0fcd0b --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/on-vm.mdx @@ -0,0 +1,43 @@ +--- +title: "Database monitoring on a VM" +description: "How the KloudMate agent monitors databases on a Linux or Windows host, covering databases on the host and reachable remote databases." +sidebar: + order: 3 +--- + +import { Steps, LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On a virtual machine, the agent monitors databases running on the host or reachable from it. For the general flow, see the [Database monitoring overview](../overview/). + +## What you can monitor + +- **Databases on the host:** a database listening on the host is discovered and prefilled in the wizard. +- **Reachable remote databases:** you can also monitor a database on another host by entering its endpoint and credentials in the wizard. + +On a virtual machine, you can collect the full set of coverages, including database logs. See [Engines and coverages](../engines-and-coverages/). + +## How local databases are identified + +A database on the host often listens on `localhost`. That address is not unique across your servers. So the agent identifies the database by the host's routable address instead, which keeps its metrics and traces lined up. See [Telemetry identity](../../reference/telemetry-identity/) for how to build dashboards on a database's server address. + +## Set up monitoring + + +1. **Open the wizard** from a discovered database, or add one manually with its endpoint. + +2. **Choose coverages** for the depth you want. See [Engines and coverages](../engines-and-coverages/). + +3. **Provide credentials** for the monitoring user, using an environment reference by default. See [Database credentials](../credentials/). + +4. **Apply the setup steps** the wizard shows, such as creating the monitoring role and granting it the right permissions. + +5. **Save.** The agent applies the configuration and starts collecting. + + +## Next steps + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/database-monitoring/overview.mdx b/src/content/docs/docs/kloudmate-agent/database-monitoring/overview.mdx new file mode 100644 index 00000000..95b74832 --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/database-monitoring/overview.mdx @@ -0,0 +1,48 @@ +--- +title: "Database monitoring overview" +description: "How the KloudMate agent monitors your databases: automatic query visibility from eBPF plus opt-in health metrics, query performance, and logs." +sidebar: + order: 1 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +Database monitoring gives you a full picture of your databases. You set it up through a wizard, not hand-written configuration. It works alongside the automatic eBPF view of your queries. + +## Two views of your databases + +The agent shows your databases in two ways that work together: + +- **From eBPF, automatically and with no credentials.** Database Activity Monitoring (DAM) shows the queries your applications run against the database, with their timing. These appear as part of your distributed traces and the service map. This turns on by default. See [eBPF observability](../../ebpf-observability/). +- **From the database, when you opt in.** Database monitoring reads the database's own health metrics (connections, cache hit ratio, replication lag, and more), query performance, and logs. This needs a monitoring user's credentials. + +The two views line up on a shared database identity, so a slow query you see in a trace matches the server's health metrics for the same database. See [Telemetry identity](../../reference/telemetry-identity/). + +## Coverages + +You choose what to monitor for a database as a set of **coverages**, such as standard metrics, query performance, locks, or logs. You pick the coverages you want, and the agent sets up the collection. See [Engines and coverages](../engines-and-coverages/) for what each engine supports. + +## The wizard + +You set up monitoring for a database through a wizard, usually starting from a database the agent already found: + +1. **Pick the engine.** The agent lists databases it discovered, so the engine and endpoint are often prefilled. +2. **Choose coverages.** Turn on the coverage areas you want, such as standard metrics, query performance, or locks. +3. **Provide credentials.** Enter the monitoring user and its secret. By default, the secret is stored as a reference and never leaves the host. See [Database credentials](../credentials/). +4. **Follow the setup steps.** The wizard shows the exact grants and prerequisites each coverage needs, ready to copy. +5. **Review and apply.** The wizard previews what will be collected, and on save the agent starts collecting. + +![The "Add database monitoring" wizard for a discovered MySQL instance, showing coverage toggles (query performance, slow query log) and the credential mode selector (environment variable or entered value).](./images/wizard-add-database.png) + +## Safe to apply + +The agent validates the setup before applying it. It handles each database on its own, so one bad database setting cannot affect the others or your other telemetry. + +## Where to run it + + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/ebpf-network-topology.mdx b/src/content/docs/docs/kloudmate-agent/ebpf-network-topology.mdx new file mode 100644 index 00000000..df41cbac --- /dev/null +++ b/src/content/docs/docs/kloudmate-agent/ebpf-network-topology.mdx @@ -0,0 +1,53 @@ +--- +title: "eBPF network topology" +description: "How the KloudMate agent builds a bounded host-to-host network map from eBPF flows on VMs, using host IP and CIDR zones instead of unbounded per-peer addresses." +sidebar: + label: "Network topology (eBPF)" + order: 5.5 +--- + +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; + +On a VM host, the agent's [eBPF monitoring](../ebpf-observability/) reports Layer 4 network flows straight from the kernel. KloudMate turns those flows into a host-to-host map of what talks to what across your fleet. + +This is a metrics-only signal. It is separate from application tracing: a host can report network flows without any application spans, and enabling it does not instrument any process. + +## Bounded by design + +The flow metric is deliberately kept low-cardinality. Each flow carries its **direction**, plus a source and destination **zone** label when you have zones configured (see below). It does **not** carry a label per peer address. + +This matters on any host that talks to the public internet. If every peer got its own address label, one internet-facing service could produce an unbounded number of metric series. By keeping the flow to direction and coarse zones, the number of series stays flat no matter how many external addresses a host reaches. + +## How KloudMate places a host on the map + +Each agent reports its **own host IP** as a resource attribute (`host.ip`). This attribute is attached to every signal the host sends, including these flow metrics. KloudMate uses it two ways: + +- **Matching IPs to hosts.** KloudMate learns which host has which IP across every agent, so it can map both ends of a flow to real hosts. +- **Finding which end is which.** The `host.ip` on a flow is the host that observed it. The other end of the flow is the peer. + +Together these let KloudMate draw an edge between two of your hosts from a flow that itself only carries direction and zones. + +## Group peers into zones with CIDRs + +Peers are grouped into zones by CIDR range. KloudMate builds that set of ranges from your fleet automatically: each host it knows about becomes a `/32`, plus a `0.0.0.0/0` catch-all for everything else. A flow to another host in the fleet lands in that host's `/32` and resolves to a machine. All other traffic falls into the catch-all, which keeps external peers from inflating the signal. + +You can add your own ranges on top. Set `KM_EBPF_NETWORK_CIDRS` to a comma-separated list of CIDRs to label traffic by your own network segments, for example an app tier or a database subnet: + +```bash +KM_EBPF_NETWORK_CIDRS="10.0.1.0/24,10.0.2.0/24,10.0.0.0/8" +``` + +## Turning it on and off + +Network flow monitoring is **on by default** wherever the kernel supports eBPF. Like the rest of eBPF, it only turns on where the kernel can load it. A host that cannot load it still sends metrics and logs instead of failing. To turn it off, set `KM_EBPF_NETWORK_ENABLED=false`. + +## Gotcha: a host can report more than one IP + +`host.ip` reports every non-loopback address on the host, which can include a Docker bridge address such as `172.17.0.1` and IPv6 link-local addresses such as `fe80::…`. Only the routable address (your VPC or LAN IP) is meaningful. If a host resolves oddly, check that its routable IP is the one being matched. + +## Related + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/ebpf-observability.mdx b/src/content/docs/docs/kloudmate-agent/ebpf-observability.mdx index 69519a79..b75589ed 100644 --- a/src/content/docs/docs/kloudmate-agent/ebpf-observability.mdx +++ b/src/content/docs/docs/kloudmate-agent/ebpf-observability.mdx @@ -1,40 +1,98 @@ --- title: "eBPF Observability" -description: "Enable KloudMate Agent eBPF features to collect APM, service dependency, network, and database telemetry without code changes." +description: "Understand and tune the KloudMate agent eBPF monitoring: RED metrics, service map, and Database Activity Monitoring from the Linux kernel, with no code changes." sidebar: - order: 3 + order: 5 --- -eBPF (Extended Berkeley Packet Filter) allows the KloudMate Agent to observe your applications directly from the Linux operating system kernel. You get deep visibility into network traffic, database queries, and application performance **without changing a single line of code or installing SDKs**. -This guide explains how to enable and configure eBPF data collection through the KloudMate web interface. +import { LinkCard, CardGrid } from '@astrojs/starlight/components'; -## Step 1: Install the KloudMate Agent +Extended Berkeley Packet Filter (eBPF) lets the KloudMate agent observe your applications directly from the Linux kernel. You get visibility into network traffic, database queries, and application performance **with no code changes and no SDKs**. -Before you can enable eBPF, the KloudMate Agent must be running on your host or cluster. eBPF requires a Linux environment (Kernel `5.8+` recommended). +eBPF is part of the agent's automatic monitoring. On a managed-mode agent, it turns on by default wherever the Linux kernel supports it, so you get Rate, Errors, and Duration (RED) metrics and a service map as soon as the agent is running. -Follow the relevant installation guide if you haven't already: -- [Linux Installation Guide](../installation/linux-agent/) -- [Kubernetes Installation Guide](../installation/kubernetes-agent/) +## What eBPF collects -## Step 2: Enable eBPF via the Config Editor +When eBPF monitoring is active, the agent collects: -The KloudMate Agent is managed centrally. You **do not** need to SSH into your servers or manually edit YAML files. You enable eBPF by updating the agent's remote configuration via the KloudMate dashboard. +- **RED metrics** (Rate, Errors, and Duration) for your services, with no instrumentation. +- **Service dependencies**, used to build the visual service map, including peer-to-peer latency. +- **Network topology:** a host-to-host map of Layer 4 flows across your fleet. See [eBPF network topology](../ebpf-network-topology/). +- **Database Activity Monitoring (DAM):** kernel-level capture of database queries on the wire, with no credentials. +- **Distributed context propagation:** the agent reads and injects HTTP trace context (`traceparent`), so requests stay on one trace across services. +- **Log-to-trace correlation:** trace identifiers are added to application logs so you can jump between a log line and its trace. -1. Log in to your KloudMate Platform. -2. Navigate to the **Agents** landing page. -3. Locate the Agent installed on your target Linux host or Kubernetes cluster. -4. Click **Collector Configuration** to open the remote YAML editor. +Because eBPF observes traffic in the kernel, it can trace a service with no code change and no restart — including Go and other services no language agent can inject. eBPF RED metrics and trace spans are a per-service choice you turn on (the **eBPF** APM mode); the network topology map and database activity monitoring run automatically. When a service also runs application APM, its eBPF spans and the application spans share the same trace, so hops you haven't put on the SDK still appear in end-to-end traces. See the [Application APM overview](../auto-instrumentation/overview/). -### Base Configuration for APM +## Deployment modes and requirements -Add or verify the following configuration blocks in the web editor to enable RED metrics, distributed traces, and network telemetry. +eBPF applies on **Linux, Kubernetes, and Docker**. It does **not** apply on **Windows**, which has no eBPF equivalent; on Windows the agent provides a comparable monitoring through Event Tracing for Windows (ETW) instead. See [Windows platform notes](../platform-notes/windows/). + +On Linux, eBPF needs: + +| Requirement | Detail | +|---|---| +| Kernel version | 4.14 or newer for eBPF monitoring. 5.8 or newer (with BPF Type Format, BTF) unlocks richer, lower-overhead attachment. Some features benefit from 5.17 or a backport. | +| Privileges | The `CAP_SYS_ADMIN` (or `CAP_BPF`) capability, or a privileged context. | +| Kernel debug filesystem | `debugfs` must be mounted. | + +The agent runs with the privileges it needs for eBPF by default on Linux. On Kubernetes, the collector pods run with the required host access. + +## When eBPF turns on + +Whether eBPF turns on, and how much of it, depends on the host's kernel version, privileges, and whether `debugfs` is mounted: + +- **Supported host:** eBPF turns on automatically. +- **Kernel 4.14 to 5.8, or no BTF:** eBPF runs with reduced features. +- **Unsupported host** (kernel older than 4.14, `debugfs` not mounted, or missing privileges): eBPF is skipped, and KloudMate shows the reason, for example "eBPF unavailable: kernel 4.9". The host still gets host metrics and logs. + +If eBPF cannot load on a host that should support it, the host keeps its metrics and logs and KloudMate reports why. + +## Tuning eBPF + +On a managed-mode agent, you tune eBPF through its settings in the web interface. On a manual-mode agent, you edit the collector YAML directly. See the [configuration model](../concepts/config-model/) for how the two modes differ. + +### Context propagation + +The `context_propagation` setting controls how far the agent carries trace context. It accepts `disabled`, `headers`, `tcp`, or `all`: + +- `headers` propagates context through HTTP headers. This is the safe default for distributed tracing across services. +- `tcp` and `all` extend propagation to non-HTTP paths and uninstrumented applications. They reach further, but you should validate them across proxies, load balancers, and network address translation (NAT) before relying on them. + +Use `track_request_headers` to capture request headers such as `User-Agent`. + +### Database tuning + +For high-load database environments, tune statement caching and heuristic detection under the `ebpf` block: + +```yaml +ebpf: + heuristic_sql_detect: true + mysql_prepared_statements_cache_size: 1024 + postgres_prepared_statements_cache_size: 1024 +``` + +### Overhead + +eBPF monitoring runs within a budget of roughly 50 to 100 MB of memory and 5 to 10 percent CPU. The agent automatically reduces buffer sizes and sampling on small instances, and you can dial the overhead down further with the tuning settings if you need to. + +## Enabling or tuning eBPF in manual mode + +On a manual-mode agent, eBPF is managed through the remote YAML editor, with no need to log in to the host or edit local files. + +1. Log in to your KloudMate platform. +2. Go to the **Agents** landing page. +3. Locate the agent on your target Linux host or Kubernetes cluster. +4. Open the collector configuration editor. + +Add or verify the following blocks to collect RED metrics, distributed traces, and network telemetry: ```yaml metrics: features: - - application # Enables HTTP, gRPC, and SQL operation metrics - - application_span # Enables Trace spans for transactions - - network # Enables L3/L4 Network flow metrics + - application # HTTP, gRPC, and SQL operation metrics + - application_span # Trace spans for transactions + - network # L3/L4 network flow metrics discovery: services: @@ -44,51 +102,35 @@ discovery: network: enable: true - source: tc + source: socket_filter # avoids clashing with other tc-based eBPF, such as a Cilium CNI direction: both attributes: kubernetes: - enable: true # Set to true if running inside Kubernetes + enable: true # Set to true when running inside Kubernetes ``` -5. **Save** the configuration. The agent will automatically restart and apply the new settings. - -## Step 3: Verify Your Data - -Once the agent restarts with the eBPF configuration, it immediately begins observing kernel traffic. +Save the configuration. The agent picks up the change on its next check-in and restarts the collector for you. -Navigate to the following views in the KloudMate dashboard to verify data collection: -- **[APM](../../apm-and-tracing/apm-views/):** Look for automatically generated Service Maps, latency, and throughput metrics. -- **[Traces](../../apm-and-tracing/):** Search for automatically generated spans representing your network traffic. -- **[Database Activity Monitoring](../../database-monitoring/database-activity-monitoring-dam/):** Look for captured database queries. +## Verify your data ---- - -## Reference: Advanced Options - -### Advanced Database Tuning +Once the agent is collecting eBPF telemetry, confirm it in these views: -For high-load database environments, you can add an `ebpf` block to tune statement caching and heuristic detection. - -```yaml -ebpf: - heuristic_sql_detect: true - mysql_prepared_statements_cache_size: 1024 - postgres_prepared_statements_cache_size: 1024 -``` +- **[APM](../../apm-and-tracing/apm-views/):** look for the service map, latency, and throughput. +- **[Traces](../../apm-and-tracing/):** search for the spans generated from your traffic. +- **[Database Activity Monitoring](../../database-monitoring/database-activity-monitoring-dam/):** look for captured database queries. -### What eBPF Collects +## Troubleshooting -When the configuration above is applied, the eBPF receiver automatically collects: -- **Application Performance Metrics**: RED metrics (Request rate, Error rate, and Duration/latency). -- **Traces**: Automatically generated trace spans for your network traffic flows. -- **Service Dependencies**: Information used to build visual Service Maps. -- **Network Telemetry**: Low-level L3 and L4 network flow metrics. -- **Database Activity**: Kernel-level database query signals. +- **No eBPF metrics or service map:** check the host detail page for an "eBPF unavailable" reason. The most common causes are a kernel older than 4.14, `debugfs` not mounted, or missing privileges. +- **Metrics are present but sparse:** the host may be on a kernel between 4.14 and 5.8, or without BTF, so eBPF is running with reduced features. Host metrics and logs are unaffected. +- **Confirm the agent's own view:** the agent reports whether eBPF loaded through its self-health metrics, and surfaces load errors through its self-logs. See [Agent health metrics](../self-observability/health-metrics/) and [Troubleshooting](../troubleshooting/). -## Related Docs +## Related -- [Configuration & Management](../management/) -- [APM Views](../../apm-and-tracing/apm-views/) -- [Zero-Downtime Observability (eBPF) Concepts](../../apm-and-tracing/auto-instrumentation/ebpf/) + + + + + + diff --git a/src/content/docs/docs/kloudmate-agent/images/log-monitoring-add-source.png b/src/content/docs/docs/kloudmate-agent/images/log-monitoring-add-source.png new file mode 100644 index 00000000..9016eaee Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/images/log-monitoring-add-source.png differ diff --git a/src/content/docs/docs/kloudmate-agent/images/log-monitoring-preview-results.png b/src/content/docs/docs/kloudmate-agent/images/log-monitoring-preview-results.png new file mode 100644 index 00000000..0a2fa7ca Binary files /dev/null and b/src/content/docs/docs/kloudmate-agent/images/log-monitoring-preview-results.png differ diff --git a/src/content/docs/docs/kloudmate-agent/index.mdx b/src/content/docs/docs/kloudmate-agent/index.mdx index 1475a5bb..687fed63 100644 --- a/src/content/docs/docs/kloudmate-agent/index.mdx +++ b/src/content/docs/docs/kloudmate-agent/index.mdx @@ -1,6 +1,6 @@ --- title: "KloudMate Agent" -description: "Install and operate the KloudMate agent across Linux, Docker, Windows, and Kubernetes." +description: "Install the KloudMate agent once to automatically collect host, network, database, and application telemetry across Linux, Windows, Kubernetes, and Docker." sidebar: label: "Overview" order: 1 @@ -8,41 +8,54 @@ sidebar: import { LinkCard, CardGrid } from '@astrojs/starlight/components'; -The **KloudMate Agent** is a lightweight program that you install on your servers, containers, or clusters. It automatically gathers health data (logs, metrics, and traces) from your infrastructure and applications, and sends it to KloudMate so you can monitor your entire system. +The **KloudMate Agent** is a single program you install on your servers, containers, or clusters. Once running, it finds what you run, collects host metrics and logs, and turns on kernel-level observability. You can then add deeper application tracing one service at a time, all from the KloudMate web interface. -Under the hood, it is powered by an enhanced version of the industry-standard **OpenTelemetry (OTel) Collector** . We've bundled a curated set of components optimized for simplified deployment, remote configuration, and **eBPF-powered kernel-level observability**. +It is built on the **OpenTelemetry (OTel) Collector** with a curated set of components. You get simple deployment, remote configuration, and **extended Berkeley Packet Filter (eBPF)** observability without writing any collector configuration. -## Architecture +The agent is managed from the KloudMate web interface. When you change a setting, the agent applies it for you, so there is no need for Secure Shell (SSH) access or local file edits. Each agent's health and status appear in your dashboards. -The KloudMate Agent runs as a background service on your host system (like Linux or Windows), as a Docker container, or as a DaemonSet on Kubernetes. +To go deeper, read [Architecture](concepts/architecture/) and the [Configuration model](concepts/config-model/). -Rather than you having to manually manage complex OpenTelemetry configurations, the KloudMate Agent manages the lifecycle of the Collector for you. It maintains a secure connection to the KloudMate platform, allowing you to configure what telemetry data to collect directly from the web interface. When you make a change in the UI, the agent automatically detects the new configuration and safely restarts the underlying Collector. +## What the agent collects -Each agent is uniquely identifiable, so you can easily track its health and status from your KloudMate dashboards. +Some of what the agent collects is automatic. The rest needs a process restart, so you turn it on per service when you are ready. -## Key Problems Solved +Automatic, as soon as the agent is installed: -- **Complex Configuration**: Eliminates the steep learning curve of OpenTelemetry Collector configuration. -- **Manual Installation**: Provides automated, one-line installation scripts for multiple environments. -- **Configuration Management**: Enables remote configuration through a web interface without needing SSH access. -- **Blind Spots in Observability**: Using eBPF, get instant visibility across all services — no SDK integration, no code changes, no application restarts required. -- **Database Visibility**: Built-in Database Activity Monitoring (DAM) captures SQL operations, latencies, and throughput directly from the kernel. +- **Discovery:** a live inventory of the services, runtimes, and databases running on each host or cluster. +- **Host metrics and logs:** CPU, memory, disk, and network metrics plus system logs, with no configuration. +- **eBPF monitoring:** Rate, Errors, and Duration (RED) metrics, a network service map, Database Activity Monitoring (DAM), and trace-context propagation, with no code changes. -### Features +Opt in per service or database: -- **Automated Installation**: One-command deployment across Linux, Docker, Windows, and Kubernetes. -- **Remote Configuration**: Configure agents through the KloudMate web interface. -- **Lifecycle Management**: Comprehensive, automated management of the OpenTelemetry Collector. -- **Multi-Platform Support**: Native support for various deployment environments (including x86_64 and ARM64). -- **eBPF-Powered Observability**: Zero-code instrumentation for instant application, network, and database visibility. -- **Database Activity Monitoring (DAM)**: Automatic profiling of MySQL, PostgreSQL, and messaging systems like Redis and Kafka. +- **Application performance monitoring (APM):** distributed traces for Java, Node.js, Python, .NET, and PHP applications. +- **Database monitoring:** health metrics, query performance, and logs for your databases. -## Supported Environments +The agent also reports its own status and error logs. -- **Linux** (Debian/Ubuntu, RHEL/CentOS) — x86\_64 and ARM64 (aarch64) -- **Docker** (Host metrics and log collection) -- **Kubernetes** (via DaemonSet and Deployment) -- **Windows** (Windows Server 2016+) +## Supported platforms + +The agent runs across four deployment modes. Not every capability applies to every mode. eBPF depends on the Linux kernel, so it does not apply on Windows. There, Event Tracing for Windows (ETW) monitoring provides the equivalent view. + +| Capability | Linux / VM | Windows | Kubernetes | Docker | +|---|---|---|---|---| +| Service discovery | Yes | Yes | Yes | Yes | +| Host metrics and logs | Yes | Yes | Yes | Yes | +| eBPF monitoring (RED, service map, DAM) | Yes | Not applicable | Yes | Yes | +| Application APM (Java, Node.js, Python, .NET) | Yes | Yes | Yes | Via eBPF | +| Application APM (PHP) | Yes | Not applicable | Not applicable | Yes | +| Database monitoring | Yes | Yes | Yes | Yes | +| Web server monitoring (nginx, Apache) | Yes | Not applicable | Yes | Yes | +| Log monitoring (custom log files) | Yes | Yes | Not applicable | Yes | +| Agent health and self-logs | Yes | Yes | Yes | Yes | + +:::note +On Linux, eBPF monitoring needs kernel 4.14 or newer, and does more on 5.8 and above. On Docker, host and container metrics and logs are automatic; application tracing is opt-in per service (Off / eBPF). PHP containers are traced in place without a redeploy, and Java, Node.js, Python, and .NET use eBPF (no SDK injection on Docker). See [Docker platform notes](platform-notes/docker/). +::: + +## Install the agent + +Network egress rules and host privileges are what commonly block an install in enterprise environments. See [Before you start](installation/before-you-start/) to check both first. @@ -51,9 +64,17 @@ Each agent is uniquely identifiable, so you can easily track its health and stat -### Future Support +## Learn the concepts + + + + + + + +## Future support In future releases, the agent will also support: -- Mac -- OCI (Oracle Cloud Infrastructure) +- macOS +- Oracle Cloud Infrastructure (OCI) diff --git a/src/content/docs/docs/kloudmate-agent/installation/aws-asg-deployment.mdx b/src/content/docs/docs/kloudmate-agent/installation/aws-asg-deployment.mdx index bef2a64c..8d424d77 100644 --- a/src/content/docs/docs/kloudmate-agent/installation/aws-asg-deployment.mdx +++ b/src/content/docs/docs/kloudmate-agent/installation/aws-asg-deployment.mdx @@ -1,244 +1,200 @@ --- -title: "AWS Auto Scaling Group (ASG)" -description: "Documentation for AWS ASG Deployment" +title: "AWS Auto Scaling & Elastic Beanstalk" +description: "Run the KloudMate agent in autonomous auto-instrument mode on Auto Scaling groups and Elastic Beanstalk, so each new instance instruments itself with no per-instance UI step." sidebar: - order: 6 + order: 7 --- -Deploy `kmagent` across an Auto Scaling Group with zero manual per-instance work. Configure once in the Launch Template, every instance (existing + future scale-outs) gets the agent automatically. -## Prerequisites +Auto Scaling groups and Elastic Beanstalk environments create and replace instances automatically. The usual install flow assumes a stable host. You install the agent, then choose which apps to instrument in the KloudMate UI. That does not work when instances come and go. There is no lasting host to configure, and a setting made on one instance is not carried to the next. -- AWS CLI configured with appropriate permissions -- An existing ASG with a Launch Template -- IAM Instance Profile attached to your ASG instances -- SSM Agent running on instances (default on Amazon Linux / Ubuntu AMIs) +Autonomous auto-instrument mode removes the per-instance UI step. The agent discovers and instruments everything on the instance itself. You provide its configuration as files through the platform's provisioning step: user-data for an Auto Scaling group, or a `.platform` hook for Elastic Beanstalk. Your API key identifies the telemetry, so a new instance starts reporting as soon as it boots. -### Step 1: Store API Key in SSM Parameter Store +An Elastic Beanstalk environment runs on an Auto Scaling group underneath. The agent configuration is the same for both. Only the way you deliver it differs. -Never hardcode secrets in user data. Use SSM Parameter Store with encryption: +## What autonomous mode collects -```bash -aws ssm put-parameter \ - --name "/kloudmate/api-key" \ - --value "your-actual-km-api-key" \ - --type SecureString \ - --region ap-south-1 -``` +With auto-instrument turned on, every instance gets the following, with no UI step: -### Step 2: IAM Permissions +- **eBPF monitoring.** HTTP server spans, RED metrics, and the service map for every app, whatever the language, with no code change. It's also the only tracing method for runtimes with no SDK injection, such as Ruby and Go. See [eBPF observability](../../ebpf-observability/). +- **SDK auto-instrumentation.** Deeper, in-process traces for Java, .NET, Node.js, Python, and PHP. The agent injects the SDK through a systemd config file and restarts each app once to pick it up. +- **Host metrics and logs.** The metrics the agent always collects, plus any log files you configure below. -Ensure your ASG's IAM Instance Profile has permission to read the SSM parameter: +Each app's `service.name` comes from its own stable identity, not from the instance. So every instance of one app rolls up as a single service in KloudMate, and a per-instance breakdown is still available. See [Telemetry identity](../../reference/telemetry-identity/). -```bash -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": "ssm:GetParameter", - "Resource": "arn:aws:ssm:ap-south-1:YOUR_ACCOUNT_ID:parameter/kloudmate/*" - } - ] -} -``` +## Before you start -If using **SSM Run Command** (Step 5, Option B), also add: +You need: -```bash -{ - "Effect": "Allow", - "Action": [ - "ssm:SendCommand", - "ssm:ListCommandInvocations" - ], - "Resource": "*" -} -``` +- Your **KloudMate API key**. +- The **collector endpoint**, `https://otel.kloudmate.com:4318`. +- Instances on a supported Linux (Amazon Linux 2 or 2023, Ubuntu, Debian, or the RHEL family). eBPF monitoring needs a modern kernel, which the standard AWS AMIs have. +- Network access and permissions from [Before you start](../before-you-start/) if your outbound traffic is restricted. + +Both examples below put the API key directly into the provisioning step. To keep it out of the launch template, read it from SSM Parameter Store or Secrets Manager instead and set `KM_API_KEY` from there. That is the only line that changes. -### Step 3: Create the User Data Script +## Auto Scaling group -Save this as `userdata.sh`: +Put this in the launch template's **user-data**. It writes the agent configuration, installs the agent with autonomous mode on, and starts it. Running the script again changes nothing, so it's safe to run on every instance launch. ```bash #!/bin/bash set -euo pipefail -# ----------------------------------------------- -# Fetch API key from SSM (IMDSv2 compatible) -# ----------------------------------------------- -REGION=$(curl -sf http://169.254.169.254/latest/meta-data/placement/region) -TOKEN=$(curl -sf -X PUT "http://169.254.169.254/latest/api/token" \ - -H "X-aws-ec2-metadata-token-ttl-seconds: 60") - -KM_API_KEY=$(aws ssm get-parameter \ - --name "/kloudmate/api-key" \ - --with-decryption \ - --query "Parameter.Value" \ - --output text \ - --region "$REGION") - -# ----------------------------------------------- -# Install kmagent -# ----------------------------------------------- -curl -fsSL https://raw.githubusercontent.com/kloudmate/km-agent/main/install.sh \ - | bash -s -- \ - --api-key "$KM_API_KEY" \ - --colector-endpoint "https://otel.kloudmate.com" - -# ----------------------------------------------- -# Verify -# ----------------------------------------------- -if systemctl is-active --quiet kmagent; then - echo "[kloudmate] Agent installed and running" -else - echo "[kloudmate] ERROR: Agent failed to start" - journalctl -u kmagent --no-pager -n 20 - exit 1 -fi +export KM_API_KEY="YOUR_API_KEY" +export KM_COLLECTOR_ENDPOINT="https://otel.kloudmate.com:4318" +export KM_AUTO_INSTRUMENT=true # autonomous from the first start: no UI step, no per-instance registration + +# 1) Optional log collection: one YAML list of sources per file, read when the agent starts (see below). +install -d -m 755 /etc/kmagent/logs.d +cat > /etc/kmagent/logs.d/app.yaml <<'EOF' +- name: my-app + include: [ /var/log/my-app/*.log ] + format: json + parsing: { timestamp_field: ts, severity_field: level, message_field: msg } + attributes: { service.name: my-app } +EOF + +# 2) Install and start. The installer detects the OS, pulls the package, and sets up the systemd service. +curl -fsSL https://cdn.kloudmate.com/scripts/install_linux.sh | bash ``` -### Step 4: Apply User Data to Launch Template +## Elastic Beanstalk +Elastic Beanstalk runs a provisioning hook on each instance. On the current Amazon Linux 2 and 2023 platforms, use a platform hook. Your environment properties are available there. -Via AWS CLI +**1. Set the environment properties** on the environment (Console → Configuration → Software → Environment properties, or `eb setenv`): - ```bash - # Create a new Launch Template version with user data - aws ec2 create-launch-template-version \ - --launch-template-name your-template-name \ - --source-version '$Latest' \ - --launch-template-data '{ - "UserData": "'$(base64 -w0 userdata.sh)'" - }' - - # Point ASG to use the latest version - aws autoscaling update-auto-scaling-group \ - --auto-scaling-group-name your-asg-name \ - --launch-template "LaunchTemplateName=your-template-name,Version=\$Latest" - ``` +```text +KM_API_KEY = YOUR_API_KEY +KM_COLLECTOR_ENDPOINT = https://otel.kloudmate.com:4318 +``` - Via AWS Console +**2. Add a predeploy hook** to your application source bundle at `.platform/hooks/predeploy/01-kmagent.sh`, and make it executable with `chmod +x`: - ```none - Go to EC2 → Launch Templates → your-template - Click Actions → Modify template (Create new version) - Scroll to Advanced details → User data - Paste the contents of userdata.sh - Click Create template version - Go to Auto Scaling Groups → your-asg - Click Edit → Launch Template → Version → Latest - Save - ``` +```bash +#!/bin/bash +set -euo pipefail +# Bring the Elastic Beanstalk environment properties into the shell. +[ -f /opt/elasticbeanstalk/deployment/env ] && { set -a; . /opt/elasticbeanstalk/deployment/env; set +a; } +: "${KM_API_KEY:?set KM_API_KEY as an Elastic Beanstalk environment property}" +: "${KM_COLLECTOR_ENDPOINT:=https://otel.kloudmate.com:4318}" +export KM_AUTO_INSTRUMENT=true + +# 1) Log collection. nginx logs every request to /var/log/nginx/access.log — the standard, app-agnostic +# source on the web tier. Add more sources under /etc/kmagent/logs.d/. +install -d -m 755 /etc/kmagent/logs.d +cat > /etc/kmagent/logs.d/beanstalk.yaml <<'EOF' +- name: nginx-access + include: [ /var/log/nginx/access.log ] + format: regex + parsing: + regex: '^(?P\S+) - (?P\S+) \[(?P