diff --git a/docs/examples/ignite/monitoring/ignite-grafana-demo.yaml b/docs/examples/ignite/monitoring/ignite-grafana-demo.yaml new file mode 100644 index 0000000000..0f40453356 --- /dev/null +++ b/docs/examples/ignite/monitoring/ignite-grafana-demo.yaml @@ -0,0 +1,27 @@ +apiVersion: kubedb.com/v1alpha2 +kind: Ignite +metadata: + name: ignite-grafana-demo + namespace: demo +spec: + version: "2.17.0" + replicas: 3 + deletionPolicy: WipeOut + podTemplate: + spec: + containers: + - name: ignite + resources: + limits: + cpu: 500m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s diff --git a/docs/examples/mongodb/monitoring/mg-grafana-demo.yaml b/docs/examples/mongodb/monitoring/mg-grafana-demo.yaml new file mode 100644 index 0000000000..73a709f800 --- /dev/null +++ b/docs/examples/mongodb/monitoring/mg-grafana-demo.yaml @@ -0,0 +1,25 @@ +apiVersion: kubedb.com/v1 +kind: MongoDB +metadata: + name: mg-grafana-demo + namespace: demo +spec: + version: "8.0.17" + replicas: 3 + replicaSet: + name: rs0 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s diff --git a/docs/examples/redis/monitoring/redis-cluster.yaml b/docs/examples/redis/monitoring/redis-cluster.yaml new file mode 100644 index 0000000000..d4f2b23a16 --- /dev/null +++ b/docs/examples/redis/monitoring/redis-cluster.yaml @@ -0,0 +1,27 @@ +apiVersion: kubedb.com/v1 +kind: Redis +metadata: + name: redis-cluster + namespace: demo +spec: + version: 8.2.2 + mode: Cluster + cluster: + shards: 3 + replicas: 2 + storageType: Durable + storage: + resources: + requests: + storage: 1Gi + storageClassName: local-path + accessModes: + - ReadWriteOnce + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s diff --git a/docs/guides/cassandra/monitoring/grafana-dashboard.md b/docs/guides/cassandra/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..3b306f2482 --- /dev/null +++ b/docs/guides/cassandra/monitoring/grafana-dashboard.md @@ -0,0 +1,345 @@ +--- +title: Visualize Cassandra Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-cas-grafana-dashboard + name: Grafana Dashboard + parent: cas-monitoring-cassandra + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Cassandra Metrics with Grafana Dashboard + +KubeDB exposes Cassandra metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Cassandra instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/cassandra/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/cassandra/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-cassandra-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Cassandra instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Cassandra with Monitoring Enabled + +Below is the Cassandra object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Cassandra +metadata: + name: cassandra-grafana-demo + namespace: demo +spec: + version: "5.0.3" + topology: + rack: + - name: r0 + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageType: Durable + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Cassandra instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/cassandra/monitoring/cassandra-grafana-demo.yaml +cassandra.kubedb.com/cassandra-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get cassandra -n demo cassandra-grafana-demo +NAME VERSION STATUS AGE +cassandra-grafana-demo 5.0.3 Ready 3m +``` + +KubeDB creates a stats service named `{cassandra-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=cassandra-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +cassandra-grafana-demo ClusterIP 10.96.10.1 9042/TCP 3m +cassandra-grafana-demo-stats ClusterIP 10.96.10.2 56790/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +cassandra-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo cassandra-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `cassandra-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Cassandra Dashboards + +The KubeDB Cassandra dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/cassandra) repository (`cassandra/` folder): + +| File | Dashboard | +|------|-----------| +| `cassandra_summary_dashboard.json` | KubeDB / Cassandra / Summary | +| `cassandra_databases_dashboard.json` | KubeDB / Cassandra / Database | +| `cassandra_pod_dashboard.json` | KubeDB / Cassandra / Pod | + +**Import steps (repeat for each file):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing both files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|-------------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Cassandra is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `cassandra-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / Cassandra / Summary** — cluster-level overview: +- **Database Status** — current health of the Cassandra cluster +- **Version** — Cassandra version running +- **Total Nodes** — number of nodes in the rack +- **Read / Write Latency** — p99 read and write request latency +- **Compaction Tasks** — pending and completed compaction tasks +- **Heap Usage** — JVM heap used vs. max across nodes +- **CPU / Network** — resource usage over time + +

+ KubeDB Cassandra Summary Dashboard +

+ +**KubeDB / Cassandra / Database** — table and keyspace metrics: +- **SSTable Count** — number of SSTables per table +- **Disk Usage** — disk space consumed per keyspace +- **Live / Tombstone Cells** — ratio of live cells to tombstones (high tombstone ratio indicates GC pressure) +- **Bloom Filter False Positive Rate** — effectiveness of bloom filters per table + +

+ KubeDB Cassandra Database Dashboard +

+ +**KubeDB / Cassandra / Pod** — per-pod operational metrics, scoped to a single Cassandra pod: +- **Node status** — whether the selected pod's Cassandra node is up +- **Cluster ops** — operations per second broken down by type (Read, Write, CasRead, CasWrite, RangeSlice, ViewWrite) +- **Operations per instance** — operations per second aggregated per pod +- **Read / Write operations** — read and write throughput for the selected pod + +

+ KubeDB Cassandra Pod Dashboard +

+ +## Cleaning up + +```bash +# Remove the Cassandra instance +kubectl delete cassandra -n demo cassandra-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Cassandra database with KubeDB using [built-in Prometheus](/docs/guides/cassandra/monitoring/using-builtin-prometheus.md). +- Monitor your Cassandra database with KubeDB using [Prometheus Operator](/docs/guides/cassandra/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/clickhouse/monitoring/grafana-dashboard.md b/docs/guides/clickhouse/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..fdf9166582 --- /dev/null +++ b/docs/guides/clickhouse/monitoring/grafana-dashboard.md @@ -0,0 +1,365 @@ +--- +title: Visualize ClickHouse Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: ch-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: ch-monitoring-clickhouse + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize ClickHouse Metrics with Grafana Dashboard + +KubeDB exposes ClickHouse metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a ClickHouse instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/clickhouse/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/clickhouse/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-clickhouse-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the ClickHouse instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy ClickHouse with Monitoring Enabled + +Below is the ClickHouse object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: ClickHouse +metadata: + name: ch-grafana-demo + namespace: demo +spec: + version: "26.2.6" + deletionPolicy: WipeOut + storage: + storageClassName: "local-path" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the ClickHouse instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/clickhouse/monitoring/coreos-prom-clickhouse.yaml +clickhouse.kubedb.com/ch-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get clickhouse -n demo ch-grafana-demo +NAME VERSION STATUS AGE +ch-grafana-demo 26.2.6 Ready 2m +``` + +KubeDB creates a stats service named `{clickhouse-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=ch-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +ch-grafana-demo ClusterIP 10.96.10.1 9000/TCP 2m +ch-grafana-demo-stats ClusterIP 10.96.10.2 9363/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +ch-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo ch-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `ch-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB ClickHouse Dashboard + +The KubeDB ClickHouse dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/clickhouse) repository (`clickhouse/` folder): + +| File | Dashboard | +|------|-----------| +| `clickhouse_summary_dashboard.json` | KubeDB / ClickHouse / Summary | +| `clickhouse_pods_dashboard.json` | KubeDB / ClickHouse / Pod | +| `clickhouse_databases_dashboard.json` | KubeDB / ClickHouse / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / ClickHouse / Summary | Cluster health, queries/sec, memory usage, disk read/write, CPU/storage | +| KubeDB / ClickHouse / Pod | Per-pod queries, memory, CPU, merge operations, replicated fetches | +| KubeDB / ClickHouse / Database | Table-level insert/select rates, part count, mutations, replication queue | + +## Step 6: Explore the Dashboard + +After opening a dashboard, you will see dropdown filters at the top. These control which data is shown across all panels — change them to focus on a specific instance without editing any queries. + +| Variable | Applies to | What to select | +|---------------|-------------------------|--------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your ClickHouse is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your ClickHouse instance (e.g., `ch-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` to see aggregated view | + +Once you set these, all panels update automatically. Below is what each dashboard shows: + +**KubeDB / ClickHouse / Summary** — start here for a cluster health overview: +- **Queries per Second** — total query throughput across the cluster +- **Memory Usage** — current memory consumption vs. configured limits +- **Merged Parts** — background merge operations per second (health indicator) +- **Inserted Rows / Bytes** — data ingestion rate +- **Replication Queue** — pending replicated operations (should be near zero in healthy clusters) +- **CPU / Storage** — resource consumption vs. requests and limits + +

+ KubeDB ClickHouse Summary Dashboard +

+ +**KubeDB / ClickHouse / Pod** — drill into a specific pod: +- **Queries** — queries executed on this specific pod +- **Memory** — per-pod memory usage over time +- **CPU** — per-pod CPU usage +- **Merge Operations** — background merges on this pod +- **Replicated Fetches** — data fetched from other replicas + +

+ KubeDB ClickHouse Pod Dashboard +

+ +**KubeDB / ClickHouse / Database** — service health and query-level metrics, grouped into three collapsible sections: + +*General* — service reachability and cluster role: +- **Service Status / Service Uptime** — whether the exporter endpoint is UP, and how long the pod has been running +- **Healthy Pods Count** — number of ClickHouse pods currently healthy +- **Cluster Status** — this pod's role (`Primary` or replica) +- **Current QPS / Active ClickHouse Connections** — live query rate and open client connections +- **ClickHouse Metrics reads vs writes** — read vs. write operation rate +- **ClickHouse network received vs sent** — network throughput in/out + +

+ KubeDB ClickHouse Database Dashboard - General +

+ +*Service Health* and *Queries Overview* — cache efficiency and query success rates: +- **ClickHouse Opened File Cache Hits / File Cache Miss Rate** — file cache effectiveness +- **Max Parts Per Partition** — highest number of active data parts in any partition (high counts may need compaction) +- **Percent of Failed SELECTs / INSERTs**, **logged errors** — error rates (should stay at 0) +- **ClickHouse Cluster Query Rate** — total queries per second across the cluster +- **Queries with MEMORY_LIMIT_EXCEEDED**, **Failed Queries**, **Failed SELECT Queries**, **Failed Asynchronous INSERT Queries** — failure counters (should stay at 0) +- **SELECT Queries**, **Synchronous / Asynchronous INSERT Queries** — query throughput by type + +

+ KubeDB ClickHouse Database Dashboard - Service Health and Queries +

+ +## Cleaning up + +```bash +# Remove the ClickHouse instance +kubectl delete clickhouse -n demo ch-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your ClickHouse database with KubeDB using [built-in Prometheus](/docs/guides/clickhouse/monitoring/using-builtin-prometheus.md). +- Monitor your ClickHouse database with KubeDB using [Prometheus Operator](/docs/guides/clickhouse/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/druid/monitoring/grafana-dashboard.md b/docs/guides/druid/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..d5001a7a3f --- /dev/null +++ b/docs/guides/druid/monitoring/grafana-dashboard.md @@ -0,0 +1,391 @@ +--- +title: Visualize Druid Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-druid-grafana-dashboard + name: Grafana Dashboard + parent: guides-druid-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Druid Metrics with Grafana Dashboard + +KubeDB exposes Druid metrics through a JMX Exporter running as a Java agent inside each Druid container. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Druid instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- Druid requires a deep storage backend and ZooKeeper. The example below uses S3-compatible deep storage with a pre-created secret. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/druid/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/druid/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-druid-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Druid instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Get External Dependencies Ready + +### Deep Storage + +One of the external dependency of Druid is deep storage where the segments are stored. It is a storage mechanism that Apache Druid does not provide. **Amazon S3**, **Google Cloud Storage**, or **Azure Blob Storage**, **S3-compatible storage** (like **Minio**), or **HDFS** are generally convenient options for deep storage. + +In this tutorial, we will run a `minio-server` as deep storage in our local `kind` cluster using `minio-operator` and create a bucket named `druid` in it, which the deployed druid database will use. + +```bash + +$ helm repo add minio https://operator.min.io/ +$ helm repo update minio +$ helm upgrade --install --namespace "minio-operator" --create-namespace "minio-operator" minio/operator --set operator.replicaCount=1 + +$ helm upgrade --install --namespace "demo" --create-namespace druid-minio minio/tenant \ +--set tenant.pools[0].servers=1 \ +--set tenant.pools[0].volumesPerServer=1 \ +--set tenant.pools[0].size=1Gi \ +--set tenant.certificate.requestAutoCert=false \ +--set tenant.buckets[0].name="druid" \ +--set tenant.pools[0].name="default" + +``` + +Now we need to create a `Secret` named `deep-storage-config`. It contains the necessary connection information using which the druid database will connect to the deep storage. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: deep-storage-config + namespace: demo +stringData: + druid.storage.type: "s3" + druid.storage.bucket: "druid" + druid.storage.baseKey: "druid/segments" + druid.s3.accessKey: "minio" + druid.s3.secretKey: "minio123" + druid.s3.protocol: "http" + druid.s3.enablePathStyleAccess: "true" + druid.s3.endpoint.signingRegion: "us-east-1" + druid.s3.endpoint.url: "http://myminio-hl.demo.svc.cluster.local:9000/" +``` + +Let’s create the `deep-storage-config` Secret shown above: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/druid/quickstart/deep-storage-config.yaml +secret/deep-storage-config created +``` +Below is the Druid object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Druid +metadata: + name: druid-grafana-demo + namespace: alert-druid +spec: + version: 36.0.0 + deepStorage: + type: s3 + configSecret: + name: deep-storage-config + topology: + routers: + replicas: 1 + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Druid instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/druid/monitoring/druid-grafana-demo.yaml +druid.kubedb.com/druid-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get druid -n demo druid-grafana-demo +NAME VERSION STATUS AGE +druid-grafana-demo 36.0.0 Ready 5m +``` + +KubeDB creates a stats service named `{druid-name}-stats` for monitoring: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=druid-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +druid-grafana-demo ClusterIP 10.96.10.1 8888/TCP 5m +druid-grafana-demo-stats ClusterIP 10.96.10.2 9101/TCP 5m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +druid-grafana-demo-stats 5m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo druid-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `druid-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 3000 +Forwarding from [::1]:3000 -> 3000 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Druid Dashboard + +The KubeDB Druid dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/druid) repository (`druid/` folder): + +| File | Dashboard | +|------|-----------| +| `druid_summary_dashboard.json` | KubeDB / Druid / Summary | +| `druid_pods_dashboard.json` | KubeDB / Druid / Pod | +| `druid_databases_dashboard.json` | KubeDB / Druid / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / Druid / Summary | Cluster-wide status, node/resource sizing, and CPU usage across all pods | +| KubeDB / Druid / Pod | Per-pod status, ZooKeeper connectivity, and JVM memory/GC metrics | +| KubeDB / Druid / Database | Cluster status, datasource/segment counts, task success, and JVM memory | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. Note that the Summary dashboard labels its filters **Namespace** and **Druid**, while the Pod and Database dashboards use lowercase **namespace** and **app** — they select the same thing. + +| Variable | Applies to | What to select | +|--------------------------|---------------------------|------------------------------------------------------------| +| **Namespace** / **namespace** | All dashboards | Namespace where your Druid is deployed (e.g., `demo`) | +| **Druid** / **app** | All dashboards | Name of your Druid instance (e.g., `druid-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, e.g. a coordinator, broker, or historical | + +**KubeDB / Druid / Summary** — start here for a cluster-level overview: +- **General Info** — database status, version, secure-transport flag, termination policy, total node count, and aggregate CPU/memory/storage requests and limits +- **CPU Info** — a CPU usage graph broken down per pod, plus a CPU Quota table showing usage vs. requests (and % of request) for each pod + +

+ KubeDB Druid Summary Dashboard +

+ +**KubeDB / Druid / Pod** — drill into a specific Druid node (selected via the **pod** filter): +- **Druid Overview** — node status (UP/DOWN) and ZooKeeper connection state for the selected pod +- **JVM Overview** — JVM memory used, JVM memory pool, JVM bufferpool count, and JVM GC CPU time for the selected pod + +

+ KubeDB Druid Pod Dashboard +

+ +**KubeDB / Druid / Database** — cluster-wide segment and task metrics: +- **Druid Overview** — Druid status, ZooKeeper connection state, total datasources, unloaded segments (count and size), successful tasks, and total segment size +- **JVM Overview** — JVM memory used and JVM memory pool, aggregated across pods + +

+ KubeDB Druid Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Druid instance +kubectl delete druid -n demo druid-grafana-demo + +# Remove the deep storage secret +kubectl delete secret deep-storage-config -n demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Druid instance with KubeDB using [Prometheus Operator](/docs/guides/druid/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/elasticsearch/monitoring/grafana-dashboard.md b/docs/guides/elasticsearch/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..7597762f5e --- /dev/null +++ b/docs/guides/elasticsearch/monitoring/grafana-dashboard.md @@ -0,0 +1,369 @@ +--- +title: Visualize Elasticsearch Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: es-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: es-monitoring-elasticsearch + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Elasticsearch Metrics with Grafana Dashboard + +KubeDB exposes Elasticsearch metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on an Elasticsearch instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `grafana-es` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns grafana-es + namespace/grafana-es created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/elasticsearch/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/elasticsearch/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-elasticsearch-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Elasticsearch instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Elasticsearch with Monitoring Enabled + +Below is the Elasticsearch object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: Elasticsearch +metadata: + name: es-grafana-topo + namespace: grafana-es +spec: + version: "xpack-9.2.3" + topology: + master: + replicas: 2 + storage: + storageClassName: "local-path" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + data: + replicas: 3 + storage: + storageClassName: "local-path" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + ingest: + replicas: 2 + storage: + storageClassName: "local-path" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Elasticsearch instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/monitoring/coreos-prom-es.yaml +elasticsearch.kubedb.com/es-grafana-topo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get elasticsearch -n grafana-es es-grafana-topo +NAME VERSION STATUS AGE +es-grafana-topo xpack-9.2.3 Ready 83m +``` + +KubeDB creates a stats service named `{elasticsearch-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n grafana-es --selector="app.kubernetes.io/instance=es-grafana-topo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +es-grafana-topo ClusterIP 10.43.201.71 9200/TCP 84m +es-grafana-topo-master ClusterIP None 9300/TCP 84m +es-grafana-topo-pods ClusterIP None 9200/TCP 84m +es-grafana-topo-stats ClusterIP 10.43.18.99 56790/TCP 84m +``` + +KubeDB also creates a `ServiceMonitor` in the `grafana-es` namespace: + +```bash +$ kubectl get servicemonitor -n grafana-es +NAME AGE +es-grafana-topo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n grafana-es es-grafana-topo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `es-grafana-topo-stats`. Its state should be **UP**. + +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +

+ Prometheus Target +

+ +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Elasticsearch Dashboard + +The KubeDB Elasticsearch dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/elasticsearch) repository (`elasticsearch/` folder): + +| File | Dashboard | +|------|-----------| +| `elasticsearch_summary_dashboard.json` | KubeDB / Elasticsearch / Summary | +| `elasticsearch_pods_dashboard.json` | KubeDB / Elasticsearch / Pod | +| `elasticsearch_databases_dashboard.json` | KubeDB / Elasticsearch / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---------------------------------------|------------------------------------------------------------------------------------------| +| KubeDB / Elasticsearch / Summary | Cluster health, shard status, JVM heap usage, CPU/memory/storage, network | +| KubeDB / Elasticsearch / Pod | Per-node JVM heap, GC time, thread pool queues and rejections, CPU/memory usage | +| KubeDB / Elasticsearch / Database | Index-level indexing rate, search rate, search latency, field data cache, segment count | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|----------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Elasticsearch is deployed (e.g., `grafana-es`) | +| **app** | All dashboards | Name of your Elasticsearch instance (e.g., `es-grafana-topo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` for an aggregated view | +| **index** | Database dashboard only | A specific index, or `All` | + +**KubeDB / Elasticsearch / Summary** — start here for a cluster health overview: +- **Cluster Health** — green/yellow/red status, active shards, relocating shards, unassigned shards +- **Node Count** — total nodes in the cluster +- **JVM Heap Used %** — aggregate heap usage across all nodes +- **CPU / Memory / Storage** — resource consumption vs. requests and limits +- **Network** — receive and transmit bandwidth + +

+ KubeDB Elasticsearch Summary Dashboard +

+ +**KubeDB / Elasticsearch / Pod** — drill into a specific node: +- **JVM Heap** — used vs. max heap per node +- **GC Collection Time** — time spent in young/old generation GC +- **Thread Pool** — queue size and rejected count per thread pool (search, index, bulk) +- **CPU / Memory** — per-pod resource usage over time + +

+ KubeDB Elasticsearch Pod Dashboard +

+ +**KubeDB / Elasticsearch / Database** — index-level metrics: +- **Indexing Rate** — documents indexed per second +- **Search Rate** — queries executed per second +- **Search Latency** — p50/p95/p99 query latency +- **Field Data Cache Evictions** — high eviction rates indicate memory pressure +- **Segment Count** — number of Lucene segments per index + +

+ KubeDB Elasticsearch Database Dashboard +

+## Cleaning up + +```bash +# Remove the Elasticsearch instance +kubectl delete elasticsearch -n grafana-es es-grafana-topo + +# Remove namespaces +kubectl delete ns grafana-es + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Elasticsearch database with KubeDB using [built-in Prometheus](/docs/guides/elasticsearch/monitoring/using-builtin-prometheus.md). +- Monitor your Elasticsearch database with KubeDB using [Prometheus Operator](/docs/guides/elasticsearch/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/hazelcast/monitoring/grafana-dashboard.md b/docs/guides/hazelcast/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..922e87066d --- /dev/null +++ b/docs/guides/hazelcast/monitoring/grafana-dashboard.md @@ -0,0 +1,402 @@ +--- +title: Visualize Hazelcast Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-hz-grafana-dashboard + name: Grafana Dashboard + parent: hz-monitoring-hazelcast + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Hazelcast Metrics with Grafana Dashboard + +KubeDB exposes Hazelcast metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Hazelcast instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- Hazelcast requires a valid license secret. Create the license secret before deploying: + + ```bash + $ kubectl create secret generic hz-license-key -n demo \ + --from-literal=license= + ``` + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/hazelcast/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/hazelcast/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-hazelcast-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Hazelcast instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Hazelcast with Monitoring Enabled + +Below is the Hazelcast object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Hazelcast +metadata: + name: hz-grafana-demo + namespace: demo +spec: + version: "5.5.2" + replicas: 3 + licenseSecret: + name: hz-license-key + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Hazelcast instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/hazelcast/monitoring/hz-grafana-demo.yaml +hazelcast.kubedb.com/hz-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get hazelcast -n demo hz-grafana-demo +NAME VERSION STATUS AGE +hz-grafana-demo 5.5.2 Ready 3m +``` + +KubeDB creates a stats service named `{hazelcast-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=hz-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +hz-grafana-demo ClusterIP 10.96.10.1 5701/TCP 3m +hz-grafana-demo-stats ClusterIP 10.96.10.2 56790/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +hz-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo hz-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `hz-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Hazelcast Dashboards + +The KubeDB Hazelcast dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/hazelcast) repository (`hazelcast/` folder): + +| File | Dashboard | +|------|-----------| +| `hazelcast_summary_dashboard.json` | KubeDB / Hazelcast / Summary | +| `hazelcast_pods_dashboard.json` | KubeDB / Hazelcast / Pod | +| `hazelcast_databases_dashboard.json` | KubeDB / Hazelcast / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|--------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Hazelcast is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `hz-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +### KubeDB / Hazelcast / Summary + +Cluster-level overview showing general health, resource usage, storage, and network metrics. + +**General Info section:** +- **Database Status** — current health (`Ready`, `NotReady`, etc.) +- **Database Up-time** — how long the cluster has been running +- **Version** — Hazelcast version (e.g., `5.5.2`) +- **Total Nodes** — number of active member pods +- **CPU / Memory Request & Limit** — configured resource bounds per pod +- **Deletion Policy** — KubeDB deletion policy (e.g., `WipeOut`) + +**CPU Info section:** +- **CPU Usage** — per-pod CPU usage over time +- **CPU Quota** — table of CPU requests, limits, and usage percentages per pod + +

+ KubeDB Hazelcast Summary Dashboard - General Info and CPU +

+ +**Memory Info section:** +- **Memory Usage** — RSS memory per pod over time +- **Memory Quota** — table of memory requests, limits, and usage percentages per pod + +**Storage Info section:** +- **Disk Usage** — per-pod disk write growth over time +- **Disk R/W Info** — read/write throughput per pod + +

+ KubeDB Hazelcast Summary Dashboard - Memory and Storage +

+ +**Network Info section:** +- **Persistent Volume Usage History** — PV capacity vs used over time +- **Receive Bandwidth** — inbound network traffic per pod +- **Transmit Bandwidth** — outbound network traffic per pod + +

+ KubeDB Hazelcast Summary Dashboard - PV and Network +

+ +### KubeDB / Hazelcast / Pod + +Per-member drill-down. Use the **pod** dropdown to select a specific `hz-grafana-demo-N` pod. + +**Hazelcast Memory Consumption section:** +- **Partition Counts** — number of partitions owned by this member +- **System Load** — CPU system load average over time +- **Heap Used / Max** — JVM heap in use vs. configured maximum +- **Physical Usage / Max** — OS physical memory used vs. total +- **Non-Heap Used / Max** — JVM non-heap (metaspace, code cache) used vs. max +- **Free Physical Memory** — OS free memory over time +- **Memory Usage** — owned, backup, and heap-cost entry memory per map + +

+ KubeDB Hazelcast Pod Dashboard - Memory Consumption +

+ +**Hazelcast Operate Rate PerMinute section:** +- **Put Rate Per Minute** — map put operations per minute on this pod +- **Get Rate Per Minute** — map get operations per minute on this pod +- **Average Get Times** — mean latency for get operations +- **Remove Rate Per Minute** — map remove operations per minute on this pod + +

+ KubeDB Hazelcast Pod Dashboard - Operate Rate +

+ +### KubeDB / Hazelcast / Database + +Cluster-wide map and memory metrics across all members. + +**Hazelcast Server section:** +- **Active Members** — total cluster member count +- **Cluster Version** — Hazelcast cluster protocol version +- **Client Connections** — number of connected Hazelcast clients +- **Partition Counts** — gauge per pod showing owned partitions + +**Hazelcast Memory Usage section:** +- **System Load** — system load over time across members +- **Heap Usage** — JVM heap usage gauge per member pod +- **Owned Entry Memory Cost** — heap memory consumed by owned map entries +- **Backup Entry Memory Cost** — heap memory consumed by backup entries + +

+ KubeDB Hazelcast Database Dashboard - Server Stats and Memory +

+ +**Hazelcast Operation Latency section:** +- **Heap Cost** — total heap cost per map across all members +- **GetLatency and GetCount** — get operation latency and throughput +- **PutLatency and PutCount** — put operation latency and throughput + +

+ KubeDB Hazelcast Database Dashboard - Operation Latency +

+ +## Cleaning up + +```bash +# Remove the Hazelcast instance +kubectl delete hazelcast -n demo hz-grafana-demo + +# Remove the license secret +kubectl delete secret hz-license-key -n demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Hazelcast database with KubeDB using [built-in Prometheus](/docs/guides/hazelcast/monitoring/prometheus-builtin.md). +- Monitor your Hazelcast database with KubeDB using [Prometheus Operator](/docs/guides/hazelcast/monitoring/prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/ignite/monitoring/grafana-dashboard.md b/docs/guides/ignite/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..d775e1f95c --- /dev/null +++ b/docs/guides/ignite/monitoring/grafana-dashboard.md @@ -0,0 +1,345 @@ +--- +title: Visualize Ignite Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-ig-grafana-dashboard + name: Grafana Dashboard + parent: ig-monitoring-ignite + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Ignite Metrics with Grafana Dashboard + +KubeDB exposes Ignite metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on an Ignite instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/ignite/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/ignite/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-ignite-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Ignite instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Ignite with Monitoring Enabled + +Below is the Ignite object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Ignite +metadata: + name: ignite-grafana-demo + namespace: demo +spec: + version: "2.17.0" + replicas: 3 + deletionPolicy: WipeOut + podTemplate: + spec: + containers: + - name: ignite + resources: + limits: + cpu: 500m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Ignite instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/ignite/monitoring/ignite-grafana-demo.yaml +ignite.kubedb.com/ignite-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get ignite -n demo ignite-grafana-demo +NAME VERSION STATUS AGE +ignite-grafana-demo 2.17.0 Ready 2m +``` + +KubeDB creates a stats service named `{ignite-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=ignite-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +ignite-grafana-demo ClusterIP 10.96.10.1 10800/TCP 2m +ignite-grafana-demo-stats ClusterIP 10.96.10.2 56790/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +ignite-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo ignite-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `ignite-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Ignite Dashboards + +The KubeDB Ignite dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/ignite) repository (`ignite/` folder): + +| File | Dashboard | +|------|-----------| +| `ignite_summary_dashboard.json` | KubeDB / Ignite / Summary | +| `ignite_pods_dashboard.json` | KubeDB / Ignite / Pod | +| `ignite_databases_dashboard.json` | KubeDB / Ignite / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|-------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Ignite is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `ignite-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / Ignite / Summary** — cluster-level overview: +- **Database Status** — current health of the Ignite cluster +- **Version** — Ignite version running +- **Cluster Nodes** — number of active nodes in the topology +- **Compute Tasks** — submitted and completed compute jobs +- **Heap / Off-Heap Memory** — JVM heap and data region memory usage +- **CPU / Network** — resource usage over time + +

+ KubeDB Ignite Summary Dashboard +

+ +**KubeDB / Ignite / Pod** — per-node drill-down: +- **Uptime** — how long this node has been running +- **Cache Operations** — put/get/remove rates on this node +- **Data Region Usage** — memory used vs. total for each data region +- **CPU / Memory** — per-pod resource usage + +

+ KubeDB Ignite Pod Dashboard +

+ +**KubeDB / Ignite / Database** — cache and table metrics: +- **Cache Entry Count** — total entries per cache +- **Cache Size** — disk and memory size per cache +- **Query Execution Time** — SQL query latency per table +- **Index Hit Rate** — how often queries use an index + +

+ KubeDB Ignite Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Ignite instance +kubectl delete ignite -n demo ignite-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Ignite database with KubeDB using [built-in Prometheus](/docs/guides/ignite/monitoring/using-builtin-prometheus.md). +- Monitor your Ignite database with KubeDB using [Prometheus Operator](/docs/guides/ignite/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/kafka/monitoring/grafana-dashboard.md b/docs/guides/kafka/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..61a468e8bd --- /dev/null +++ b/docs/guides/kafka/monitoring/grafana-dashboard.md @@ -0,0 +1,382 @@ +--- +title: Visualize Kafka Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: kf-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: kf-monitoring-kafka + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Kafka Metrics with Grafana Dashboard + +KubeDB exposes Kafka metrics — including JVM, broker/topic, and KRaft controller/quorum metrics — through a JMX Exporter running as a Java agent inside each Kafka container. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Kafka instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/kafka/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/kafka/monitoring) and [docs/examples/kafka/tls](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/kafka/tls) folders in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-kafka-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Kafka instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Kafka with Monitoring Enabled + +Kafka runs in KRaft mode (no ZooKeeper), so the combined broker+controller nodes need at least 3 replicas to form a working Raft quorum — this is also what lets you see meaningful data in the dashboard's KRaft Controller and KRaft Quorum panels later on. + +This example also enables TLS, so first create a self-signed `Issuer` that KubeDB will use to issue certificates for the cluster: + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=kafka/O=kubedb" + +$ kubectl create secret tls kafka-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/kafka-ca created + +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/kafka/tls/kf-Issuer.yaml +issuer.cert-manager.io/kafka-ca-issuer created +``` + +Below is the Kafka object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: Kafka +metadata: + name: kafka-grafana-demo + namespace: demo +spec: + version: "4.2.0" + replicas: 1 + deletionPolicy: WipeOut + storage: + storageClassName: "local-path" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `replicas: 3` deploys a 3-node combined KRaft cluster (each node acts as both broker and controller), which is required for the quorum panels to show meaningful data. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.exporter.port` sets the port the JMX exporter serves metrics on. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Kafka instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/kafka/monitoring/kf-with-monitoring.yaml +kafka.kubedb.com/kafka created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get kafka -n demo kafka +NAME VERSION STATUS AGE +kafka 3.9.0 Ready 5m +``` + +KubeDB creates a stats service named `{kafka-name}-stats` for monitoring: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=kafka" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kafka-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 5m +kafka-stats ClusterIP 10.96.10.2 56790/TCP 5m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +kafka-stats 5m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo kafka-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `kafka-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Kafka Dashboard + +The KubeDB Kafka dashboard is distributed as a single JSON file: `kafka_database_dashboard.json` in the [opnpulse/dashboards](https://github.com/opnpulse/dashboards/tree/master/kafka) repository (`kafka/` folder). The JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Download `kafka_database_dashboard.json` from that repository, then: + +1. In Grafana, click the **+** icon in the left sidebar and select **Import**. +2. Click **Upload JSON file** and select the downloaded file. +3. In the **datasource** dropdown that appears, select your Prometheus data source. +4. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing, the dashboard appears as **KubeDB Kafka Dashboard** under **Dashboards** in the left sidebar. It contains four sections: **Kafka Server**, **Broker Topic Metrics**, **KRaft Controller Monitoring Metrics**, and **KRaft Quorum Monitoring Metrics**. + +## Step 6: Explore the Dashboard + +Use the dropdown filters at the top of the dashboard to focus on a specific instance. + +| Variable | What to select | +|------------------|--------------------------------------------------------------| +| **datasource** | Your Prometheus data source | +| **namespace** | Namespace where your Kafka is deployed (e.g., `demo`) | +| **service** | Stats service of your Kafka instance (e.g., `kafka-stats`) | +| **pod** | A specific broker pod, or `All` for an aggregated view | +| **container** | Container to inspect (`kafka`) | + +**Kafka Server** — JVM and process-level health of the selected broker: +- **Status / Uptime / Start time** — whether the broker's JMX exporter is reachable, and how long it has been up +- **JVM Version** — the JVM the broker is running on +- **Average number of CPUs used** — CPU consumption of the process +- **Memory area [heap] / [nonheap]** — heap and non-heap JVM memory usage over time +- **GC time increase / GC count increase** — garbage collection pauses and frequency, by generation +- **JVM classes loaded** — number of loaded classes (a rough proxy for a leak if it keeps growing) +- **Threads used** — current and daemon JVM thread counts + +

+ KubeDB Kafka Server Dashboard +

+ +

+ KubeDB Kafka Server Dashboard JVM Panels +

+ +**Broker Topic Metrics** — throughput at the broker/topic level: +- **Messages in topics** — incoming message rate +- **Byte in / out rate from clients** — client-facing produce/consume throughput +- **Byte in / out rate from / to other brokers** — inter-broker replication traffic +- **Fetch request rate / Produce request rate** — request throughput +- **Failed produce request rate** — produce requests that failed (should stay at 0) + +

+ KubeDB Kafka Broker Topic Metrics +

+ +

+ KubeDB Kafka Broker Topic Metrics Produce Panels +

+ +**KRaft Controller Monitoring Metrics** — health of the KRaft metadata quorum's controller layer: +- **Number of Active Brokers / Number of active controllers** — cluster membership; exactly one broker should be the active controller +- **Fenced Broker Count** — brokers the controller has fenced out (non-zero means a broker is unhealthy) +- **Metadata Error Count** — errors while applying metadata records (should stay at 0) +- **Global Partition Count / Global Topic count** — partitions and topics tracked cluster-wide +- **Offline Partition Count** — partitions with no leader (non-zero means data unavailability) +- **Preferred Replica Imbalance Count** — partitions not currently led by their preferred replica + +

+ KubeDB Kafka KRaft Controller Monitoring Metrics +

+ +**KRaft Quorum Monitoring Metrics** — health of the Raft replication protocol itself: +- **Current Leader ID / Current quorum epoch** — which node is the metadata leader, and the current election epoch +- **High Watermark / Raft Log End Offset** — how far the metadata log has been committed and written +- **Number of Unknown Voter Connections** — connections to voters outside the current quorum (should stay at 0) +- **Average Commit Latency** — time to commit a metadata record to the quorum +- **Append Records Rate** — rate of metadata records appended to the log, per node +- **Current Voted** — which candidate each node voted for in the current epoch +- **Average Poll Idle Ratio** — fraction of time each node's Raft I/O thread spends idle (a low value means the thread is saturated) + +

+ KubeDB Kafka KRaft Quorum Monitoring Metrics +

+ +## Cleaning up + +```bash +# Remove the Kafka instance +kubectl delete kafka -n demo kafka + +# Remove the issuer and CA secret +kubectl delete issuer -n demo kafka-ca-issuer +kubectl delete secret -n demo kafka-ca + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Kafka instance with KubeDB using [built-in Prometheus](/docs/guides/kafka/monitoring/using-builtin-prometheus.md). +- Monitor your Kafka instance with KubeDB using [Prometheus Operator](/docs/guides/kafka/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mariadb/monitoring/grafana-dashboard.md b/docs/guides/mariadb/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..ba5d9e73c6 --- /dev/null +++ b/docs/guides/mariadb/monitoring/grafana-dashboard.md @@ -0,0 +1,364 @@ +--- +title: Visualize MariaDB Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-mariadb-grafana-dashboard + name: Grafana Dashboard + parent: guides-mariadb-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize MariaDB Metrics with Grafana Dashboard + +KubeDB exposes MariaDB metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a MariaDB instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/mariadb/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/mariadb/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-mariadb-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the MariaDB instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy MariaDB with Monitoring Enabled + +Below is the MariaDB object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: MariaDB +metadata: + name: mariadb-grafana-demo + namespace: demo +spec: + version: "11.5.2" + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the MariaDB instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/mariadb/monitoring/coreos-prom-mariadb.yaml +mariadb.kubedb.com/mariadb-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get mariadb -n demo mariadb-grafana-demo +NAME VERSION STATUS AGE +mariadb-grafana-demo 11.5.2 Ready 2m +``` + +KubeDB creates a stats service named `{mariadb-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=mariadb-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +mariadb-grafana-demo ClusterIP 10.96.10.1 3306/TCP 2m +mariadb-grafana-demo-stats ClusterIP 10.96.10.2 9104/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +mariadb-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo mariadb-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `mariadb-grafana-demo-stats`. Its state should be **UP**. + +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +

+ Prometheus Target +

+ +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB MariaDB Dashboard + +The KubeDB MariaDB dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Four dashboards are available. Download all JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/mariadb) repository (`mariadb/` folder): + +| File | Dashboard | +|------|-----------| +| `mariadb_summary.json` | KubeDB / MariaDB / Summary | +| `mariadb_pod.json` | KubeDB / MariaDB / Pod | +| `mariadb_databases.json` | KubeDB / MariaDB / Database | +| `mariadb_galera.json` | KubeDB / MariaDB / Galera Cluster | + +> The Galera Cluster dashboard is only relevant for MariaDB Galera cluster deployments (`spec.topology.mode: GaleraCluster`). + +**Import steps (repeat for each file):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|-----------------------------------|--------------------------------------------------------------------------------------------------| +| KubeDB / MariaDB / Summary | Instance overview: status, version, node count, resource requests/limits, CPU/memory usage | +| KubeDB / MariaDB / Pod | Per-pod summary, CPU/memory/file descriptor stats, connections, client threads | +| KubeDB / MariaDB / Database | Service status/uptime, cluster size/status, QPS, connections, disk and network I/O | +| KubeDB / MariaDB / Galera Cluster | Cluster size, node state, wsrep_ready, flow control, replication bytes, commit/cert failure rate | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|---------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your MariaDB is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your MariaDB instance (e.g., `mariadb-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` for an aggregated view | + +**KubeDB / MariaDB / Summary** — start here for an instance overview: +- **Database Status / Version** — current health of the instance and MariaDB version running +- **Require Secure Transport / Deletion Policy** — whether TLS is enforced and the cleanup policy for the instance +- **Total Nodes** — number of replicas in the instance +- **CPU / Memory / Storage Request & Limit** — configured resource requests and limits +- **CPU Info / CPU Quota** — CPU usage over time and per-pod quota utilization +- **Memory Info** — memory usage over time and per-pod quota utilization + +

+ KubeDB MariaDB Summary Dashboard +

+ +**KubeDB / MariaDB / Pod** — drill into a specific pod: +- **Pod Summary** — pod name, MySQL uptime, version, current QPS, InnoDB buffer pool size +- **CPU, Memory and File Descriptor Stats** — per-pod CPU usage, memory usage, and open file descriptors +- **Connections** — MySQL connections and aborted connections +- **Client Threads** — client thread activity and thread cache + +

+ KubeDB MariaDB Pod Dashboard +

+ +**KubeDB / MariaDB / Database** — cluster and query metrics: +- **Service Status / Uptime** — per-pod health and how long each pod has been serving +- **Cluster Size / Cluster Status** — number of nodes and Galera cluster state (Primary/Non-Primary) +- **Current QPS** — query throughput +- **MySQL Connections** — current vs. max connections +- **MySQL Disk Reads vs Writes** — disk I/O throughput +- **MySQL Network Received vs Sent** — network throughput + +

+ KubeDB MariaDB Database Dashboard +

+ +**KubeDB / MariaDB / Galera Cluster** — Galera-specific metrics: +- **Cluster Size** — number of nodes in the cluster +- **Local State** — wsrep state per node (Synced, Donor, Joiner, etc.) +- **wsrep_ready** — whether each node is ready to accept queries +- **Flow Control Paused** — percentage of time replication was paused due to flow control +- **Replication Bytes** — bytes sent and received via Galera replication per node +- **Local Commits / Cert Failures** — commit throughput and certification conflict rate + +

+ KubeDB MariaDB Galera Cluster Dashboard +

+ +## Cleaning up + +```bash +# Remove the MariaDB instance +kubectl delete mariadb -n demo mariadb-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your MariaDB database with KubeDB using [built-in Prometheus](/docs/guides/mariadb/monitoring/builtin-prometheus/). +- Monitor your MariaDB database with KubeDB using [Prometheus Operator](/docs/guides/mariadb/monitoring/prometheus-operator/). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/memcached/monitoring/grafana-dashboard.md b/docs/guides/memcached/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..9479218153 --- /dev/null +++ b/docs/guides/memcached/monitoring/grafana-dashboard.md @@ -0,0 +1,349 @@ +--- +title: Visualize Memcached Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-mc-grafana-dashboard + name: Grafana Dashboard + parent: mc-monitoring-memcached + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Memcached Metrics with Grafana Dashboard + +KubeDB exposes Memcached metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Memcached instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/memcached/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/memcached/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-memcached-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Memcached instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Memcached with Monitoring Enabled + +Below is the Memcached object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: Memcached +metadata: + name: memcached-grafana-demo + namespace: demo +spec: + version: "1.6.22" + replicas: 1 + deletionPolicy: WipeOut + podTemplate: + spec: + containers: + - name: memcached + resources: + limits: + cpu: 500m + memory: 128Mi + requests: + cpu: 250m + memory: 64Mi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Memcached instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/memcached/monitoring/memcached-grafana-demo.yaml +memcached.kubedb.com/memcached-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get memcached -n demo memcached-grafana-demo +NAME VERSION STATUS AGE +memcached-grafana-demo 1.6.22 Ready 2m +``` + +KubeDB creates a stats service named `{memcached-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=memcached-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +memcached-grafana-demo ClusterIP 10.96.10.1 11211/TCP 2m +memcached-grafana-demo-stats ClusterIP 10.96.10.2 56790/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +memcached-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo memcached-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `memcached-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Memcached Dashboards + +The KubeDB Memcached dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download the JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/memcached) repository (`memcached/` folder): + +| File | Dashboard | +|------|-----------| +| `memcached_summary_dashboard.json` | KubeDB / Memcached / Summary | +| `memcached_pods_dashboard.json` | KubeDB / Memcached / Pod | +| `memcached_databases_dashboard.json` | KubeDB / Memcached / Database | + +**Import steps (repeat for each file):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing the files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|----------------|-------------------|----------------------------------------------------------------| +| **namespace** | Summary dashboard | Namespace where your Memcached is deployed (e.g., `demo`) | +| **memcached** | Summary dashboard | Name of your instance (e.g., `memcached-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, e.g. `memcached-grafana-demo-0` | +| **job** | Database dashboard| The stats job for your instance, e.g. `memcached-grafana-demo-stats` | + +**KubeDB / Memcached / Summary** — instance-level overview: +- **General Info** — database status, version, whether secure transport is required, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — CPU usage over time and per-pod quota utilization (usage vs. requests/limits) + +

+ KubeDB Memcached Summary Dashboard +

+ +**KubeDB / Memcached / Pod** — per-pod drill-down: +- **% Hit Ratio / % Get Ratio** — cache effectiveness for this pod +- **Commands** — rate of `cas`, `decr`, `delete`, `flush`, `get`, `incr`, `set`, `touch` operations +- **Connections** — active client connections on this pod +- **Storage** — memory used vs. free +- **Items** — number of items stored, used vs. free capacity +- **Evictions / Reclaimed** — items evicted due to memory pressure vs. reclaimed from expired entries +- **Network** — read and write throughput + +

+ KubeDB Memcached Pod Dashboard +

+ +**KubeDB / Memcached / Database** — instance health and throughput: +- **Instance State** — whether the instance is UP or DOWN +- **Up Time** — how long the instance has been running +- **Connections** — current active connections +- **QPS** — queries per second +- **Memory Usage Ratio** — memory used as a percentage of the configured limit +- **Items** — number of items currently stored +- **QPS / Connections / Read-Write** — the same metrics plotted over time for trend analysis + +

+ KubeDB Memcached Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Memcached instance +kubectl delete memcached -n demo memcached-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Memcached database with KubeDB using [built-in Prometheus](/docs/guides/memcached/monitoring/using-builtin-prometheus.md). +- Monitor your Memcached database with KubeDB using [Prometheus Operator](/docs/guides/memcached/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/monitoring/grafana-dashboard.md b/docs/guides/mongodb/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..ee36fb5a5a --- /dev/null +++ b/docs/guides/mongodb/monitoring/grafana-dashboard.md @@ -0,0 +1,351 @@ +--- +title: Visualize MongoDB Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: mg-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: mg-monitoring-mongodb + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize MongoDB Metrics with Grafana Dashboard + +KubeDB exposes MongoDB metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a MongoDB instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/mongodb/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/mongodb/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-mongodb-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the MongoDB instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy MongoDB with Monitoring Enabled + +Below is the MongoDB object with monitoring configured to use Prometheus Operator. It deploys a 3-member replica set — this is what lets you see meaningful data in the dashboard's replication and oplog panels later on. + +```yaml +apiVersion: kubedb.com/v1 +kind: MongoDB +metadata: + name: mg-grafana-demo + namespace: demo +spec: + version: "8.0.17" + replicas: 3 + replicaSet: + name: rs0 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `replicas: 3` and `replicaSet.name: rs0` deploy a 3-member replica set named `rs0`, which is required for the replication and oplog panels to show meaningful data. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the MongoDB instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/mongodb/monitoring/mg-grafana-demo.yaml +mongodb.kubedb.com/mg-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get mongodb -n demo mg-grafana-demo +NAME VERSION STATUS AGE +mg-grafana-demo 8.0.17 Ready 5m +``` + +KubeDB creates a stats service named `{mongodb-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=mg-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +mg-grafana-demo ClusterIP 10.96.10.1 27017/TCP 5m +mg-grafana-demo-pods ClusterIP None 27017/TCP 5m +mg-grafana-demo-stats ClusterIP 10.96.10.2 56790/TCP 5m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +mg-grafana-demo-stats 5m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo mg-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `mg-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB MongoDB Dashboard + +The KubeDB MongoDB dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [opnpulse/dashboards](https://github.com/opnpulse/dashboards/tree/master/mongodb) repository (`mongodb/` folder): + +| File | Dashboard | +|------|-----------| +| `mongodb-summary-dashboard.json` | KubeDB / MongoDB / Summary | +| `mongodb-pod-dashboard.json` | KubeDB / MongoDB / Pod | +| `mongodb-database-replset-dashboard.json` | KubeDB / MongoDB / Database (ReplicaSet) | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / MongoDB / Summary | Database status, uptime, version, node count, resource requests/limits, CPU/memory usage | +| KubeDB / MongoDB / Pod | Per-pod uptime, QPS, latency, command operations, connections, cursors | +| KubeDB / MongoDB / Database (ReplicaSet) | Replica set state, member count, last election, replication lag, oplog metrics | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|------------------|----------------------------|-----------------------------------------------------------------| +| **Datasource** | All dashboards | Your Prometheus data source | +| **Interval** | Pod, Database dashboards | Query resolution/step interval (e.g., `auto`, `1m`) | +| **namespace** | All dashboards | Namespace where your MongoDB is deployed (e.g., `demo`) | +| **MongoDB** | Summary, Pod dashboards | Name of your MongoDB instance (e.g., `mg-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod (e.g., `mg-grafana-demo-0`) | +| **Replica Set** | Database dashboard | The replica set to inspect (e.g., `rs0`) | + +**KubeDB / MongoDB / Summary** — start here for an instance overview: +- **General Info** — database status, up-time, version, total nodes, deletion policy +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — CPU usage over time and per-pod quota utilization +- **Memory Info** — memory usage over time and per-pod quota utilization + +

+ KubeDB MongoDB Summary Dashboard +

+ +**KubeDB / MongoDB / Pod** — drill into a specific pod: +- **Overview** — pod name, uptime, QPS, latency +- **Command Operations** — query/update operation rate +- **Latency Detail** — read and write latency +- **Connections / Cursors** — active connections and open cursors on this pod + +

+ KubeDB MongoDB Pod Dashboard +

+ +**KubeDB / MongoDB / Database (ReplicaSet)** — replication health: +- **Overview** — replica set state, member count, time since last election, replication lag, storage engine +- **Replication Info** — replication operations (insert/delete) and replication lag over time +- **Oplog Info** — oplog getmore time, oplog buffer capacity, oplog operations, buffered operations + +

+ KubeDB MongoDB Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the MongoDB instance +kubectl delete mongodb -n demo mg-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your MongoDB database with KubeDB using [built-in Prometheus](/docs/guides/mongodb/monitoring/using-builtin-prometheus.md). +- Monitor your MongoDB database with KubeDB using [Prometheus Operator](/docs/guides/mongodb/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mssqlserver/monitoring/grafana-dashboard.md b/docs/guides/mssqlserver/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..712f1e9717 --- /dev/null +++ b/docs/guides/mssqlserver/monitoring/grafana-dashboard.md @@ -0,0 +1,383 @@ +--- +title: Visualize MSSQLServer Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: ms-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: ms-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize MSSQLServer Metrics with Grafana Dashboard + +KubeDB exposes MSSQLServer metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a MSSQLServer instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- MSSQLServer requires TLS to be enabled. You need a cert-manager `Issuer` in the `demo` namespace before deploying. If cert-manager is not installed, install it first: + + ```bash + $ helm repo add jetstack https://charts.jetstack.io + $ helm repo update + $ helm upgrade --install cert-manager jetstack/cert-manager \ + --namespace cert-manager --create-namespace \ + --set crds.enabled=true + ``` + + Then create a self-signed `Issuer` in the `demo` namespace: + + ```yaml + apiVersion: cert-manager.io/v1 + kind: Issuer + metadata: + name: mssqlserver-ca-issuer + namespace: demo + spec: + selfSigned: {} + ``` + + ```bash + $ kubectl apply -f issuer.yaml + issuer.cert-manager.io/mssqlserver-ca-issuer created + ``` + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/mssqlserver/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/mssqlserver/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-mssqlserver-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the MSSQLServer instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy MSSQLServer with Monitoring Enabled + +Below is the MSSQLServer object with TLS and monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: MSSQLServer +metadata: + name: mssql-grafana-demo + namespace: demo +spec: + version: "2022-cu12" + replicas: 1 + tls: + issuerRef: + name: mssqlserver-ca-issuer + kind: Issuer + apiGroup: "cert-manager.io" + clientTLS: false + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `tls.issuerRef` is required for MSSQLServer; it references the cert-manager Issuer created above. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the MSSQLServer instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/mssqlserver/monitoring/mssql-grafana-demo.yaml +mssqlserver.kubedb.com/mssql-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get mssqlserver -n demo mssql-grafana-demo +NAME VERSION STATUS AGE +mssql-grafana-demo 2022-cu12 Ready 3m +``` + +KubeDB creates a stats service named `{mssqlserver-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=mssql-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +mssql-grafana-demo ClusterIP 10.96.10.1 1433/TCP 3m +mssql-grafana-demo-stats ClusterIP 10.96.10.2 9399/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +mssql-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo mssql-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `mssql-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB MSSQLServer Dashboard + +The KubeDB MSSQLServer dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/mssqlserver) repository (`mssqlserver/` folder): + +| File | Dashboard | +|------|-----------| +| `mssqlserver_summary_dashboard.json` | KubeDB / MSSQLServer / Summary | +| `mssqlserver_pods_dashboard.json` | KubeDB / MSSQLServer / Pod | +| `mssqlserver_databases_dashboard.json` | KubeDB / MSSQLServer / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / MSSQLServer / Summary | Instance status, version, node count, resource requests/limits, CPU usage | +| KubeDB / MSSQLServer / Pod | Per-pod status, role (Primary/Secondary), uptime, server resource overview, connections | +| KubeDB / MSSQLServer / Database | Service status/uptime, AG cluster replica roles, cluster status, SQL compilations, batch requests | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|------------------|--------------------|-------------------------------------------------------------------| +| **Namespace** | All dashboards | Namespace where your MSSQLServer is deployed (e.g., `demo`) | +| **mssqlserver** | All dashboards | Name of your instance (e.g., `mssql-grafana-demo`) | +| **Pod Name** | Pod dashboard | A specific pod (e.g., `mssql-grafana-demo-0`) | +| **Job / database** | Pod dashboard | The stats job and target database to inspect | + +**KubeDB / MSSQLServer / Summary** — start here for an instance overview: +- **General Info** — database status, version, whether secure transport is required, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization + +

+ KubeDB MSSQLServer Summary Dashboard +

+ +**KubeDB / MSSQLServer / Pod** — drill into a specific pod: +- **Pod Name / Status / Role / Uptime** — pod identity, running status, Availability Group role (Primary/Secondary), and uptime +- **Server Resource Overview** — server local time, total and used RAM, pagefile size and usage +- **Server Resource Overview (2)** — total page faults, batch requests/sec, page life expectancy, deadlocks, user errors/sec, kill connection errors/sec +- **Summary** — current database connections, log growth since last restart, total I/O stall wait time + +

+ KubeDB MSSQLServer Pod Dashboard +

+ +**KubeDB / MSSQLServer / Database** — Availability Group cluster health: +- **Service Status / Uptime** — per-pod health and how long each pod has been serving +- **AG Cluster Active Replica** — which pod is acting as the active replica +- **Cluster Status** — count of Primary vs. Secondary replicas +- **SQL Compilations/sec** — per-pod query compilation rate +- **Batch Requests per Second** — per-pod T-SQL batch throughput + +

+ KubeDB MSSQLServer Database Dashboard +

+ + +## Cleaning up + +```bash +# Remove the MSSQLServer instance +kubectl delete mssqlserver -n demo mssql-grafana-demo + +# Remove the TLS Issuer +kubectl delete issuer mssqlserver-ca-issuer -n demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your MSSQLServer instance with KubeDB using [Prometheus Operator](/docs/guides/mssqlserver/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mysql/monitoring/grafana-dashboard.md b/docs/guides/mysql/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..259b3e07ef --- /dev/null +++ b/docs/guides/mysql/monitoring/grafana-dashboard.md @@ -0,0 +1,359 @@ +--- +title: Visualize MySQL Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-mysql-grafana-dashboard + name: Grafana Dashboard + parent: guides-mysql-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize MySQL Metrics with Grafana Dashboard + +KubeDB exposes MySQL metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a MySQL instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/mysql/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/mysql/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-mysql-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the MySQL instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy MySQL with Monitoring Enabled + +Below is the MySQL object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: MySQL +metadata: + name: mysql-grafana-demo + namespace: demo +spec: + version: "9.6.0" + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the MySQL instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/mysql/monitoring/coreos-prom-mysql.yaml +mysql.kubedb.com/mysql-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get mysql -n demo mysql-grafana-demo +NAME VERSION STATUS AGE +mysql-grafana-demo 9.6.0 Ready 2m +``` + +KubeDB creates a stats service named `{mysql-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=mysql-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +mysql-grafana-demo ClusterIP 10.96.10.1 3306/TCP 2m +mysql-grafana-demo-stats ClusterIP 10.96.10.2 9104/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +mysql-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo mysql-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `mysql-grafana-demo-stats`. Its state should be **UP**. + +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +

+ Prometheus Target +

+ +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB MySQL Dashboard + +The KubeDB MySQL dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Four dashboards are available. Download all JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/mysql) repository (`mysql/` folder): + +| File | Dashboard | +|------|-----------| +| `mysql_summary_dashboard.json` | KubeDB / MySQL / Summary | +| `mysql_pods_dashboard.json` | KubeDB / MySQL / Pod | +| `mysql_databases_dashboard.json` | KubeDB / MySQL / Database | +| `mysql_group_replication_dashboard.json` | KubeDB / MySQL / Group Replication | + +> The Group Replication dashboard is only relevant for MySQL Group Replication deployments (`spec.topology.mode: GroupReplication`). + +**Import steps (repeat for each file):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---------------------------------------|----------------------------------------------------------------------------------------------------| +| KubeDB / MySQL / Summary | Instance overview: status, version, node count, resource requests/limits, CPU/memory usage | +| KubeDB / MySQL / Pod | Per-pod uptime, version, QPS, InnoDB buffer pool size, CPU/memory, connections | +| KubeDB / MySQL / Database | Per-pod service status/uptime, QPS, connections, disk and network I/O | +| KubeDB / MySQL / Group Replication | Group member status and primary node, replication lag, transport time | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|-------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your MySQL is deployed (e.g., `demo`) | +| **MySQL** | All dashboards | Name of your MySQL instance (e.g., `mysql-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / MySQL / Summary** — start here for an instance overview: +- **General Info** — database status, version, user address type, whether secure transport is required, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization +- **Memory Info** — memory usage over time and per-pod quota utilization + +

+ KubeDB MySQL Summary Dashboard +

+ +**KubeDB / MySQL / Pod** — drill into a specific pod: +- **MySQL Pod Summary** — pod name, uptime, version, current QPS, InnoDB buffer pool size per pod +- **Pod CPU, Memory and File Descriptor Stats** — per-pod CPU usage, memory usage, open file descriptors +- **Connections** — MySQL connections and aborted connections +- **Client Threads** — client thread activity and thread cache + +

+ KubeDB MySQL Pod Dashboard +

+ +**KubeDB / MySQL / Database** — per-pod service and throughput metrics: +- **Service Status / Uptime** — per-pod health and how long each pod has been serving +- **Current QPS** — query throughput per pod +- **MySQL Connections** — current vs. max connections per pod +- **MySQL Disk Reads vs Writes** — per-pod disk I/O throughput +- **MySQL Network Received vs Sent** — per-pod network throughput + +

+ KubeDB MySQL Database Dashboard +

+ +**KubeDB / MySQL / Group Replication** — replication group health: +- **Group Replication Node Title** — ONLINE/OFFLINE status of each member +- **Primary Node** — which member currently holds the PRIMARY role +- **Replication Lag** — per-member lag behind the primary +- **Transport Time / Replication Delay** — time spent transporting and applying transactions + +

+ KubeDB MySQL Group Replication Dashboard +

+ +## Cleaning up + +```bash +# Remove the MySQL instance +kubectl delete mysql -n demo mysql-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your MySQL database with KubeDB using [built-in Prometheus](/docs/guides/mysql/monitoring/builtin-prometheus/). +- Monitor your MySQL database with KubeDB using [Prometheus Operator](/docs/guides/mysql/monitoring/prometheus-operator/). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/neo4j/monitoring/grafana-dashboard.md b/docs/guides/neo4j/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..833b19d048 --- /dev/null +++ b/docs/guides/neo4j/monitoring/grafana-dashboard.md @@ -0,0 +1,325 @@ +--- +title: Visualize Neo4j Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-neo4j-grafana-dashboard + name: Grafana Dashboard + parent: neo4j-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Neo4j Metrics with Grafana Dashboard + +KubeDB exposes Neo4j metrics by enabling Neo4j's built-in metrics support. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Neo4j instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/neo4j/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/neo4j/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-neo4j-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Neo4j instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Neo4j with Monitoring Enabled + +Below is the Neo4j object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Neo4j +metadata: + name: neo4j-grafana-demo + namespace: demo +spec: + version: "2025.11.2" + replicas: 3 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Neo4j instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/neo4j/monitoring/neo4j-grafana-demo.yaml +neo4j.kubedb.com/neo4j-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get neo4j -n demo neo4j-grafana-demo +NAME VERSION STATUS AGE +neo4j-grafana-demo 2025.11.2 Ready 3m +``` + +KubeDB creates a stats service named `{neo4j-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=neo4j-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +neo4j-grafana-demo ClusterIP 10.96.10.1 7687/TCP,7474/TCP 3m +neo4j-grafana-demo-stats ClusterIP 10.96.10.2 2004/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +neo4j-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo neo4j-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `neo4j-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Neo4j Dashboards + +The KubeDB Neo4j dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/neo4j) repository (`neo4j/` folder): + +| File | Dashboard | +|------|-----------| +| `neo4j-summary.json` | KubeDB / Neo4j / Summary | +| `neo4j-pod.json` | KubeDB / Neo4j / Pod | +| `neo4j-database.json` | KubeDB / Neo4j / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|--------------------------|-------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Neo4j is deployed (e.g., `demo`) | +| **Neo4j** | All dashboards | Name of your instance (e.g., `neo4j-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod (e.g., `neo4j-grafana-demo-0`) | +| **database** | Pod, Database dashboards | Target database name (e.g., `neo4j`, `system`) | +| **leader** | Pod, Database dashboards | The pod currently holding database leadership | + +**KubeDB / Neo4j / Summary** — instance-level overview: + +- **General Info** — database status, version, whether secure transport is required, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization + +

+ KubeDB Neo4j Summary Dashboard +

+ +**KubeDB / Neo4j / Pod** — per-pod drill-down: + +- **Neo4j** — status, uptime, and active Bolt connections for the selected pod +- **CPU And Memory Usage Stats** — total memory used and average CPU usage over time +- **Transactions** — last committed write transaction ID, committed transaction speed, property creation speed, rolled-back transaction speed, committed transaction rate + +

+ KubeDB Neo4j Pod Dashboard +

+ +**KubeDB / Neo4j / Database** — graph database metrics: + +- **Neo4j** — uptime, node count, and relationship count for the selected database +- **CPU And Memory Usage Stats** — total memory used and average CPU usage over time +- **Transactions** — last committed write transaction ID, committed transaction rate, committed write transaction rate + +

+ KubeDB Neo4j Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Neo4j instance +kubectl delete neo4j -n demo neo4j-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Neo4j database with KubeDB using [built-in Prometheus](/docs/guides/neo4j/monitoring/using-builtin-prometheus.md). +- Monitor your Neo4j database with KubeDB using [Prometheus Operator](/docs/guides/neo4j/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/percona-xtradb/monitoring/grafana-dashboard.md b/docs/guides/percona-xtradb/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..e8b6a7b35e --- /dev/null +++ b/docs/guides/percona-xtradb/monitoring/grafana-dashboard.md @@ -0,0 +1,357 @@ +--- +title: Visualize PerconaXtraDB Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-perconaxtradb-grafana-dashboard + name: Grafana Dashboard + parent: guides-perconaxtradb-monitoring + weight: 40 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize PerconaXtraDB Metrics with Grafana Dashboard + +KubeDB exposes PerconaXtraDB metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a PerconaXtraDB instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/percona-xtradb/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/percona-xtradb/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-perconaxtradb-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the PerconaXtraDB instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy PerconaXtraDB with Monitoring Enabled + +PerconaXtraDB requires at least 3 replicas for a Galera cluster. Below is the PerconaXtraDB object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: PerconaXtraDB +metadata: + name: pxc-grafana-demo + namespace: demo +spec: + version: "8.4.3" + replicas: 3 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the PerconaXtraDB instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/percona-xtradb/monitoring/coreos-prom-pxc.yaml +perconaxtradb.kubedb.com/pxc-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get perconaxtradb -n demo pxc-grafana-demo +NAME VERSION STATUS AGE +pxc-grafana-demo 8.4.3 Ready 3m +``` + +KubeDB creates a stats service named `{perconaxtradb-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=pxc-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +pxc-grafana-demo ClusterIP 10.96.10.1 3306/TCP 3m +pxc-grafana-demo-stats ClusterIP 10.96.10.2 9104/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +pxc-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo pxc-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `pxc-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB PerconaXtraDB Dashboard + +The KubeDB PerconaXtraDB dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Four dashboards are available. Download all JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/perconaxtradb) repository (`perconaxtradb/` folder): + +| File | Dashboard | +|------|-----------| +| `perconaxtradb_summary_dashboard.json` | KubeDB / PerconaXtraDB / Summary | +| `perconaxtradb_pods_dashboard.json` | KubeDB / PerconaXtraDB / Pod | +| `perconaxtradb_databases_dashboard.json` | KubeDB / PerconaXtraDB / Database | +| `perconaxtradb_galera_cluster_dashboard.json` | KubeDB / PerconaXtraDB / Galera Cluster | + +**Import steps (repeat for each file):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all four files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / PerconaXtraDB / Summary | Instance overview: status, version, node count, resource requests/limits, CPU usage | +| KubeDB / PerconaXtraDB / Pod | Per-pod uptime, version, QPS, InnoDB buffer pool size, CPU/memory, connections | +| KubeDB / PerconaXtraDB / Database | Per-pod service status/uptime, cluster size/status, QPS, connections, disk and network I/O | +| KubeDB / PerconaXtraDB / Galera Cluster | Cluster name, per-node online status, Galera replication latency | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|----------------------|-------------------------|-----------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your PerconaXtraDB is deployed (e.g., `demo`) | +| **PerconaXtraDB** | All dashboards | Name of your instance (e.g., `pxc-grafana-demo`) | +| **Pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / PerconaXtraDB / Summary** — start here for a cluster overview: +- **General Info** — database status, version, whether secure transport is required, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization + +

+ KubeDB PerconaXtraDB Summary Dashboard +

+ +**KubeDB / PerconaXtraDB / Pod** — drill into a specific node: +- **PerconaXtraDB Pod Summary** — pod name, MySQL uptime, version, current QPS, InnoDB buffer pool size per pod +- **Pod CPU, Memory and File Descriptor Stats** — per-pod CPU usage, memory usage, open file descriptors +- **Connections** — MySQL connections and aborted connections +- **Client Threads** — client thread activity and thread cache + +

+ KubeDB PerconaXtraDB Pod Dashboard +

+ +**KubeDB / PerconaXtraDB / Database** — cluster and query metrics: +- **Service Status / Uptime** — per-pod health and how long each pod has been serving +- **Cluster Size / Cluster Status** — number of nodes and Primary/Non-Primary state per node +- **Current QPS** — query throughput +- **MySQL Connections** — current vs. max connections +- **MySQL Disk Reads vs Writes** — disk I/O throughput +- **MySQL Network Received vs Sent** — network throughput + +

+ KubeDB PerconaXtraDB Database Dashboard +

+ +**KubeDB / PerconaXtraDB / Galera Cluster** — Galera-specific metrics: +- **Cluster Name** — name of the Galera cluster +- **Cluster Members** — ONLINE/OFFLINE status per node +- **Replication Delay Details** — average and standard deviation of Galera replication latency per node + +

+ KubeDB PerconaXtraDB Galera Cluster Dashboard +

+ +## Cleaning up + +```bash +# Remove the PerconaXtraDB instance +kubectl delete perconaxtradb -n demo pxc-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your PerconaXtraDB database with KubeDB using [built-in Prometheus](/docs/guides/percona-xtradb/monitoring/builtin-prometheus/). +- Monitor your PerconaXtraDB database with KubeDB using [Prometheus Operator](/docs/guides/percona-xtradb/monitoring/prometheus-operator/). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/pgbouncer/monitoring/grafana-dashboard.md b/docs/guides/pgbouncer/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..c314873cf1 --- /dev/null +++ b/docs/guides/pgbouncer/monitoring/grafana-dashboard.md @@ -0,0 +1,361 @@ +--- +title: Visualize PgBouncer Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: pb-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: pb-monitoring-pgbouncer + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize PgBouncer Metrics with Grafana Dashboard + +KubeDB exposes PgBouncer metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a PgBouncer instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- PgBouncer sits in front of a PostgreSQL server. Prepare a KubeDB Postgres instance (for example `ha-postgres` in the `demo` namespace) following the [streaming replication guide](/docs/guides/postgres/clustering/streaming_replication.md), or use any externally managed Postgres. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/pgbouncer/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/pgbouncer/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-pgbouncer-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the PgBouncer instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy PgBouncer with Monitoring Enabled + +Below is the PgBouncer object pointing at the `ha-postgres` backend, with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: PgBouncer +metadata: + name: pb-grafana-demo + namespace: demo +spec: + replicas: 1 + version: "1.24.0" + database: + syncUsers: true + databaseName: "postgres" + databaseRef: + name: "ha-postgres" + namespace: demo + connectionPool: + poolMode: session + port: 5432 + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `database.databaseRef` points at the backend Postgres instance PgBouncer pools connections for. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the PgBouncer instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/pgbouncer/monitoring/coreos-prom-pb.yaml +pgbouncer.kubedb.com/pb-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get pb -n demo pb-grafana-demo +NAME TYPE VERSION STATUS AGE +pb-grafana-demo kubedb.com/v1 1.24.0 Ready 65s +``` + +KubeDB creates a stats service named `{pgbouncer-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=pb-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +pb-grafana-demo ClusterIP 10.96.201.180 5432/TCP 2m +pb-grafana-demo-pods ClusterIP None 5432/TCP 2m +pb-grafana-demo-stats ClusterIP 10.96.73.22 9719/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +pb-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo pb-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `pb-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB PgBouncer Dashboard + +The KubeDB PgBouncer dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/pgbouncer) repository (`pgbouncer/` folder): + +| File | Dashboard | +|------|-----------| +| `pgbouncer_summary_dashboard.json` | KubeDB / PgBouncer / Summary | +| `pgbouncer_pods_dashboard.json` | KubeDB / PgBouncer / Pod | +| `pgbouncer_databases_dashboard.json` | KubeDB / PgBouncer / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / PgBouncer / Summary | Total/client/server connections, active pools, query throughput, CPU/memory/storage | +| KubeDB / PgBouncer / Pod | Per-pod connections, pools, waiting clients, query rate, CPU/memory | +| KubeDB / PgBouncer / Database | Per-database pool state, active/idle/waiting clients, max connections, query rate | + +## Step 6: Explore the Dashboard + +After opening a dashboard, you will see dropdown filters at the top. These control which data is shown across all panels — change them to focus on a specific instance without editing any queries. + +| Variable | Applies to | What to select | +|---------------|-------------------------|-------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your PgBouncer is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your PgBouncer instance (e.g., `pb-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` to see aggregated view | + +Once you set these, all panels update automatically. Below is what each dashboard shows: + +**KubeDB / PgBouncer / Summary** — start here for a pool-wide overview: +- **Total / Client / Server Connections** — connections across all pools +- **Active Pools** — number of connection pools to the backend +- **Queries per Second** — query throughput routed through PgBouncer +- **Waiting Clients** — clients queued waiting for a server connection (sustained non-zero indicates pool exhaustion) +- **Avg Query Time** — average time queries spend waiting + executing +- **CPU / Memory / Storage** — resource consumption vs. requests and limits + +

+ KubeDB PgBouncer Summary Dashboard +

+ +**KubeDB / PgBouncer / Pod** — drill into a specific pod: +- **Client / Server Connections** — connections held on this pod +- **Active Pools** — pools managed by this pod +- **Queries per Second** — per-pod query throughput +- **Waiting Clients** — queued clients on this pod +- **CPU / Memory** — per-pod resource usage + +

+ KubeDB PgBouncer Pod Dashboard +

+ + +**KubeDB / PgBouncer / Database** — per-database pool metrics: +- **Pool State** — cl_active, cl_waiting, sv_active, sv_idle per database +- **Active / Idle Server Connections** — backend connection usage per database +- **Max Client Connections** — configured limit per database +- **Query Rate** — queries per second per database +- **Total Xact / Query Count** — transaction and query counters per database + +

+ KubeDB PgBouncer Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the PgBouncer instance +kubectl delete pb -n demo pb-grafana-demo + +# Remove the backend Postgres instance +kubectl delete pg -n demo ha-postgres + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your PgBouncer database with KubeDB using [built-in Prometheus](/docs/guides/pgbouncer/monitoring/using-builtin-prometheus.md). +- Monitor your PgBouncer database with KubeDB using [Prometheus Operator](/docs/guides/pgbouncer/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/pgpool/monitoring/grafana-dashboard.md b/docs/guides/pgpool/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..9516b3a7ac --- /dev/null +++ b/docs/guides/pgpool/monitoring/grafana-dashboard.md @@ -0,0 +1,354 @@ +--- +title: Visualize Pgpool Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: pp-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: pp-monitoring-pgpool + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Pgpool Metrics with Grafana Dashboard + +KubeDB exposes Pgpool metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Pgpool instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- Pgpool sits in front of a PostgreSQL server. Prepare a KubeDB Postgres instance (for example `ha-postgres` in the `demo` namespace) following the [streaming replication guide](/docs/guides/postgres/clustering/streaming_replication.md), or use any externally managed Postgres. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/pgpool/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/pgpool/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-pgpool-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Pgpool instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Pgpool with Monitoring Enabled + +Below is the Pgpool object pointing at the `ha-postgres` backend, with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Pgpool +metadata: + name: pp-grafana-demo + namespace: demo +spec: + version: "4.6.0" + postgresRef: + name: ha-postgres + namespace: demo + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `postgresRef` points at the backend Postgres instance Pgpool load-balances connections across. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Pgpool instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/pgpool/monitoring/coreos-prom-pp.yaml +pgpool.kubedb.com/pp-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get pp -n demo pp-grafana-demo +NAME TYPE VERSION STATUS AGE +pp-grafana-demo kubedb.com/v1alpha2 4.6.0 Ready 65s +``` + +KubeDB creates a stats service named `{pgpool-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=pp-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +pp-grafana-demo ClusterIP 10.96.201.180 9999/TCP,9595/TCP 2m +pp-grafana-demo-pods ClusterIP None 9999/TCP 2m +pp-grafana-demo-stats ClusterIP 10.96.73.22 9719/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +pp-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo pp-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `pp-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Pgpool Dashboard + +The KubeDB Pgpool dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/pgpool) repository (`pgpool/` folder): + +| File | Dashboard | +|------|-----------| +| `pgpool_summary_dashboard.json` | KubeDB / Pgpool / Summary | +| `pgpool_pods_dashboard.json` | KubeDB / Pgpool / Pod | +| `pgpool_databases_dashboard.json` | KubeDB / Pgpool / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / Pgpool / Summary | Node health, client/server connections, query throughput, replication delay, CPU/memory/storage | +| KubeDB / Pgpool / Pod | Per-pod connections, query rate, cache hit ratio, CPU/memory | +| KubeDB / Pgpool / Database | Per-backend connections, replication status, statement throughput | + +## Step 6: Explore the Dashboard + +After opening a dashboard, you will see dropdown filters at the top. These control which data is shown across all panels — change them to focus on a specific instance without editing any queries. + +| Variable | Applies to | What to select | +|---------------|-------------------------|-----------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Pgpool is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your Pgpool instance (e.g., `pp-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` to see aggregated view | + +Once you set these, all panels update automatically. Below is what each dashboard shows: + +**KubeDB / Pgpool / Summary** — start here for a pool-wide overview: +- **Node Health** — Pgpool node status and process liveness +- **Client / Server Connections** — total client and backend Postgres connections +- **Queries per Second** — statement throughput routed through Pgpool +- **Replication Delay** — lag between primary and replica backends (for load-balanced setups) +- **Cache Hit Ratio** — in-memory query cache effectiveness +- **CPU / Memory / Storage** — resource consumption vs. requests and limits + +

+ KubeDB Pgpool Summary Dashboard +

+ + +**KubeDB / Pgpool / Pod** — drill into a specific pod: +- **Client / Server Connections** — connections held on this pod +- **Queries per Second** — per-pod statement throughput +- **Cache Hit Ratio** — per-pod query cache effectiveness +- **Backend Pool Status** — active/idle backend connections per pod +- **CPU / Memory** — per-pod resource usage + +

+ KubeDB Pgpool Pod Dashboard +

+ +**KubeDB / Pgpool / Database** — per-backend and per-database metrics: +- **Backend Connection Status** — up/down/recovering status per backend Postgres node +- **Connections per Backend** — active and idle connections per backend +- **Replication Status** — primary/standby role and replication health per backend +- **Statement Throughput** — SELECT/INSERT/UPDATE/DELETE rate per database +- **Backend Select / Load Balance Ratio** — query distribution across backends + +

+ KubeDB Pgpool Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Pgpool instance +kubectl delete pp -n demo pp-grafana-demo + +# Remove the backend Postgres instance +kubectl delete pg -n demo ha-postgres + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Pgpool database with KubeDB using [built-in Prometheus](/docs/guides/pgpool/monitoring/using-builtin-prometheus.md). +- Monitor your Pgpool database with KubeDB using [Prometheus Operator](/docs/guides/pgpool/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/postgres/monitoring/grafana-dashboard.md b/docs/guides/postgres/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..f28d770489 --- /dev/null +++ b/docs/guides/postgres/monitoring/grafana-dashboard.md @@ -0,0 +1,357 @@ +--- +title: Visualize PostgreSQL Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: pg-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: pg-monitoring-postgres + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize PostgreSQL Metrics with Grafana Dashboard + +KubeDB exposes PostgreSQL metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a PostgreSQL instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/postgres/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/postgres/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-postgresql-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the PostgreSQL instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +### Step 1: Deploy PostgreSQL with Monitoring Enabled + +Below is the PostgreSQL object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: Postgres +metadata: + name: pg-grafana-demo + namespace: demo +spec: + version: "13.13" + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the PostgreSQL instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/postgres/monitoring/coreos-prom-postgres.yaml +postgres.kubedb.com/pg-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get postgres -n demo pg-grafana-demo +NAME VERSION STATUS AGE +pg-grafana-demo 13.13 Ready 2m +``` + +KubeDB creates a stats service named `{postgres-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=pg-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +pg-grafana-demo ClusterIP 10.96.10.1 5432/TCP 2m +pg-grafana-demo-replicas ClusterIP 10.96.10.2 5432/TCP 2m +pg-grafana-demo-stats ClusterIP 10.96.10.3 56790/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +pg-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo pg-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `pg-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB PostgreSQL Dashboard + +The KubeDB Postgres dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/postgres) repository (`postgres/` folder): + +| File | Dashboard | +|------|-----------| +| `postgres_summary_dashboard.json` | KubeDB / Postgres / Summary | +| `postgres_pods_dashboard.json` | KubeDB / Postgres / Pod | +| `postgres_databases_dashboard.json` | KubeDB / Postgres / Database | + +> A Perses-format version of each dashboard (`*-perses.json`) is also available in the same folder if you use Perses instead of Grafana. + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---------------------------------|-------------------------------------------------------------------------------------| +| KubeDB / Postgres / Summary | Overall summary: status, connections, replication lag, CPU, memory, storage, network | +| KubeDB / Postgres / Pod | Per-pod metrics: server role, connections, CPU/memory usage, PostgreSQL settings | +| KubeDB / Postgres / Database | Database-level metrics: QPS, transactions, cache hit rate, sessions, locks | + +## Step 6: Explore the Dashboard + +After opening a dashboard, you will see dropdown filters at the top. These control which data is shown across all panels — change them to focus on a specific instance or database without editing any queries. + +| Variable | Applies to | What to select | +|---------------|-------------------------|---------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Postgres is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your Postgres instance (e.g., `pg-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` to see aggregated view | +| **datname** | Database dashboard only | A specific database inside Postgres, or `All` | + +Once you set these, all panels update automatically. Below is what each dashboard shows: + +**KubeDB / Postgres / Summary** — start here for a health overview +- **General Info** — version, uptime, total replicas, database status, SSL mode, deletion policy +- **Connections** — current active connections +- **Postgres Replication Lag** — replication lag for standby replicas (relevant for HA setups) +- **CPU / Memory / Storage Usage** — resource consumption vs. requests and limits +- **Network** — receive and transmit bandwidth + +

+ KubeDB Postgres Summary Dashboard +

+ + +**KubeDB / Postgres / Pod** — drill into a specific pod +- **Server Up / Role** — whether the pod is alive and whether it is the primary or a replica +- **Max Connections** — connection limit configured for this pod +- **CPU / Memory Usage** — per-pod resource usage over time +- **Settings** — active runtime config: shared buffers, effective cache, work mem, max WAL size + +

+ KubeDB Postgres Pod Dashboard +

+ +**KubeDB / Postgres / Database** — drill into a specific database +- **QPS** — queries per second hitting this database +- **Transactions** — commits and rollbacks over time +- **Cache Hit Rate** — percentage of reads served from shared buffers; aim for > 99% +- **Active / Idle Sessions** — how many connections are executing vs. waiting +- **Lock Tables** — tables with active locks (high counts can indicate contention) +- **Conflicts / Deadlocks** — events that abort transactions; spikes indicate application issues +- **Rows** — insert, update, delete, fetch, and return activity per second + +

+ KubeDB Postgres Database Dashboard +

+ + +## Cleaning up + +```bash +# Remove the PostgreSQL instance +kubectl delete postgres -n demo pg-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring +kubectl delete ns kubeops +``` + +## Next Steps + +- Monitor your PostgreSQL database with KubeDB using [built-in Prometheus](/docs/guides/postgres/monitoring/using-builtin-prometheus.md). +- Monitor your PostgreSQL database with KubeDB using [Prometheus Operator](/docs/guides/postgres/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/proxysql/monitoring/grafana-dashboard.md b/docs/guides/proxysql/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..987898cd30 --- /dev/null +++ b/docs/guides/proxysql/monitoring/grafana-dashboard.md @@ -0,0 +1,339 @@ +--- +title: Visualize ProxySQL Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-proxysql-grafana-dashboard + name: Grafana Dashboard + parent: guides-proxysql-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize ProxySQL Metrics with Grafana Dashboard + +KubeDB exposes ProxySQL metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a ProxySQL instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/proxysql/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/proxysql/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-proxysql-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the ProxySQL instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy ProxySQL with Monitoring Enabled + +ProxySQL requires a backend MySQL cluster. The example below assumes a MySQL cluster named `mysql-server` already exists in the `demo` namespace. Below is the ProxySQL object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: ProxySQL +metadata: + name: proxysql-grafana-demo + namespace: demo +spec: + version: "2.4.4-debian" + replicas: 1 + backend: + name: mysql-server + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the ProxySQL instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/proxysql/monitoring/coreos-prom-proxysql.yaml +proxysql.kubedb.com/proxysql-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get proxysql -n demo proxysql-grafana-demo +NAME VERSION STATUS AGE +proxysql-grafana-demo 2.4.4-debian Ready 2m +``` + +KubeDB creates a stats service named `{proxysql-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=proxysql-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +proxysql-grafana-demo ClusterIP 10.96.10.1 6033/TCP 2m +proxysql-grafana-demo-stats ClusterIP 10.96.10.2 6070/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +proxysql-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo proxysql-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `proxysql-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB ProxySQL Dashboard + +The KubeDB ProxySQL dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/proxysql) repository (`proxysql/` folder): + +| File | Dashboard | +|------|-----------| +| `proxysql_summary_dashboard.json` | KubeDB / ProxySQL / Summary | +| `proxysql_pods_dashboard.json` | KubeDB / ProxySQL / Pod | +| `proxysql_databases_dashboard.json` | KubeDB / ProxySQL / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / ProxySQL / Summary | Instance overview: status, version, backend server, node count, resource requests/limits, CPU usage | +| KubeDB / ProxySQL / Pod | Per-pod uptime, version, CPU/memory, connections, aborted connections, temporary objects/slow queries | +| KubeDB / ProxySQL / Database | Per-pod service status/uptime, connections, network received/sent, slow queries, aborted connections | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|------------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your ProxySQL is deployed (e.g., `demo`) | +| **proxysql** | All dashboards | Name of your ProxySQL instance (e.g., `proxysql-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod (e.g., `proxysql-grafana-demo-0`) | + +**KubeDB / ProxySQL / Summary** — start here for an instance overview: +- **General Info** — database status, version, backend server, frontend SSL, total nodes, termination policy +- **Resource Requests / Limits** — configured CPU and memory requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization +- **Memory Info** — memory usage over time + +

+ KubeDB ProxySQL Summary Dashboard +

+ +**KubeDB / ProxySQL / Pod** — drill into a specific proxy pod: +- **MySQL Pod Summary** — pod name, version, and ProxySQL uptime +- **Pod CPU, Memory and File Descriptor Stats** — per-pod CPU and memory usage +- **Connections** — ProxySQL connections and aborted connections +- **Temporary Objects & Slow Queries** — slow query rate on this pod + +

+ KubeDB ProxySQL Pod Dashboard +

+ +**KubeDB / ProxySQL / Database** — per-pod service and network metrics: +- **Service Status / Uptime** — per-pod health and how long each pod has been serving +- **Proxy Network** — ProxySQL connections and network received vs. sent per pod +- **ProxySQL Slow Queries / Aborted Connections** — per-pod query and connection error rates + +

+ KubeDB ProxySQL Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the ProxySQL instance +kubectl delete proxysql -n demo proxysql-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your ProxySQL instance with KubeDB using [built-in Prometheus](/docs/guides/proxysql/monitoring/builtin-prometheus/). +- Monitor your ProxySQL instance with KubeDB using [Prometheus Operator](/docs/guides/proxysql/monitoring/prometheus-operator/). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/qdrant/monitoring/grafana-dashboard.md b/docs/guides/qdrant/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..d0580e3e09 --- /dev/null +++ b/docs/guides/qdrant/monitoring/grafana-dashboard.md @@ -0,0 +1,348 @@ +--- +title: Visualize Qdrant Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-qdrant-grafana-dashboard + name: Grafana Dashboard + parent: qdrant-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Qdrant Metrics with Grafana Dashboard + +KubeDB exposes Qdrant metrics through its built-in Prometheus endpoint. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Qdrant instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/qdrant/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/qdrant/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-qdrant-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Qdrant instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Qdrant with Monitoring Enabled + +Below is the Qdrant object with monitoring configured to use Prometheus Operator. Qdrant exposes metrics natively on port `6333` so no separate exporter sidecar is needed. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Qdrant +metadata: + name: qdrant-grafana-demo + namespace: demo +spec: + version: "1.17.0" + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 6333 + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.exporter.port: 6333` points the scrape target at Qdrant's native metrics endpoint. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). + +Create the Qdrant instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/qdrant/monitoring/qdrant-grafana-demo.yaml +qdrant.kubedb.com/qdrant-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get qdrant -n demo qdrant-grafana-demo +NAME VERSION STATUS AGE +qdrant-grafana-demo 1.17.0 Ready 2m +``` + +KubeDB creates a stats service named `{qdrant-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=qdrant-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +qdrant-grafana-demo ClusterIP 10.96.10.1 6333/TCP 2m +qdrant-grafana-demo-stats ClusterIP 10.96.10.2 6333/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +qdrant-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo qdrant-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Select the `serviceMonitor/demo/qdrant-grafana-demo-stats/0` pool from the dropdown. The target at `http://:6333/metrics` should show state **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Qdrant Dashboards + +The KubeDB Qdrant dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/qdrant) repository (`qdrant/` folder): + +| File | Dashboard | +|------|-----------| +| `qdrant_summary_dashboard.json` | KubeDB / Qdrant / Summary | +| `qdrant_pods_dashboard.json` | KubeDB / Qdrant / Pod | +| `qdrant_databases_dashboard.json` | KubeDB / Qdrant / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Qdrant is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `qdrant-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / Qdrant / Summary** — instance-level overview: + +**General Info section:** +- **Database Status** — current health of the Qdrant instance +- **Version** — Qdrant version running +- **TLS Mode** — TLS, Client, and P2P encryption status +- **Deletion Policy** — KubeDB deletion policy (e.g., `WipeOut`) +- **Total Nodes** — number of active member pods +- **CPU Request / CPU Limit** — configured CPU bounds per pod (in cores) +- **Memory Request / Memory Limit** — configured memory bounds per pod +- **Storage Request** — configured storage request per pod + +**CPU Info section:** +- **CPU Usage** — per-pod CPU consumption over time +- **CPU Quota** — table of CPU requests, limits, and usage percentages per pod + +

+ KubeDB Qdrant Summary Dashboard +

+ +**KubeDB / Qdrant / Pod** — per-pod drill-down. Use the **Pod** dropdown to select a specific `qdrant-grafana-demo-N` pod. + +**Overview section:** +- **Pod Name** — name of the selected pod +- **Status** — current health status of the selected pod +- **Uptime** — how long this pod has been running + +**Memory & Performance section:** +- **Memory Usage** — RSS memory consumed by this pod over time +- **CPU Usage by Collection** — CPU usage broken down by Qdrant collection on this pod + +**REST Requests Summary section:** +- **REST Request Rate by Endpoint** — per-endpoint request throughput over time +- **REST Request Distribution by Endpoint** — breakdown of request share per endpoint +- **REST Endpoint Performance Summary** — latency and throughput breakdown per REST endpoint + +

+ KubeDB Qdrant Pod Dashboard +

+ +**KubeDB / Qdrant / Database** — cluster-wide service and performance view: + +**Overview section:** +- **Cluster Mode** — whether Qdrant is running in standalone or distributed mode +- **Service Status** — overall service health +- **Service Uptime** — total uptime of the Qdrant service + +**Memory & Performance section:** +- **Memory Usage** — total memory consumed across the cluster over time +- **CPU Usage** — total CPU consumption across the cluster over time + +

+ KubeDB Qdrant Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Qdrant instance +kubectl delete qdrant -n demo qdrant-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Qdrant database with KubeDB using [Prometheus Operator](/docs/guides/qdrant/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/rabbitmq/monitoring/grafana-dashboard.md b/docs/guides/rabbitmq/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..8a79a7ba28 --- /dev/null +++ b/docs/guides/rabbitmq/monitoring/grafana-dashboard.md @@ -0,0 +1,349 @@ +--- +title: Visualize RabbitMQ Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: rm-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: rm-monitoring-guides + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize RabbitMQ Metrics with Grafana Dashboard + +KubeDB exposes RabbitMQ metrics through a built-in Prometheus plugin. Once Prometheus scrapes those metrics, you can visualize them in Grafana using a pre-built KubeDB dashboard. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a RabbitMQ instance, and importing the Grafana dashboard. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/rabbitmq/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/rabbitmq/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-rabbitmq-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the RabbitMQ instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy RabbitMQ with Monitoring Enabled + +Below is the RabbitMQ object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: RabbitMQ +metadata: + name: rmq-grafana-demo + namespace: demo +spec: + version: "4.0.4" + replicas: 1 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the RabbitMQ instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/rabbitmq/monitoring/coreos-prom-rabbitmq.yaml +rabbitmq.kubedb.com/rmq-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get rabbitmq -n demo rmq-grafana-demo +NAME VERSION STATUS AGE +rmq-grafana-demo 4.0.4 Ready 2m +``` + +KubeDB creates a stats service named `{rabbitmq-name}-stats` for monitoring: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=rmq-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +rmq-grafana-demo ClusterIP 10.96.10.1 5672/TCP 2m +rmq-grafana-demo-stats ClusterIP 10.96.10.2 15692/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +rmq-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo rmq-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `rmq-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB RabbitMQ Dashboard + +The KubeDB RabbitMQ dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/rabbitmq) repository (`rabbitmq/` folder): + +| File | Dashboard | +|------|-----------| +| `rabbitmq_summary_dashboard.json` | KubeDB / RabbitMQ / Summary | +| `rabbitmq_pods_dashboard.json` | KubeDB / RabbitMQ / Pod | +| `rabbitmq_databases_dashboard.json` | KubeDB / RabbitMQ / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / RabbitMQ / Summary | Node/queue/connection overview, message rates, memory/disk alarms, CPU/memory/storage | +| KubeDB / RabbitMQ / Pod | Per-pod message rates, file descriptors, memory usage, CPU/memory | +| KubeDB / RabbitMQ / Database | Queue depth, consumer utilisation, message age, publish/deliver/ack rates per queue | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|-------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your RabbitMQ is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your RabbitMQ instance (e.g., `rmq-grafana-demo`) | +| **pod** | Pod, Database dashboards | A specific pod, or `All` for an aggregated view | +| **vhost** | Database dashboard only | A specific virtual host, or `All` | + +**KubeDB / RabbitMQ / Summary** — start here for a node and cluster overview: +- **Node Health** — running/stopped/disk alarm/memory alarm status per node +- **Queue Count** — total queues in the cluster +- **Message Rates** — publish rate, deliver rate, acknowledge rate +- **Messages Ready / Unacknowledged** — total backlog depth +- **Connection / Channel Count** — active connections and open channels +- **CPU / Memory / Disk Free** — resource consumption per node + +

+ KubeDB RabbitMQ Summary Dashboard +

+ +**KubeDB / RabbitMQ / Pod** — drill into a specific node: +- **Erlang Process Count** — number of Erlang processes (high counts indicate load) +- **Memory Breakdown** — code, heap, binaries, ETS table memory +- **Socket Descriptors** — used vs. available file descriptors for connections +- **GC** — garbage collection runs and bytes reclaimed per second +- **CPU / Memory** — per-pod resource usage over time + +

+ KubeDB RabbitMQ Pod Dashboard +

+ +**KubeDB / RabbitMQ / Database** — per-queue and per-vhost metrics: +- **Queue Depth** — messages ready + unacknowledged per queue +- **Publish / Deliver Rate** — throughput per queue +- **Consumer Count** — active consumers per queue +- **Oldest Unacknowledged Message** — age of the oldest pending message (latency indicator) + +

+ KubeDB RabbitMQ Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the RabbitMQ instance +kubectl delete rabbitmq -n demo rmq-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your RabbitMQ instance with KubeDB using [built-in Prometheus](/docs/guides/rabbitmq/monitoring/using-builtin-prometheus.md). +- Monitor your RabbitMQ instance with KubeDB using [Prometheus Operator](/docs/guides/rabbitmq/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/redis/monitoring/grafana-dashboard.md b/docs/guides/redis/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..761572c163 --- /dev/null +++ b/docs/guides/redis/monitoring/grafana-dashboard.md @@ -0,0 +1,353 @@ +--- +title: Visualize Redis Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: rd-grafana-dashboard-monitoring + name: Grafana Dashboard + parent: rd-monitoring-redis + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Redis Metrics with Grafana Dashboard + +KubeDB exposes Redis metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Redis instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/redis/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/redis/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-redis-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Redis instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Redis with Monitoring Enabled + +Below is the Redis object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1 +kind: Redis +metadata: + name: redis-cluster + namespace: demo +spec: + version: 8.2.2 + mode: Cluster + cluster: + shards: 3 + replicas: 2 + storageType: Durable + storage: + resources: + requests: + storage: 1Gi + storageClassName: local-path + accessModes: + - ReadWriteOnce + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Redis instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/redis/monitoring/redis-cluster.yaml +redis.kubedb.com/redis-cluster created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get redis -n demo redis-cluster +NAME VERSION STATUS AGE +redis-cluster 8.2.2 Ready 5m +``` + +KubeDB creates a stats service named `{redis-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=redis-cluster" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +redis-cluster ClusterIP 10.96.10.1 6379/TCP 5m +redis-cluster-stats ClusterIP 10.96.10.2 56790/TCP 5m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +redis-cluster-stats 5m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo redis-cluster-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `redis-cluster-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Redis Dashboard + +The KubeDB Redis dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download the JSON files from the [opnpulse/dashboards](https://github.com/opnpulse/dashboards/tree/master/redis) repository (`redis/` folder): + +| File | Dashboard | +|------|-----------| +| `redis_summary_dashboard.json` | KubeDB / Redis / Summary | +| `redis_pod_dashboard.json` | KubeDB / Redis / Pod | +| `redis_shards_dashboard.json` | KubeDB / Redis / Shard | + +> The Shard dashboard is relevant for Redis Cluster mode (`spec.mode: Cluster`); its panels stay empty for a standalone (non-cluster) Redis instance. + +**Import steps (repeat for each file you need):** + +1. In Grafana, click the `+` icon in the left sidebar. +2. Select `Import` from the menu. +3. Click `Upload JSON file` and select one of the downloaded `.json` files. +4. In the `Prometheus` dropdown that appears, select your Prometheus data source. +5. Click `Import`. + +The import page looks like this — click **Upload dashboard JSON file** to select the file: + +

+ Grafana Import Dashboard +

+ +After importing the files you need, they will appear under `Dashboards` in the left sidebar. + +| Dashboard Name | Description | +|---|---| +| KubeDB / Redis / Summary | Instance overview: status, version, mode, node count, resource requests/limits, CPU usage | +| KubeDB / Redis / Pod | Per-pod role, master/slaves, connected clients, memory, commands/sec, network I/O, CPU/memory | +| KubeDB / Redis / Shard | Cluster shard slot health, node/slave count, per-slave status, cluster mode | + +## Step 6: Explore the Dashboard + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|----------------|--------------------------|--------------------------------------------------------------| +| **datasource** | All dashboards | Your Prometheus data source | +| **Namespace** | All dashboards | Namespace where your Redis is deployed (e.g., `demo`) | +| **app** | Summary dashboard | Name of your Redis instance (e.g., `redis-cluster`) | +| **redis** | Pod, Shard dashboards | Name of your Redis instance (e.g., `redis-cluster`) | +| **Pod Name** | Pod, Shard dashboards | A specific pod (e.g., `redis-cluster-shard0-0`) | +| **Filters** | Shard dashboard | Additional label filters for the selected shard | + +**KubeDB / Redis / Summary** — start here for an instance overview: +- **General Info** — database status, version, max clients, Redis mode, deletion policy, total nodes +- **Resource Requests / Limits** — configured CPU, memory, and storage requests and limits +- **CPU Info / CPU Quota** — per-pod CPU usage over time and quota utilization + +

+ KubeDB Redis Summary Dashboard +

+ +**KubeDB / Redis / Pod** — drill into a specific pod: +- **General Counters And File Descriptor Stats** — status, role (master/slave), my master, my slaves, connected clients, Go routines +- **Uptime / Memory Usage / Commands Executed / Hits-Misses** — pod uptime, memory usage, command execution rate, cache hit/miss rate +- **Network I/O / Command Calls / Connected Clients** — network throughput, per-command call breakdown, connected client count over time +- **CPU And Memory Usage Stats** — total memory usage, average CPU usage, average memory usage + +

+ KubeDB Redis Pod Dashboard +

+ +**KubeDB / Redis / Shard** — cluster shard health for Cluster mode: +- **Cluster Shard Slots / Cluster Shard Slots Failed** — hash slot coverage and any failed slots +- **Cluster Nodes / Cluster Masters** — total nodes and master count in the cluster +- **Connected Slaves / My Slaves** — number of connected slaves and their IP, port, and online status +- **Mode** — confirms the instance is running in `cluster` mode + +

+ KubeDB Redis Shard Dashboard +

+ +## Cleaning up + +```bash +# Remove the Redis instance +kubectl delete redis -n demo redis-cluster + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Redis database with KubeDB using [built-in Prometheus](/docs/guides/redis/monitoring/using-builtin-prometheus.md). +- Monitor your Redis database with KubeDB using [Prometheus Operator](/docs/guides/redis/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/singlestore/monitoring/grafana-dashboard.md b/docs/guides/singlestore/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..6f9dbd1965 --- /dev/null +++ b/docs/guides/singlestore/monitoring/grafana-dashboard.md @@ -0,0 +1,386 @@ +--- +title: Visualize Singlestore Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-sdb-grafana-dashboard + name: Grafana Dashboard + parent: guides-sdb-monitoring + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Singlestore Metrics with Grafana Dashboard + +KubeDB exposes Singlestore metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Singlestore instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- Singlestore requires a valid license secret. Create the license secret before deploying: + + ```bash + $ kubectl create secret generic license-secret -n demo \ + --from-literal=username=license \ + --from-literal=password= + ``` + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/singlestore/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/singlestore/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-singlestore-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Singlestore instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Singlestore with Monitoring Enabled + +Below is the Singlestore object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Singlestore +metadata: + name: sdb-grafana-demo + namespace: demo +spec: + version: "8.7.10" + licenseSecret: + kind: Secret + name: license-secret + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the Singlestore instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/singlestore/monitoring/sdb-grafana-demo.yaml +singlestore.kubedb.com/sdb-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get singlestore -n demo sdb-grafana-demo +NAME VERSION STATUS AGE +sdb-grafana-demo 8.7.10 Ready 3m +``` + +KubeDB creates a stats service named `{singlestore-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=sdb-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +sdb-grafana-demo ClusterIP 10.96.10.1 3306/TCP 3m +sdb-grafana-demo-stats ClusterIP 10.96.10.2 9104/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +sdb-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo sdb-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `sdb-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Singlestore Dashboards + +The KubeDB Singlestore dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/singlestore) repository (`singlestore/` folder): + +| File | Dashboard | +|------|-----------| +| `singlestore_summary_dashboard.json` | KubeDB / Singlestore / Summary | +| `singlestore_pods_dashboard.json` | KubeDB / Singlestore / Pod | +| `singlestore_databases_dashboard.json` | KubeDB / Singlestore / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|-------------------------|---------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Singlestore is deployed (e.g., `demo`) | +| **singlestore** | All dashboards | Name of your instance (e.g., `sdb-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +### KubeDB / Singlestore / Summary + +Start here for a cluster-level overview. The dashboard has three sections: + +**General Info** — status cards at the top: +- **Database Status** — current health (e.g., `Ready`) +- **Version** — Singlestore version running (e.g., `8.7.10`) +- **Require Secure TLS** — whether TLS is enforced +- **Deletion Policy** — configured deletion policy (e.g., `WipeOut`) +- **Total Nodes**, **CPU Request/Limit**, **Memory Request/Limit**, **Storage Request** — resource configuration at a glance + +**CPU Info**: +- **CPU Usage** — CPU consumption over time per pod +- **CPU Quota** — table showing CPU usage vs. requests and the usage percentage + +**Memory Info / Storage Info**: +- **Memory Quota** — memory usage vs. requests and limits (RSS and cache breakdown) +- **Disk Usage** — disk consumed over time +- **Disk R/W Info** — read vs. write bytes per second +- **IOPS** — combined reads and writes per second +- **ThroughPut** — read and write throughput + +

+ KubeDB Singlestore Summary Dashboard +

+ +### KubeDB / Singlestore / Pod + +Drill into a specific pod. The dashboard has three sections: + +**Singlestore Pod Summary** — stat cards at the top: +- **Singlestore Uptime** — how long the pod has been running +- **Version** — Singlestore engine version on this pod +- **Current QPS** — queries per second on this pod +- **Transaction Buffer** — current transaction buffer size + +**Pod CPU, Memory and File Descriptor Stats**: +- **CPU Usage** — per-pod CPU consumption over time +- **Memory Usage** — per-pod memory consumption over time +- **Open File Descriptors** — number of open file handles + +**Connections**: +- **Singlestore Connections** — active client connections on this pod +- **Singlestore Aborted Connections** — connection attempts that failed +- **Client Threads** — threads handling client requests + +

+ KubeDB Singlestore Pod Dashboard +

+ +### KubeDB / Singlestore / Database + +Database-level operational metrics: + +- **Service Status** — table listing pod endpoints and their service names +- **Service Uptime** — how long each service has been available +- **Current QPS** — query throughput across the database +- **Singlestore Connections** — active connections to the database +- **Disk Reads vs Writes** — per-pod read and write activity over time +- **Network Received vs Sent** — inbound and outbound network traffic per pod + +

+ KubeDB Singlestore Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Singlestore instance +kubectl delete singlestore -n demo sdb-grafana-demo + +# Remove the license secret +kubectl delete secret license-secret -n demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Singlestore instance with KubeDB using [Prometheus Operator](/docs/guides/singlestore/monitoring/prometheus-operator/index.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/solr/monitoring/grafana-dashboard.md b/docs/guides/solr/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..56871f84bb --- /dev/null +++ b/docs/guides/solr/monitoring/grafana-dashboard.md @@ -0,0 +1,349 @@ +--- +title: Visualize Solr Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-sl-grafana-dashboard + name: Grafana Dashboard + parent: sl-monitoring-solr + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize Solr Metrics with Grafana Dashboard + +KubeDB exposes Solr metrics through the built-in Prometheus exporter module. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a Solr instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- Solr requires a ZooKeeper cluster for coordination. Deploy a ZooKeeper instance in the `demo` namespace before creating the Solr instance. See the [ZooKeeper quickstart](/docs/guides/zookeeper/quickstart/quickstart.md) for details. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/solr/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/solr/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-solr-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the Solr instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy Solr with Monitoring Enabled + +Below is the Solr object with monitoring configured to use Prometheus Operator. The `prometheus-exporter` module must be listed in `solrModules` to enable the metrics endpoint. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Solr +metadata: + name: solr-grafana-demo + namespace: demo +spec: + version: "9.8.0" + replicas: 1 + solrModules: + - prometheus-exporter + zookeeperRef: + name: zk-grafana-demo + namespace: demo + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `solrModules: [prometheus-exporter]` enables Solr's built-in Prometheus metrics module. +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). + +Create the Solr instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/solr/monitoring/solr-grafana-demo.yaml +solr.kubedb.com/solr-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get solr -n demo solr-grafana-demo +NAME VERSION STATUS AGE +solr-grafana-demo 9.8.0 Ready 3m +``` + +KubeDB creates a stats service named `{solr-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=solr-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +solr-grafana-demo ClusterIP 10.96.10.1 8983/TCP 3m +solr-grafana-demo-stats ClusterIP 10.96.10.2 9854/TCP 3m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +solr-grafana-demo-stats 3m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo solr-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `solr-grafana-demo-stats`. Its state should be **UP**. + +

+ Prometheus Target +

+ +If the target is missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +

+ Grafana Login +

+ +After a successful login you will see the Grafana home page: + +

+ Grafana Home +

+ +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB Solr Dashboards + +The KubeDB Solr dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/solr) repository (`solr/` folder): + +| File | Dashboard | +|------|-----------| +| `solr_summary_dashboard.json` | KubeDB / Solr / Summary | +| `solr_pods_dashboard.json` | KubeDB / Solr / Pod | +| `solr_databases_dashboard.json` | KubeDB / Solr / Database | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +The import page looks like this: + +

+ Grafana Import Dashboard +

+ +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|----------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your Solr is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `solr-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / Solr / Summary** — cluster-level overview: +- **Database Status** — current health of the Solr cluster +- **Version** — Solr version running +- **Queries per Second** — total query throughput across the cluster +- **Update Rate** — documents added and deleted per second +- **Cache Hit Rates** — query result cache and filter cache effectiveness +- **CPU / Memory** — resource usage over time + +

+ KubeDB Solr Summary Dashboard +

+ +**KubeDB / Solr / Pod** — per-node drill-down: +- **Uptime** — how long this Solr node has been running +- **JVM Heap** — heap used vs. max on this node +- **GC Activity** — garbage collection pause frequency and duration +- **Requests on Pod** — queries and updates handled by this node +- **CPU / Memory** — per-pod resource usage + +

+ KubeDB Solr Pod Dashboard +

+ +**KubeDB / Solr / Database** — collection-level metrics: +- **Document Count** — total documents per collection +- **Index Size** — disk space used by each collection's index +- **Shard Distribution** — documents and replicas across shards +- **Merge Activity** — index segment merges in progress + +

+ KubeDB Solr Database Dashboard +

+ +## Cleaning up + +```bash +# Remove the Solr instance +kubectl delete solr -n demo solr-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your Solr cluster with KubeDB using [built-in Prometheus](/docs/guides/solr/monitoring/prometheus-builtin.md). +- Monitor your Solr cluster with KubeDB using [Prometheus Operator](/docs/guides/solr/monitoring/prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/zookeeper/monitoring/grafana-dashboard.md b/docs/guides/zookeeper/monitoring/grafana-dashboard.md new file mode 100644 index 0000000000..2bea0bba21 --- /dev/null +++ b/docs/guides/zookeeper/monitoring/grafana-dashboard.md @@ -0,0 +1,340 @@ +--- +title: Visualize ZooKeeper Metrics with Grafana Dashboard +menu: + docs_{{ .version }}: + identifier: guides-zk-grafana-dashboard + name: Grafana Dashboard + parent: zk-monitoring-guides + weight: 25 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Visualize ZooKeeper Metrics with Grafana Dashboard + +KubeDB exposes ZooKeeper metrics through a sidecar exporter. Once Prometheus scrapes those metrics, you can visualize them in Grafana using pre-built KubeDB dashboards. This tutorial walks through the full setup: deploying the monitoring stack, enabling monitoring on a ZooKeeper instance, and importing the Grafana dashboards. + +## Before You Begin + +- You need a Kubernetes cluster with `kubectl` configured. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- KubeDB must be installed in your cluster with `kubedb-metrics` enabled. Follow the setup guide [here](/docs/setup/README.md) and make sure to include the flag below during installation: + + ```bash + --set kubedb-metrics.enabled=true + ``` + + `kubedb-metrics` creates `MetricsConfiguration` objects for each database type, which Panopticon (Step 2) uses to expose metrics to Prometheus. + +- To keep monitoring resources isolated, we use a separate `monitoring` namespace and deploy the database in the `demo` namespace. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/zookeeper/monitoring](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/zookeeper/monitoring) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Configuration + +> These two steps — deploying `kube-prometheus-stack` and installing Panopticon — are shared prerequisites for all KubeDB database monitoring guides. If you have already completed them in another guide, skip to [Step 1](#step-1-deploy-zookeeper-with-monitoring-enabled). + +### Step 1: Deploy kube-prometheus-stack + +`kube-prometheus-stack` installs Prometheus, Prometheus Operator, Alertmanager, and Grafana together. This is the recommended way to get the full monitoring stack on Kubernetes. + +Add the prometheus-community Helm repo and install: + +```bash +$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +$ helm repo update + +$ helm upgrade --install prometheus prometheus-community/kube-prometheus-stack \ + --namespace monitoring \ + --set grafana.image.tag=7.5.5 +``` + +Wait for all pods to be ready: + +```bash +$ kubectl get pods -n monitoring +NAME READY STATUS RESTARTS AGE +alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 2m +prometheus-grafana-xxxx 3/3 Running 0 2m +prometheus-kube-prometheus-operator-xxxx 1/1 Running 0 2m +prometheus-kube-prometheus-prometheus-0 2/2 Running 0 2m +prometheus-kube-state-metrics-xxxx 1/1 Running 0 2m +``` + +Find the `serviceMonitorSelector` label that Prometheus uses to pick up `ServiceMonitor` objects. You will need this label when enabling monitoring on the ZooKeeper instance. + +```bash +$ kubectl get prometheus -n monitoring -o jsonpath='{.items[0].spec.serviceMonitorSelector}' +{"matchLabels":{"release":"prometheus"}} +``` + +The label is `release: prometheus`. + +### Step 2: Install Panopticon + +Panopticon is the Appscode operator that reads `MetricsConfiguration` objects created by `kubedb-metrics` and exposes them to Prometheus. It must be installed before enabling `kubedb-metrics`. + +```bash +$ helm repo add appscode https://charts.appscode.com/stable/ +$ helm repo update + +$ helm upgrade --install panopticon appscode/panopticon \ + --version v2026.4.30 \ + --namespace kubeops --create-namespace \ + --set monitoring.enabled=true \ + --set monitoring.agent=prometheus.io/operator \ + --set monitoring.serviceMonitor.labels.release=prometheus \ + --set-file license=/path/to/kubedb-license.txt \ + --wait --timeout 5m0s +``` + +Verify panopticon is running: + +```bash +$ kubectl get pods -n kubeops +NAME READY STATUS RESTARTS AGE +panopticon-xxxx 1/1 Running 0 1m +``` + +## Setup + +## Step 1: Deploy ZooKeeper with Monitoring Enabled + +Below is the ZooKeeper object with monitoring configured to use Prometheus Operator. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: ZooKeeper +metadata: + name: zk-grafana-demo + namespace: demo +spec: + version: "3.8.3" + replicas: 3 + deletionPolicy: WipeOut + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 100Mi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s +``` + +Here, + +- `monitor.agent: prometheus.io/operator` tells KubeDB to create a `ServiceMonitor` for this instance. +- `monitor.prometheus.serviceMonitor.labels` must match the `serviceMonitorSelector` label of your Prometheus (`release: prometheus`). +- `monitor.prometheus.serviceMonitor.interval` sets the scrape interval to 10 seconds. + +Create the ZooKeeper instance: + +```bash +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/zookeeper/monitoring/zk-grafana-demo.yaml +zookeeper.kubedb.com/zk-grafana-demo created +``` + +Wait for it to be `Ready`: + +```bash +$ kubectl get zookeeper -n demo zk-grafana-demo +NAME VERSION STATUS AGE +zk-grafana-demo 3.8.3 Ready 2m +``` + +KubeDB creates a stats service named `{zookeeper-name}-stats` for the exporter: + +```bash +$ kubectl get svc -n demo --selector="app.kubernetes.io/instance=zk-grafana-demo" +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +zk-grafana-demo ClusterIP 10.96.10.1 2181/TCP 2m +zk-grafana-demo-stats ClusterIP 10.96.10.2 7000/TCP 2m +``` + +KubeDB also creates a `ServiceMonitor` in the `demo` namespace: + +```bash +$ kubectl get servicemonitor -n demo +NAME AGE +zk-grafana-demo-stats 2m +``` + +Verify it carries the correct label: + +```bash +$ kubectl get servicemonitor -n demo zk-grafana-demo-stats -o jsonpath='{.metadata.labels}' +{"release":"prometheus", ...} +``` + +## Step 2: Verify Prometheus is Scraping + +Port-forward the Prometheus pod: + +```bash +$ kubectl port-forward -n monitoring \ + prometheus-prometheus-kube-prometheus-prometheus-0 9090 +Forwarding from 127.0.0.1:9090 -> 9090 +Forwarding from [::1]:9090 -> 9090 +``` + +Open [http://localhost:9090/targets](http://localhost:9090/targets) in your browser. Look for an entry whose `service` label matches `zk-grafana-demo-stats`. Its state should be **UP**. For a 3-replica ensemble, all three pods will appear as separate targets. + +

+ Prometheus Target +

+ +If the targets are missing, check that the `ServiceMonitor` label (`release: prometheus`) matches the Prometheus `serviceMonitorSelector`. + +## Step 3: Access Grafana + +Port-forward the Grafana service: + +```bash +$ kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80 +Forwarding from 127.0.0.1:3000 -> 80 +``` + +Open [http://localhost:3000](http://localhost:3000). The username is `admin`. Retrieve the auto-generated password from the secret: + +```bash +$ kubectl get secret -n monitoring prometheus-grafana \ + -o jsonpath='{.data.admin-password}' | base64 -d +``` + +| Field | Value | +|----------|-----------------------------| +| Username | `admin` | +| Password | output of the command above | + +## Step 4: Configure Prometheus as a Data Source + +If you installed Grafana via `kube-prometheus-stack`, Prometheus is already configured as the default data source — skip to Step 5. + +For a standalone Grafana installation: + +1. Go to **Connections** → **Data sources** → **Add new data source**. +2. Select **Prometheus**. +3. Set the URL to your Prometheus service: + + ``` + http://prometheus-operated.monitoring.svc:9090 + ``` + +4. Click **Save & test**. You should see `Data source is working`. + +## Step 5: Import KubeDB ZooKeeper Dashboards + +The KubeDB ZooKeeper dashboards are distributed as JSON files. Each JSON file is a complete dashboard definition — panels, queries, variables, and layout — that Grafana loads in one shot. Without importing, you would have to build every panel and write every PromQL query by hand. Importing lets you skip that entirely. + +Three dashboards are available. Download all three JSON files from the [appscode/grafana-dashboards](https://github.com/appscode/grafana-dashboards/tree/master/zookeeper) repository (`zookeeper/` folder): + +| File | Dashboard | +|------|-----------| +| `zookeeper_summary_dashboard.json` | KubeDB / ZooKeeper / Summary | +| `zookeeper_pod_dashboard.json` | KubeDB / ZooKeeper / Pod | +| `zookeeper_ensemble_dashboard.json` | KubeDB / ZooKeeper / Ensemble | + +**Import steps (repeat for each of the three files):** + +1. In Grafana, click **Dashboards** in the left sidebar. +2. Select **Import** from the menu. +3. Click **Upload dashboard JSON file** and select one of the downloaded `.json` files. +4. In the **Prometheus** dropdown that appears, select your Prometheus data source. +5. Click **Import**. + +After importing all three files, they will appear under **Dashboards** in the left sidebar. + +## Step 6: Explore the Dashboards + +After opening a dashboard, use the dropdown filters at the top to focus on a specific instance. + +| Variable | Applies to | What to select | +|---------------|----------------|--------------------------------------------------------------| +| **namespace** | All dashboards | Namespace where your ZooKeeper is deployed (e.g., `demo`) | +| **app** | All dashboards | Name of your instance (e.g., `zk-grafana-demo`) | +| **pod** | Pod dashboard | A specific pod, or `All` for an aggregated view | + +**KubeDB / ZooKeeper / Summary** — instance-level overview: + +- **Database Status** — current health of the ZooKeeper ensemble +- **Version** — ZooKeeper version running +- **Quorum Size** — number of replicas in the ensemble +- **Leader** — current leader node endpoint +- **Deletion Policy** — configured deletion policy +- **Total Nodes** — number of pods in the ensemble +- **CPU / Memory Request / Limit** — configured resource requests and limits +- **Storage Request** — configured storage request per pod +- **CPU Usage / CPU Quota** — CPU consumption over time and quota utilization per pod +- **Memory Info** — memory usage over time per pod + +

+ KubeDB ZooKeeper Summary Dashboard +

+ +**KubeDB / ZooKeeper / Pod** — per-pod drill-down: + +- **Status** — current health status of the selected pod +- **Role** — whether this pod is leader or follower +- **Uptime** — how long this pod has been running +- **Snap Count** — number of snapshots taken +- **Commit Count** — number of committed transactions +- **Looking Count** — number of times this node entered leader election +- **znode_count** — number of znodes on this pod over time +- **znode_count_rate** — rate of znode creation/deletion +- **global_sessions / local_sessions** — active client sessions on this pod +- **write_per_namespace / read_per_namespace** — write and read throughput on this pod, broken down by namespace + +

+ KubeDB ZooKeeper Pod Dashboard +

+ +**KubeDB / ZooKeeper / Ensemble** — cross-node aggregate view: + +- **znode_count** — total znodes across all ensemble members over time +- **znode_count_rate** — aggregate rate of znode operations +- **global_sessions** — active global sessions per node endpoint +- **local_sessions** — active local sessions per node endpoint +- **write_per_namespace** — write throughput broken down by namespace +- **read_per_namespace** — read throughput broken down by namespace + +

+ KubeDB ZooKeeper Ensemble Dashboard +

+ +## Cleaning up + +```bash +# Remove the ZooKeeper instance +kubectl delete zookeeper -n demo zk-grafana-demo + +# Remove namespaces +kubectl delete ns demo + +# Uninstall monitoring stack (optional) +helm uninstall prometheus -n monitoring +helm uninstall panopticon -n kubeops +kubectl delete ns monitoring kubeops +``` + +## Next Steps + +- Monitor your ZooKeeper cluster with KubeDB using [built-in Prometheus](/docs/guides/zookeeper/monitoring/using-builtin-prometheus.md). +- Monitor your ZooKeeper cluster with KubeDB using [Prometheus Operator](/docs/guides/zookeeper/monitoring/using-prometheus-operator.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/images/cassandra/monitoring/cas-grafana-database.png b/docs/images/cassandra/monitoring/cas-grafana-database.png new file mode 100644 index 0000000000..26edc8b805 Binary files /dev/null and b/docs/images/cassandra/monitoring/cas-grafana-database.png differ diff --git a/docs/images/cassandra/monitoring/cas-grafana-pod.png b/docs/images/cassandra/monitoring/cas-grafana-pod.png new file mode 100644 index 0000000000..2f6e339cbf Binary files /dev/null and b/docs/images/cassandra/monitoring/cas-grafana-pod.png differ diff --git a/docs/images/cassandra/monitoring/cas-grafana-summary.png b/docs/images/cassandra/monitoring/cas-grafana-summary.png new file mode 100644 index 0000000000..813bae4897 Binary files /dev/null and b/docs/images/cassandra/monitoring/cas-grafana-summary.png differ diff --git a/docs/images/cassandra/monitoring/cas-prom-targets.png b/docs/images/cassandra/monitoring/cas-prom-targets.png new file mode 100644 index 0000000000..355e1da8ef Binary files /dev/null and b/docs/images/cassandra/monitoring/cas-prom-targets.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-database-2.png b/docs/images/clickhouse/monitoring/ch-grafana-database-2.png new file mode 100644 index 0000000000..3260a855e2 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-database-2.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-database.png b/docs/images/clickhouse/monitoring/ch-grafana-database.png new file mode 100644 index 0000000000..21f5f9c163 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-database.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-home.png b/docs/images/clickhouse/monitoring/ch-grafana-home.png new file mode 100644 index 0000000000..ac9810c49c Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-home.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-import.png b/docs/images/clickhouse/monitoring/ch-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-import.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-login.png b/docs/images/clickhouse/monitoring/ch-grafana-login.png new file mode 100644 index 0000000000..88f51a9fad Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-login.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-pod-2.png b/docs/images/clickhouse/monitoring/ch-grafana-pod-2.png new file mode 100644 index 0000000000..933c3e6816 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-pod-2.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-pod.png b/docs/images/clickhouse/monitoring/ch-grafana-pod.png new file mode 100644 index 0000000000..933c3e6816 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-pod.png differ diff --git a/docs/images/clickhouse/monitoring/ch-grafana-summary.png b/docs/images/clickhouse/monitoring/ch-grafana-summary.png new file mode 100644 index 0000000000..fea7c67f14 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-grafana-summary.png differ diff --git a/docs/images/clickhouse/monitoring/ch-prom-targets.png b/docs/images/clickhouse/monitoring/ch-prom-targets.png new file mode 100644 index 0000000000..4a83a8cfd5 Binary files /dev/null and b/docs/images/clickhouse/monitoring/ch-prom-targets.png differ diff --git a/docs/images/druid/monitoring/druid-database.png b/docs/images/druid/monitoring/druid-database.png new file mode 100644 index 0000000000..3a2991077e Binary files /dev/null and b/docs/images/druid/monitoring/druid-database.png differ diff --git a/docs/images/druid/monitoring/druid-grafana-home.png b/docs/images/druid/monitoring/druid-grafana-home.png new file mode 100644 index 0000000000..e966ce4568 Binary files /dev/null and b/docs/images/druid/monitoring/druid-grafana-home.png differ diff --git a/docs/images/druid/monitoring/druid-grafana-import.png b/docs/images/druid/monitoring/druid-grafana-import.png new file mode 100644 index 0000000000..0ac3d0a918 Binary files /dev/null and b/docs/images/druid/monitoring/druid-grafana-import.png differ diff --git a/docs/images/druid/monitoring/druid-grafana-login.png b/docs/images/druid/monitoring/druid-grafana-login.png new file mode 100644 index 0000000000..6a5e8a488d Binary files /dev/null and b/docs/images/druid/monitoring/druid-grafana-login.png differ diff --git a/docs/images/druid/monitoring/druid-pod.png b/docs/images/druid/monitoring/druid-pod.png new file mode 100644 index 0000000000..83eab06b95 Binary files /dev/null and b/docs/images/druid/monitoring/druid-pod.png differ diff --git a/docs/images/druid/monitoring/druid-prom-targets.png b/docs/images/druid/monitoring/druid-prom-targets.png new file mode 100644 index 0000000000..ba4666e00f Binary files /dev/null and b/docs/images/druid/monitoring/druid-prom-targets.png differ diff --git a/docs/images/druid/monitoring/druid-summary.png b/docs/images/druid/monitoring/druid-summary.png new file mode 100644 index 0000000000..8e0ef8a055 Binary files /dev/null and b/docs/images/druid/monitoring/druid-summary.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-database.png b/docs/images/elasticsearch/monitoring/es-grafana-database.png new file mode 100644 index 0000000000..69c93cc158 Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-database.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-home.png b/docs/images/elasticsearch/monitoring/es-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-home.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-import.png b/docs/images/elasticsearch/monitoring/es-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-import.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-login.png b/docs/images/elasticsearch/monitoring/es-grafana-login.png new file mode 100644 index 0000000000..c140417ef6 Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-login.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-pod.png b/docs/images/elasticsearch/monitoring/es-grafana-pod.png new file mode 100644 index 0000000000..555f164b2b Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-pod.png differ diff --git a/docs/images/elasticsearch/monitoring/es-grafana-summary.png b/docs/images/elasticsearch/monitoring/es-grafana-summary.png new file mode 100644 index 0000000000..e808dcafac Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-grafana-summary.png differ diff --git a/docs/images/elasticsearch/monitoring/es-prom-targets.png b/docs/images/elasticsearch/monitoring/es-prom-targets.png new file mode 100644 index 0000000000..d2b4e81add Binary files /dev/null and b/docs/images/elasticsearch/monitoring/es-prom-targets.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-database-2.png b/docs/images/hazelcast/monitoring/hz-grafana-database-2.png new file mode 100644 index 0000000000..08883bc19e Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-database-2.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-database.png b/docs/images/hazelcast/monitoring/hz-grafana-database.png new file mode 100644 index 0000000000..c0e8bd0415 Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-database.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-pod-2.png b/docs/images/hazelcast/monitoring/hz-grafana-pod-2.png new file mode 100644 index 0000000000..49ee49034d Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-pod-2.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-pod.png b/docs/images/hazelcast/monitoring/hz-grafana-pod.png new file mode 100644 index 0000000000..50e77dfc81 Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-pod.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-summary-2.png b/docs/images/hazelcast/monitoring/hz-grafana-summary-2.png new file mode 100644 index 0000000000..7241d42b6b Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-summary-2.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-summary-3.png b/docs/images/hazelcast/monitoring/hz-grafana-summary-3.png new file mode 100644 index 0000000000..cee293b0b5 Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-summary-3.png differ diff --git a/docs/images/hazelcast/monitoring/hz-grafana-summary.png b/docs/images/hazelcast/monitoring/hz-grafana-summary.png new file mode 100644 index 0000000000..c88c60855c Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-grafana-summary.png differ diff --git a/docs/images/hazelcast/monitoring/hz-prom-targets.png b/docs/images/hazelcast/monitoring/hz-prom-targets.png new file mode 100644 index 0000000000..2f16596552 Binary files /dev/null and b/docs/images/hazelcast/monitoring/hz-prom-targets.png differ diff --git a/docs/images/ignite/monitoring/.png b/docs/images/ignite/monitoring/.png new file mode 100644 index 0000000000..13e14751e7 Binary files /dev/null and b/docs/images/ignite/monitoring/.png differ diff --git a/docs/images/ignite/monitoring/ig-grafana-database.png b/docs/images/ignite/monitoring/ig-grafana-database.png new file mode 100644 index 0000000000..1e02ef3201 Binary files /dev/null and b/docs/images/ignite/monitoring/ig-grafana-database.png differ diff --git a/docs/images/ignite/monitoring/ig-grafana-pod.png b/docs/images/ignite/monitoring/ig-grafana-pod.png new file mode 100644 index 0000000000..78f2c517f5 Binary files /dev/null and b/docs/images/ignite/monitoring/ig-grafana-pod.png differ diff --git a/docs/images/ignite/monitoring/ig-grafana-summary.png b/docs/images/ignite/monitoring/ig-grafana-summary.png new file mode 100644 index 0000000000..358a81fb58 Binary files /dev/null and b/docs/images/ignite/monitoring/ig-grafana-summary.png differ diff --git a/docs/images/ignite/monitoring/ig-prom-targets.png b/docs/images/ignite/monitoring/ig-prom-targets.png new file mode 100644 index 0000000000..118d589577 Binary files /dev/null and b/docs/images/ignite/monitoring/ig-prom-targets.png differ diff --git a/docs/images/kafka/monitoring/Kafka-broker-topic-metrics-0.png b/docs/images/kafka/monitoring/Kafka-broker-topic-metrics-0.png new file mode 100644 index 0000000000..038196d064 Binary files /dev/null and b/docs/images/kafka/monitoring/Kafka-broker-topic-metrics-0.png differ diff --git a/docs/images/kafka/monitoring/Kafka-server-metrics-0.png b/docs/images/kafka/monitoring/Kafka-server-metrics-0.png new file mode 100644 index 0000000000..3a66d65566 Binary files /dev/null and b/docs/images/kafka/monitoring/Kafka-server-metrics-0.png differ diff --git a/docs/images/kafka/monitoring/kafka-broker-topic-metrics-1.png b/docs/images/kafka/monitoring/kafka-broker-topic-metrics-1.png new file mode 100644 index 0000000000..2422df1103 Binary files /dev/null and b/docs/images/kafka/monitoring/kafka-broker-topic-metrics-1.png differ diff --git a/docs/images/kafka/monitoring/kafka-kraft-controller-monitoring-metrics.png b/docs/images/kafka/monitoring/kafka-kraft-controller-monitoring-metrics.png new file mode 100644 index 0000000000..d81e17abcc Binary files /dev/null and b/docs/images/kafka/monitoring/kafka-kraft-controller-monitoring-metrics.png differ diff --git a/docs/images/kafka/monitoring/kafka-kraft-quorum-monitoring-metrics.png b/docs/images/kafka/monitoring/kafka-kraft-quorum-monitoring-metrics.png new file mode 100644 index 0000000000..5a000f7e63 Binary files /dev/null and b/docs/images/kafka/monitoring/kafka-kraft-quorum-monitoring-metrics.png differ diff --git a/docs/images/kafka/monitoring/kafka-server-metrics-1.png b/docs/images/kafka/monitoring/kafka-server-metrics-1.png new file mode 100644 index 0000000000..6bcb77a760 Binary files /dev/null and b/docs/images/kafka/monitoring/kafka-server-metrics-1.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-database.png b/docs/images/kafka/monitoring/kf-grafana-database.png new file mode 100644 index 0000000000..fce47c583b Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-database.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-home.png b/docs/images/kafka/monitoring/kf-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-home.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-import.png b/docs/images/kafka/monitoring/kf-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-import.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-login.png b/docs/images/kafka/monitoring/kf-grafana-login.png new file mode 100644 index 0000000000..795c6bf378 Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-login.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-pod.png b/docs/images/kafka/monitoring/kf-grafana-pod.png new file mode 100644 index 0000000000..2fa3177273 Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-pod.png differ diff --git a/docs/images/kafka/monitoring/kf-grafana-summary.png b/docs/images/kafka/monitoring/kf-grafana-summary.png new file mode 100644 index 0000000000..ff13e0c22d Binary files /dev/null and b/docs/images/kafka/monitoring/kf-grafana-summary.png differ diff --git a/docs/images/kafka/monitoring/kf-prom-targets.png b/docs/images/kafka/monitoring/kf-prom-targets.png new file mode 100644 index 0000000000..4085a4e356 Binary files /dev/null and b/docs/images/kafka/monitoring/kf-prom-targets.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-database.png b/docs/images/mariadb/monitoring/mariadb-grafana-database.png new file mode 100644 index 0000000000..355013a4a4 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-database.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-galera.png b/docs/images/mariadb/monitoring/mariadb-grafana-galera.png new file mode 100644 index 0000000000..059af3b00e Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-galera.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-home.png b/docs/images/mariadb/monitoring/mariadb-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-home.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-import.png b/docs/images/mariadb/monitoring/mariadb-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-import.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-login.png b/docs/images/mariadb/monitoring/mariadb-grafana-login.png new file mode 100644 index 0000000000..88f51a9fad Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-login.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-pod.png b/docs/images/mariadb/monitoring/mariadb-grafana-pod.png new file mode 100644 index 0000000000..1bf5b732e6 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-pod.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-grafana-summary.png b/docs/images/mariadb/monitoring/mariadb-grafana-summary.png new file mode 100644 index 0000000000..c2df15de36 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-grafana-summary.png differ diff --git a/docs/images/mariadb/monitoring/mariadb-prom-targets.png b/docs/images/mariadb/monitoring/mariadb-prom-targets.png new file mode 100644 index 0000000000..502b1183a1 Binary files /dev/null and b/docs/images/mariadb/monitoring/mariadb-prom-targets.png differ diff --git a/docs/images/memcached/monitoring/mc-grafana-database.png b/docs/images/memcached/monitoring/mc-grafana-database.png new file mode 100644 index 0000000000..a9cde1caac Binary files /dev/null and b/docs/images/memcached/monitoring/mc-grafana-database.png differ diff --git a/docs/images/memcached/monitoring/mc-grafana-pod.png b/docs/images/memcached/monitoring/mc-grafana-pod.png new file mode 100644 index 0000000000..cce774beb9 Binary files /dev/null and b/docs/images/memcached/monitoring/mc-grafana-pod.png differ diff --git a/docs/images/memcached/monitoring/mc-grafana-summary.png b/docs/images/memcached/monitoring/mc-grafana-summary.png new file mode 100644 index 0000000000..bd5930f933 Binary files /dev/null and b/docs/images/memcached/monitoring/mc-grafana-summary.png differ diff --git a/docs/images/memcached/monitoring/mc-prom-targets.png b/docs/images/memcached/monitoring/mc-prom-targets.png new file mode 100644 index 0000000000..ce19a2e257 Binary files /dev/null and b/docs/images/memcached/monitoring/mc-prom-targets.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-database.png b/docs/images/mongodb/monitoring/mg-grafana-database.png new file mode 100644 index 0000000000..f8f3d4f7c1 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-database.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-home.png b/docs/images/mongodb/monitoring/mg-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-home.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-import.png b/docs/images/mongodb/monitoring/mg-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-import.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-login.png b/docs/images/mongodb/monitoring/mg-grafana-login.png new file mode 100644 index 0000000000..88f51a9fad Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-login.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-pod.png b/docs/images/mongodb/monitoring/mg-grafana-pod.png new file mode 100644 index 0000000000..ca55d26038 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-pod.png differ diff --git a/docs/images/mongodb/monitoring/mg-grafana-summary.png b/docs/images/mongodb/monitoring/mg-grafana-summary.png new file mode 100644 index 0000000000..4f1b1b9ee2 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-grafana-summary.png differ diff --git a/docs/images/mongodb/monitoring/mg-prom-targets.png b/docs/images/mongodb/monitoring/mg-prom-targets.png new file mode 100644 index 0000000000..055a660c88 Binary files /dev/null and b/docs/images/mongodb/monitoring/mg-prom-targets.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-database.png b/docs/images/mssqlserver/monitoring/ms-grafana-database.png new file mode 100644 index 0000000000..8cc39b7138 Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-database.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-home.png b/docs/images/mssqlserver/monitoring/ms-grafana-home.png new file mode 100644 index 0000000000..8c1f22a8fb Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-home.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-import.png b/docs/images/mssqlserver/monitoring/ms-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-import.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-login.png b/docs/images/mssqlserver/monitoring/ms-grafana-login.png new file mode 100644 index 0000000000..c140417ef6 Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-login.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-pod.png b/docs/images/mssqlserver/monitoring/ms-grafana-pod.png new file mode 100644 index 0000000000..10e19adb0b Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-pod.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-grafana-summary.png b/docs/images/mssqlserver/monitoring/ms-grafana-summary.png new file mode 100644 index 0000000000..c7a71eacdd Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-grafana-summary.png differ diff --git a/docs/images/mssqlserver/monitoring/ms-prom-targets.png b/docs/images/mssqlserver/monitoring/ms-prom-targets.png new file mode 100644 index 0000000000..63830a01c8 Binary files /dev/null and b/docs/images/mssqlserver/monitoring/ms-prom-targets.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-database.png b/docs/images/mysql/monitoring/mysql-grafana-database.png new file mode 100644 index 0000000000..b071a45477 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-database.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-group-replication.png b/docs/images/mysql/monitoring/mysql-grafana-group-replication.png new file mode 100644 index 0000000000..48ee460108 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-group-replication.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-home.png b/docs/images/mysql/monitoring/mysql-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-home.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-import.png b/docs/images/mysql/monitoring/mysql-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-import.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-login.png b/docs/images/mysql/monitoring/mysql-grafana-login.png new file mode 100644 index 0000000000..8bf1fb48b5 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-login.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-pod.png b/docs/images/mysql/monitoring/mysql-grafana-pod.png new file mode 100644 index 0000000000..2072959fc1 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-pod.png differ diff --git a/docs/images/mysql/monitoring/mysql-grafana-summary.png b/docs/images/mysql/monitoring/mysql-grafana-summary.png new file mode 100644 index 0000000000..9ad1f36ed1 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-grafana-summary.png differ diff --git a/docs/images/mysql/monitoring/mysql-prom-targets.png b/docs/images/mysql/monitoring/mysql-prom-targets.png new file mode 100644 index 0000000000..8759bd6b63 Binary files /dev/null and b/docs/images/mysql/monitoring/mysql-prom-targets.png differ diff --git a/docs/images/neo4j/monitoring/neo4j-grafana-database.png b/docs/images/neo4j/monitoring/neo4j-grafana-database.png new file mode 100644 index 0000000000..79a3603f6f Binary files /dev/null and b/docs/images/neo4j/monitoring/neo4j-grafana-database.png differ diff --git a/docs/images/neo4j/monitoring/neo4j-grafana-pod.png b/docs/images/neo4j/monitoring/neo4j-grafana-pod.png new file mode 100644 index 0000000000..b5c487ad27 Binary files /dev/null and b/docs/images/neo4j/monitoring/neo4j-grafana-pod.png differ diff --git a/docs/images/neo4j/monitoring/neo4j-grafana-summary.png b/docs/images/neo4j/monitoring/neo4j-grafana-summary.png new file mode 100644 index 0000000000..e647c96d34 Binary files /dev/null and b/docs/images/neo4j/monitoring/neo4j-grafana-summary.png differ diff --git a/docs/images/neo4j/monitoring/neo4j-prom-targets.png b/docs/images/neo4j/monitoring/neo4j-prom-targets.png new file mode 100644 index 0000000000..e1cc693d68 Binary files /dev/null and b/docs/images/neo4j/monitoring/neo4j-prom-targets.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-database.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-database.png new file mode 100644 index 0000000000..6ef2709595 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-database.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-galera.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-galera.png new file mode 100644 index 0000000000..7d61262ddd Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-galera.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-home.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-home.png new file mode 100644 index 0000000000..5510225218 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-home.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-import.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-import.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-login.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-login.png new file mode 100644 index 0000000000..c140417ef6 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-login.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-pod.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-pod.png new file mode 100644 index 0000000000..fa49752a4f Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-pod.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-grafana-summary.png b/docs/images/percona-xtradb/monitoring/pxc-grafana-summary.png new file mode 100644 index 0000000000..dc77e189b0 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-grafana-summary.png differ diff --git a/docs/images/percona-xtradb/monitoring/pxc-prom-targets.png b/docs/images/percona-xtradb/monitoring/pxc-prom-targets.png new file mode 100644 index 0000000000..a2d7f232e7 Binary files /dev/null and b/docs/images/percona-xtradb/monitoring/pxc-prom-targets.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-database.png b/docs/images/pgbouncer/monitoring/pb-grafana-database.png new file mode 100644 index 0000000000..5ccafd473b Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-database.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-home.png b/docs/images/pgbouncer/monitoring/pb-grafana-home.png new file mode 100644 index 0000000000..bbd8ec90b8 Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-home.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-import.png b/docs/images/pgbouncer/monitoring/pb-grafana-import.png new file mode 100644 index 0000000000..b4e6f1133f Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-import.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-login.png b/docs/images/pgbouncer/monitoring/pb-grafana-login.png new file mode 100644 index 0000000000..12b7cd050d Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-login.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-pod.png b/docs/images/pgbouncer/monitoring/pb-grafana-pod.png new file mode 100644 index 0000000000..625f8c31e9 Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-pod.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-grafana-summary.png b/docs/images/pgbouncer/monitoring/pb-grafana-summary.png new file mode 100644 index 0000000000..a1602b67f6 Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-grafana-summary.png differ diff --git a/docs/images/pgbouncer/monitoring/pb-prom-targets.png b/docs/images/pgbouncer/monitoring/pb-prom-targets.png new file mode 100644 index 0000000000..fdee9728c9 Binary files /dev/null and b/docs/images/pgbouncer/monitoring/pb-prom-targets.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-database.png b/docs/images/pgpool/monitoring/pp-grafana-database.png new file mode 100644 index 0000000000..5a214fbeba Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-database.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-home.png b/docs/images/pgpool/monitoring/pp-grafana-home.png new file mode 100644 index 0000000000..cbf1c11842 Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-home.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-import.png b/docs/images/pgpool/monitoring/pp-grafana-import.png new file mode 100644 index 0000000000..0ac3d0a918 Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-import.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-login.png b/docs/images/pgpool/monitoring/pp-grafana-login.png new file mode 100644 index 0000000000..6a5e8a488d Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-login.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-pod.png b/docs/images/pgpool/monitoring/pp-grafana-pod.png new file mode 100644 index 0000000000..f21c1d3bd9 Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-pod.png differ diff --git a/docs/images/pgpool/monitoring/pp-grafana-summary.png b/docs/images/pgpool/monitoring/pp-grafana-summary.png new file mode 100644 index 0000000000..f5a2dfe134 Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-grafana-summary.png differ diff --git a/docs/images/pgpool/monitoring/pp-prom-targets.png b/docs/images/pgpool/monitoring/pp-prom-targets.png new file mode 100644 index 0000000000..0f2ac74712 Binary files /dev/null and b/docs/images/pgpool/monitoring/pp-prom-targets.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-database-2.png b/docs/images/postgres/monitoring/pg-grafana-database-2.png new file mode 100644 index 0000000000..41642fd24d Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-database-2.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-database.png b/docs/images/postgres/monitoring/pg-grafana-database.png new file mode 100644 index 0000000000..41642fd24d Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-database.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-home.png b/docs/images/postgres/monitoring/pg-grafana-home.png new file mode 100644 index 0000000000..3f43f584e7 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-home.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-import.png b/docs/images/postgres/monitoring/pg-grafana-import.png new file mode 100644 index 0000000000..be4c04a632 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-import.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-login.png b/docs/images/postgres/monitoring/pg-grafana-login.png new file mode 100644 index 0000000000..97cc3b5cb4 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-login.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-pod-2.png b/docs/images/postgres/monitoring/pg-grafana-pod-2.png new file mode 100644 index 0000000000..8582b874bc Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-pod-2.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-pod.png b/docs/images/postgres/monitoring/pg-grafana-pod.png new file mode 100644 index 0000000000..8582b874bc Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-pod.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-summary-2.png b/docs/images/postgres/monitoring/pg-grafana-summary-2.png new file mode 100644 index 0000000000..070b643910 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-summary-2.png differ diff --git a/docs/images/postgres/monitoring/pg-grafana-summary.png b/docs/images/postgres/monitoring/pg-grafana-summary.png new file mode 100644 index 0000000000..35a888ce80 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-grafana-summary.png differ diff --git a/docs/images/postgres/monitoring/pg-prom-targets.png b/docs/images/postgres/monitoring/pg-prom-targets.png new file mode 100644 index 0000000000..55bcb35925 Binary files /dev/null and b/docs/images/postgres/monitoring/pg-prom-targets.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-database.png b/docs/images/proxysql/monitoring/proxysql-grafana-database.png new file mode 100644 index 0000000000..71e6968e6a Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-database.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-home.png b/docs/images/proxysql/monitoring/proxysql-grafana-home.png new file mode 100644 index 0000000000..ac9810c49c Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-home.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-import.png b/docs/images/proxysql/monitoring/proxysql-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-import.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-login.png b/docs/images/proxysql/monitoring/proxysql-grafana-login.png new file mode 100644 index 0000000000..795c6bf378 Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-login.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-pod.png b/docs/images/proxysql/monitoring/proxysql-grafana-pod.png new file mode 100644 index 0000000000..a0bccebba5 Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-pod.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-grafana-summary.png b/docs/images/proxysql/monitoring/proxysql-grafana-summary.png new file mode 100644 index 0000000000..817fa7e021 Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-grafana-summary.png differ diff --git a/docs/images/proxysql/monitoring/proxysql-prom-targets.png b/docs/images/proxysql/monitoring/proxysql-prom-targets.png new file mode 100644 index 0000000000..b354060aec Binary files /dev/null and b/docs/images/proxysql/monitoring/proxysql-prom-targets.png differ diff --git a/docs/images/qdrant/monitoring/qdrant-grafana-database.png b/docs/images/qdrant/monitoring/qdrant-grafana-database.png new file mode 100644 index 0000000000..fe77337409 Binary files /dev/null and b/docs/images/qdrant/monitoring/qdrant-grafana-database.png differ diff --git a/docs/images/qdrant/monitoring/qdrant-grafana-pod.png b/docs/images/qdrant/monitoring/qdrant-grafana-pod.png new file mode 100644 index 0000000000..413d6ce1e8 Binary files /dev/null and b/docs/images/qdrant/monitoring/qdrant-grafana-pod.png differ diff --git a/docs/images/qdrant/monitoring/qdrant-grafana-summary.png b/docs/images/qdrant/monitoring/qdrant-grafana-summary.png new file mode 100644 index 0000000000..9a0fd6b79c Binary files /dev/null and b/docs/images/qdrant/monitoring/qdrant-grafana-summary.png differ diff --git a/docs/images/qdrant/monitoring/qdrant-prom-targets.png b/docs/images/qdrant/monitoring/qdrant-prom-targets.png new file mode 100644 index 0000000000..06e9516049 Binary files /dev/null and b/docs/images/qdrant/monitoring/qdrant-prom-targets.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-database-2.png b/docs/images/rabbitmq/monitoring/rmq-grafana-database-2.png new file mode 100644 index 0000000000..70e909a48a Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-database-2.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-database.png b/docs/images/rabbitmq/monitoring/rmq-grafana-database.png new file mode 100644 index 0000000000..70e909a48a Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-database.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-home.png b/docs/images/rabbitmq/monitoring/rmq-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-home.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-import.png b/docs/images/rabbitmq/monitoring/rmq-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-import.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-login.png b/docs/images/rabbitmq/monitoring/rmq-grafana-login.png new file mode 100644 index 0000000000..88f51a9fad Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-login.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-pod-2.png b/docs/images/rabbitmq/monitoring/rmq-grafana-pod-2.png new file mode 100644 index 0000000000..33a5bc3031 Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-pod-2.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-pod.png b/docs/images/rabbitmq/monitoring/rmq-grafana-pod.png new file mode 100644 index 0000000000..33a5bc3031 Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-pod.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-grafana-summary.png b/docs/images/rabbitmq/monitoring/rmq-grafana-summary.png new file mode 100644 index 0000000000..75ea656b79 Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-grafana-summary.png differ diff --git a/docs/images/rabbitmq/monitoring/rmq-prom-targets.png b/docs/images/rabbitmq/monitoring/rmq-prom-targets.png new file mode 100644 index 0000000000..7b51ed56ae Binary files /dev/null and b/docs/images/rabbitmq/monitoring/rmq-prom-targets.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-home.png b/docs/images/redis/monitoring/rd-grafana-home.png new file mode 100644 index 0000000000..19b6e887d3 Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-home.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-import.png b/docs/images/redis/monitoring/rd-grafana-import.png new file mode 100644 index 0000000000..b2b85699e4 Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-import.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-login.png b/docs/images/redis/monitoring/rd-grafana-login.png new file mode 100644 index 0000000000..795c6bf378 Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-login.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-pod.png b/docs/images/redis/monitoring/rd-grafana-pod.png new file mode 100644 index 0000000000..5c40920786 Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-pod.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-shard.png b/docs/images/redis/monitoring/rd-grafana-shard.png new file mode 100644 index 0000000000..d9c00d55b9 Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-shard.png differ diff --git a/docs/images/redis/monitoring/rd-grafana-summary.png b/docs/images/redis/monitoring/rd-grafana-summary.png new file mode 100644 index 0000000000..d045ece70d Binary files /dev/null and b/docs/images/redis/monitoring/rd-grafana-summary.png differ diff --git a/docs/images/redis/monitoring/rd-prom-targets.png b/docs/images/redis/monitoring/rd-prom-targets.png new file mode 100644 index 0000000000..7a560d3835 Binary files /dev/null and b/docs/images/redis/monitoring/rd-prom-targets.png differ diff --git a/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 11-50-07.png b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 11-50-07.png new file mode 100644 index 0000000000..2f5d2b6a83 Binary files /dev/null and b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 11-50-07.png differ diff --git a/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-25-30.png b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-25-30.png new file mode 100644 index 0000000000..943dd56a21 Binary files /dev/null and b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-25-30.png differ diff --git a/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-26-39.png b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-26-39.png new file mode 100644 index 0000000000..0983f005cf Binary files /dev/null and b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-26-39.png differ diff --git a/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-51-23.png b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-51-23.png new file mode 100644 index 0000000000..4a0ed9c04b Binary files /dev/null and b/docs/images/singlestore/monitoring/Screenshot from 2026-06-26 12-51-23.png differ diff --git a/docs/images/singlestore/monitoring/sdb-grafana-database.png b/docs/images/singlestore/monitoring/sdb-grafana-database.png new file mode 100644 index 0000000000..1908c57da0 Binary files /dev/null and b/docs/images/singlestore/monitoring/sdb-grafana-database.png differ diff --git a/docs/images/singlestore/monitoring/sdb-grafana-import.png b/docs/images/singlestore/monitoring/sdb-grafana-import.png new file mode 100644 index 0000000000..000a969a79 Binary files /dev/null and b/docs/images/singlestore/monitoring/sdb-grafana-import.png differ diff --git a/docs/images/singlestore/monitoring/sdb-grafana-pod.png b/docs/images/singlestore/monitoring/sdb-grafana-pod.png new file mode 100644 index 0000000000..219d6ce588 Binary files /dev/null and b/docs/images/singlestore/monitoring/sdb-grafana-pod.png differ diff --git a/docs/images/singlestore/monitoring/sdb-grafana-summary.png b/docs/images/singlestore/monitoring/sdb-grafana-summary.png new file mode 100644 index 0000000000..170cfa7a19 Binary files /dev/null and b/docs/images/singlestore/monitoring/sdb-grafana-summary.png differ diff --git a/docs/images/singlestore/monitoring/sdb-prom-targets.png b/docs/images/singlestore/monitoring/sdb-prom-targets.png new file mode 100644 index 0000000000..2654490794 Binary files /dev/null and b/docs/images/singlestore/monitoring/sdb-prom-targets.png differ diff --git a/docs/images/solr/monitoring/sl-grafana-database.png b/docs/images/solr/monitoring/sl-grafana-database.png new file mode 100644 index 0000000000..2c474c3666 Binary files /dev/null and b/docs/images/solr/monitoring/sl-grafana-database.png differ diff --git a/docs/images/solr/monitoring/sl-grafana-pod.png b/docs/images/solr/monitoring/sl-grafana-pod.png new file mode 100644 index 0000000000..f8b632d25c Binary files /dev/null and b/docs/images/solr/monitoring/sl-grafana-pod.png differ diff --git a/docs/images/solr/monitoring/sl-grafana-summary.png b/docs/images/solr/monitoring/sl-grafana-summary.png new file mode 100644 index 0000000000..052208c027 Binary files /dev/null and b/docs/images/solr/monitoring/sl-grafana-summary.png differ diff --git a/docs/images/solr/monitoring/sl-prom-targets.png b/docs/images/solr/monitoring/sl-prom-targets.png new file mode 100644 index 0000000000..fd82c9dd9a Binary files /dev/null and b/docs/images/solr/monitoring/sl-prom-targets.png differ diff --git a/docs/images/zookeeper/monitoring/zk-grafana-ensemble.png b/docs/images/zookeeper/monitoring/zk-grafana-ensemble.png new file mode 100644 index 0000000000..c63e8150a0 Binary files /dev/null and b/docs/images/zookeeper/monitoring/zk-grafana-ensemble.png differ diff --git a/docs/images/zookeeper/monitoring/zk-grafana-pod.png b/docs/images/zookeeper/monitoring/zk-grafana-pod.png new file mode 100644 index 0000000000..df87c03630 Binary files /dev/null and b/docs/images/zookeeper/monitoring/zk-grafana-pod.png differ diff --git a/docs/images/zookeeper/monitoring/zk-grafana-summary.png b/docs/images/zookeeper/monitoring/zk-grafana-summary.png new file mode 100644 index 0000000000..060f4d5906 Binary files /dev/null and b/docs/images/zookeeper/monitoring/zk-grafana-summary.png differ diff --git a/docs/images/zookeeper/monitoring/zk-prom-targets.png b/docs/images/zookeeper/monitoring/zk-prom-targets.png new file mode 100644 index 0000000000..dc8f4a3cb4 Binary files /dev/null and b/docs/images/zookeeper/monitoring/zk-prom-targets.png differ