r/grafana

Grafana Labs security update: Latest on TanStack npm supply chain ransomware incident
▲ 46 r/grafana

Grafana Labs security update: Latest on TanStack npm supply chain ransomware incident

This weekend, we confirmed a targeted attack by a cybercrime group that gained unauthorized access to our GitHub repositories and downloaded our codebase. 

Grafana Labs CISO Joe McManus has published a blog post that provides the latest update about our investigations. Copied and pasted below....

On May 16, 2026, Grafana Labs confirmed a targeted attack by a cybercrime group that gained unauthorized access to our GitHub repositories and downloaded our codebase. They then issued a ransom demand under threat of data disclosure. 

Since we posted our initial findings that day, our investigation has continued, and we are publishing this blog to share more details about our incident response and mitigation. A post-incident report will be published when our investigation is complete.

To date, the investigation has found no evidence that customer production systems or operations have been compromised. This incident was strictly limited to the Grafana Labs GitHub environment and did not affect our production systems or the Grafana Cloud platform.

After the initial assessment, we found that in addition to source code, the downloaded content included GitHub repositories that some Grafana Labs teams use to collaborate on and store internal operational information and other details about our business. This includes business contact names and email addresses that would be exchanged in a professional relationship context, not information pulled from or processed through the use of production systems or the Grafana Cloud platform. 

To be clear to the users of Grafana Labs' open source projects and the Grafana Cloud platform: our codebase was downloaded, but it was not altered. No action is needed from our customers or open source users at this time.

Our investigation is ongoing as we continue to review logs, telemetry, and all available data within our company-wide GitHub repos. Should we ever determine that any customer's systems or operations are impacted, we will notify them directly.

At Grafana Labs, earning and maintaining our community’s trust is foundational to everything we do. We recognize that customers rely on us as a trusted partner, and we do not take that responsibility lightly. We are sharing this update in the spirit of transparency because we understand you may have questions and because we take this matter seriously.

Summary and background

The incident originated from a TanStack npm supply chain attack via the Mini Shai-Hulud campaign. We detected the malicious activity on May 11 and immediately initiated our incident response plan. 

We performed analysis and quickly rotated a significant number of GitHub workflow tokens, but a missed token led to the attackers gaining access to our GitHub repositories. A subsequent review confirmed that a specific GitHub workflow we originally deemed not impacted had, in fact, been compromised.

On May 16, we received a demand from a bad actor for a ransom payment to prevent the release of our codebase. Grafana Labs determined the appropriate path forward is not to pay the ransom. This decision aligns with the FBI’s formal position that paying a ransom does not guarantee security and only serves to incentivize further criminal enterprise.

As soon as we were contacted by the ransom gang, we launched mitigation efforts, which have included rotating automation tokens, implementing enhanced monitoring, auditing all commits since the May 11 incident, and significantly hardening our GitHub security posture.

We have also notified federal law enforcement and will maintain an ongoing dialogue with them about the situation.  

Impact and response

Current findings indicate the scope of this incident is limited to the Grafana Labs GitHub repositories, which include public and private source code along with internal GitHub repos. 

There is no evidence that customer production systems or operations have been compromised. 

As part of our standard security practices, we will share additional information from our post-incident review when our investigation is complete.

Grafana Labs is also taking steps to increase security measures to protect our systems. We are currently implementing significant measures to further secure our CI/CD (continuous integration and continuous deployment) pipelines and prevent a recurrence of this type of issue.

Our teams remain focused on the continued investigation and the deployment of increased security controls.

u/vidamon — 2 days ago

Why does the sample compose.yaml for Grafana Loki use three instances of loki? And how do I use local storage instead of minio?

Grafana Loki was recommended to me for centralized logging and so I set up Loki using the provided sample files here: https://grafana.com/docs/loki/latest/setup/install/docker/#install-with-docker-compose but looking at it I don't understand the need for three different Loki containers. Which ones do I actually need? And how do I have Loki use standard filesystem storage instead of minio?

u/UntouchedWagons — 3 days ago
▲ 39 r/grafana+14 crossposts

I added dedicated AWS / EKS support to KubeShark.

Mini recap:

KubeShark is my Kubernetes skill for Claude Code and Codex.

It helps AI agents generate, review, and refactor Kubernetes manifests without falling into the usual LLM traps: missing security contexts, deprecated API versions, broken selectors, wildcard RBAC, unsafe probes, missing resource requests, and rollout configs that look okay but fail under real traffic.

The important part is that KubeShark is failure-mode-first. It does not just tell the model “write good Kubernetes”. It forces the model to reason about what can go wrong before it generates YAML, and then return validation and rollback guidance as part of the answer.

That matters a lot with Kubernetes, because many bad manifests are accepted by the API server and only fail later at runtime.

Repo: https://github.com/LukasNiessen/kubernetes-skill

---

Now what’s new:

KubeShark now has special dedicated AWS / EKS support.

When the task involves EKS, AWS, IRSA, EKS Pod Identity, AWS Load Balancer Controller, EBS/EFS CSI, AWS VPC CNI, or Karpenter, KubeShark switches into EKS-aware guidance.

That matters because EKS is “just Kubernetes” until identity, load balancing, storage, pod networking, and node provisioning enter the picture.

Common LLM mistakes include:

  • putting AWS access keys into Kubernetes Secrets
  • mixing IRSA and EKS Pod Identity assumptions
  • using nginx annotations with AWS Load Balancer Controller
  • treating EBS like ReadWriteMany storage
  • recommending Karpenter while omitting resource requests
  • assuming NetworkPolicy works without checking the CNI/policy engine

Example guidance KubeShark now keeps in mind:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app
  namespace: payments
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/payments-app

It also knows that EBS is usually RWO and zone-sensitive, EFS is the RWX option, and Karpenter depends heavily on good workload requests.

So instead of generic Kubernetes advice, you get EKS-aware manifest generation and review.

u/trolleid — 4 days ago
▲ 11 r/grafana

[Tool] Grafana Final Scanner - Mass CVE Testing Script with All Public CVEs Aggregated.

Hey everyone, I aggregated and curated all public Grafana CVEs into a single, high-speed Python script to make testing mass targets easier for bug hunters and red teamers. Zero dependencies, clean terminal output, and ready for automation.
github: https://github.com/Zierax/Grafana-Final-Scanner

u/Decent-Assistance-50 — 4 days ago

Provisioned Grafana Dashboard Issues

Can anyone explain to me the intended workflow of provisioned dashboards? I can't for the life of me figure out how they work.

My previous setup in v12 allowed me to copy the code out to a .json file and the dashboard would automatically provision if placed in the appropriate directory. Now there is an "edit as code" that provides one configuration and a "Export as code" but neither one works. It just provisions a blank dashboard with no panels. I can seemingly take the "edit as code" and paste it in manually to a new dashboard and that works but I'd much rather manage my config as code if at all possible.

I've spent hours chasing this down. I'm on the latest Grafana version (v13.0.1)... I've read through the documentation, I have to be missing something obvious.

reddit.com
u/plzreboot — 6 days ago
▲ 13 r/grafana

Compare time-shifted queries and alerts across datasources, now with Grafana 13+ support

Hi r/grafana,

I built **CompareQueries**, a Grafana datasource plugin for comparing time-shifted queries across datasources in one panel.

The latest version 2.1.0 supports Grafana 11, 12, and 13+. It is useful for workflows like:

- Time-shifted dashboard queries, eg. today vs yesterday dashboard data

- this week vs last week latency

- before vs after release error rates

- custom time-shifted comparisons in dashboards

- shifted comparison queries in Grafana Alerting

It works with Prometheus, Loki, Elasticsearch, SQL, CSV, and other Grafana datasources through their native query editors.

https://preview.redd.it/r9ptoiu4j91h1.png?width=2632&format=png&auto=webp&s=6dd84a2245baee16028d2ddb6474395391462a89

GitHub:

https://github.com/leoswing/comparequeries-datasource-rc

Grafana Plugin Catalog:

https://grafana.com/grafana/plugins/leoswing-comparequeries-datasource/

Feedback, dashboard examples, feature requests, and bug reports are very welcome.

If CompareQueries helps with your Grafana dashboards or alerting workflows, please consider giving the GitHub repo a star so more users can discover it.

reddit.com
u/Necessary_Soup3876 — 7 days ago
▲ 13 r/grafana

r/grafana updates for better experience: flairs

Some updates:

  • Post flairs: We've added post flairs so Redditors here can easily see what topic a post is about and participate accordingly. Current flairs:
    • Alerting
    • Alloy
    • Assistant
    • Beyla (donated to CNCF and renamed: OpenTelemetry eBPF Instrumentation)
    • Faro
    • Grafana
    • Grafana Cloud
    • k6
    • Kubernetes
    • Loki
    • Mimir
    • OpenTelemetry
    • Prometheus
    • Pyroscope
    • Tanka
    • Tempo
    • Miscellaneous
    • Check it out
  • Grafanista (Grafana Labs employee flairs): Our Grafanistas do a pretty good job identifying themselves as employees when they respond to questions. However, we're taking this one step further by adding these flairs so it's more clear.

Thanks for being part of our this community. Open to feedback + suggestions as we continue making the Reddit experience easier and fun for everyone here.

[Edit: Added Kubernetes to the list]

reddit.com
u/vidamon — 7 days ago
▲ 37 r/grafana+4 crossposts

I built a repo of ready-to-run OpenTelemetry Collector configs (Prometheus, Jaeger, Dynatrace, Datadog, Loki, k8s), feedback welcome

I just open-sourced a collection of ready-to-run OpenTelemetry

Collector configurations, because finding complete, working configs

for your specific backend always takes hours of trial and error.

It now includes examples for:

  • Prometheus
  • Jaeger
  • Grafana Loki
  • Dynatrace
  • Datadog
  • Kubernetes Operator
  • Kubernetes Pod Annotation Scraping (with full relabeling)
  • Debug (no backend needed, perfect for local dev)

Each example includes Docker Compose so you can run it in 60 seconds.

The k8s pod annotation scraping example includes relabeling for

prometheus.io/scrape, prometheus.io/port, and prometheus.io/path

annotations, the config everyone googles when setting up k8s monitoring.

I also actively contribute to the OpenTelemetry open source project,

recently got PRs merged into open-telemetry/otel-arrow and have PRs

open in opentelemetry-android, opentelemetry-helm-charts, and

opentelemetry-dotnet-instrumentation.

https://github.com/Cloud-Architect-Emma/opentelemetry-collector-examples

Feedback and contributions welcome! ⭐ if it's useful.

#OpenTelemetry #DevOps #Observability #Kubernetes #SRE #Monitoring #CloudNative #OpenSource

u/EmmaOpu — 11 days ago
▲ 25 r/grafana+4 crossposts

paradedb/benchmarker: a workload agnostic, multi-backend benchmarking tool.

Hi r/postgresql!

We just open sourced ParadeDB Benchmarker, a multi-backend benchmarking framework built on top of the excellent Grafana k6 (blog post).

One of the goals was avoiding a shared query abstraction layer. PostgreSQL queries stay PostgreSQL queries, with their own driver and native SQL.

Supports PostgreSQL, Elasticsearch, OpenSearch, ClickHouse, MongoDB, and ParadeDB with:

  • mixed read/write workloads
  • support for docker-compose profiles per backend
  • dataset loader
  • config and setup capture
  • live metrics + exported reports

One of the ah-ha moments I had building this was using the pgx Go driver in anger for the first time, I'm a Rust guy, but I'm seriously impressed with pgx and what it can do.

Any comments welcome, we will be using this to benchmark ParadeDB, but you can write your own datasets and workloads which have nothing to do with full-text search.

github.com
u/jamesgresql — 9 days ago
▲ 21 r/grafana

Grafana dashboard for Claude Code CLI metrics on a Prometheus-compatible backend

Hi! I'm an SRE who got pretty excited when Claude Code added the ability to emit OpenTelemetry metrics. Felt like that capability landed pretty quietly out there, so I built a Grafana dashboard on top.

It consumes Claude Code's OTLP metrics on Prometheus-compatible backends (Prometheus, VictoriaMetrics, Mimir, Thanos), all queries in PromQL.

https://preview.redd.it/91di760hoo0h1.png?width=1840&format=png&auto=webp&s=4f36834f24ff6f38c840ed23d37add196557e2dd

Panels: cost by model/project/user, cache hit ratio, active time, edit-decision breakdowns, leaderboards. Custom labels for per-team / per-project views via OTEL_RESOURCE_ATTRIBUTES.

Parallel implementation of dashboard 25052 by 1w2w3y, which targets Azure Application Insights via KQL. Every panel rewritten in PromQL for the OSS metrics stack. Credit to that author for the original concept.

https://preview.redd.it/8bzzqlikoo0h1.png?width=1833&format=png&auto=webp&s=0343f83bb6e092c5e6ed8e4a25496d48b07e1c90

Direct download: https://grafana.com/grafana/dashboards/25255-claude-code-metrics-prometheus/

Article: https://rockdarko.dev/posts/grafana-dashboard-for-claude-code-on-prometheus/

Repo (MIT, PRs welcome): https://github.com/rockdarko/claude-code-metrics-prometheus

Happy to answer questions about the panel queries or extend with what people want.

reddit.com
u/rockdarko — 10 days ago

Listen syslog with different formats in the same alloy instance

So we wan't to use Alloy as a log collector and parser for all of our network devices. We have different brands and models that use several syslog formats (rfc3164, rfc5424, fortinet...) but we want to use the same alloy instance for all of them.

How should we implement it? Should each format have its own listener with a custom port?

reddit.com
u/Budget-Industry-3125 — 9 days ago

Overview

Writes

Working dashboard

I don't know what happened, out of nowhere mimir dashboard data is missing. Not only the overview but other mimir dashboards as well. I'm using the helm chart mimir-distributed. The values are just limits and nothing special. These are my monitoring values though.

metaMonitoring:
  dashboards:
    enabled: true
    labels:
      grafana_dashboard: "1"
  serviceMonitor:
    enabled: true
    interval: 15s

Could I be missing something? Any help would be appreciated.

reddit.com
u/csantve — 14 days ago