u/GroundbreakingBed597

Hi. There is a lot of great documentation and sample on how to observe your application with OTel. Whether its how you instrument (manual, vs automated, vs zero code), best practices on OTel collectors, sampling ...

Most of those examples however are based on demo apps. While those demo apps have a certain complexity they typically dont reflect multi-stage enterprise environments with shared services across many differerent digital services.

I have also seen the new OTel Blueprints which is a great way to see how some orgs are implementing OTel at Enterprise Scale

I am just curious to learn from the reddit audience on what else someone that tries to go from Demo App to Enterprise Scale needs to consider. Here some questions I have and I hope everyone can chime in:

* How to best validate quality of instrumentation?

* Besides the semantic conventions - what other metadata on logs, metrics, spans ... are essential

* Is anyone really doing tail-based sampling at scale or is it just too costly?

* How do you rollout and manage large sets of collectors?

Thanks in advance

I just watched a presentation from the OpenSource Summit Noram done by Henrik Rexed.

He presented his OpenTelemetry Collector processor called gatewayapiprocessor that enriches spans, logs, and metrics with normalized Kubernetes Gateway API attributes — k8s.gateway.*, k8s.httproute.*, k8s.gatewayclass.* — parsed from the opaque route_name strings emitted by Envoy-family controllers (Envoy Gateway, Kgateway, Istio) and from Linkerd's route labels.

Really neat project that makes it easier when analyzing your observability data coming out of your service meshes.

I am not sure if I am allowed to post links here - but - if you are interested in this you can easily find his github repo and the recording of his talk on YouTube with the title "The Legend of Config: Breath of the Cluster"