CVE-2023-45142 OpenTelemetry Go Contrib HTTP DoS Cardinality Fix 0.44.0

OpenTelemetry‑Go Contrib’s HTTP instrumentation contains a subtle but serious denial‑of‑service vector: unbounded cardinality in HTTP labels allows an attacker to exhaust memory through repeated requests that introduce ever‑new label values, a flaw tracked as CVE‑2023‑45142 and fixed in the 0.44.0 release of the contrib modules.

Background / Overview​

OpenTelemetry for Go (commonly referred to as OpenTelemetry‑Go) is a de facto standard for instrumenting Go services with traces, metrics, and logs. The OpenTelemetry‑Go Contrib repository provides convenient wrappers and instrumentation adapters for many popular HTTP frameworks and libraries, exposing automatic metrics and attributes for incoming requests to speed adoption and reduce boilerplate.
One of those helpers, the HTTP handler wrapper (commonly created with otelhttp.NewHandler), automatically attaches request attributes such as HTTP method and User‑Agent as metric labels. Those labels are useful for diagnosing traffic patterns, but they become dangerous when their cardinality is not limited: if the instrumentation records every distinct method string or User‑Agent string as a separate label value, an attacker can feed a continuous stream of unique values and force the exporter or metrics pipeline to allocate unbounded memory for label sets. Multiple vulnerability trackers and security advisories characterize this weakness as a memory‑exhaustion / DoS issue.
This article explains the technical root cause, maps affected components and versions, reviews real‑world exploit scenarios and detection strategies, and provides concrete remediation and mitigation guidance for operators, developers, and security teams. The analysis cross‑references vendor advisories, third‑party vulnerability databases, and security vendor writeups to ensure accuracy and give administrators the context they need to make informed decisions.

---

How the vulnerability works: unbound cardinality in metrics​

Instrumentation vs. observability tradeoffs​

Observability frameworks routinely convert request attributes into dimensions (labels/tags) on metrics. High‑value attributes — such as HTTP status codes, service names, or static route identifiers — are low cardinality and safe to use as labels. By contrast, attributes that vary per request or are directly controlled by external clients — such as User‑Agent headers, IP addresses, or custom method strings — are high cardinality and can blow up label cardinality if naively exposed.
OpenTelemetry‑Go Contrib’s otelhttp instrumentation originally recorded these higher‑cardinality attributes without imposing limits. The library used internal helpers (notably httpconv.ServerRequest) that would record each distinct value seen for an attribute like HTTP method or User‑Agent. In environments where metrics are aggregated and label values are retained in memory (for example, by the SDK’s in‑process metric storage or by a metrics exporter that keeps a registry keyed by label sets), that behavior creates a direct avenue for unbounded memory growth.

The attack pattern​

• Attacker crafts HTTP requests with unique or randomized header values that are mapped directly into metric labels — e.g., a different User‑Agent string per request or unusual HTTP method strings not commonly used by legitimate clients.
• The instrumented handler records each unique value as a separate label value, which forces the metrics storage layer to allocate new state for each combination of metric name and label set.
• Over time, as the number of distinct label values grows, memory consumption increases until the process exhausts available memory or the metrics pipeline becomes saturated and starts failing — producing an operational denial‑of‑service condition.

This pattern is straightforward to trigger because both the HTTP method and User‑Agent headers are trivial to set in many HTTP clients, including simple command‑line tools or small scripts. Security advisories characterize exploitation as easy when the vulnerable wrappers are used without additional filtering.

---

Affected components and versions​

Directly affected modules​

Vulnerability trackers and vendor advisories identify the problem in the OpenTelemetry‑Go Contrib ecosystem, with the otelhttp handler being the canonical vulnerable component. The issue also affected several framework‑specific instrumentations that rely on the same attribute‑collection mechanisms, including (but not necessarily limited to):

• go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp (core HTTP wrapper)
• go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful
• go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin
• go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux
• go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho
• go.opentelemetry.io/contrib/instrumentation/gopkg.in/macaron.v1/otelmacaron
• go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace

Multiple vulnerability summaries list the affected components as any contrib packages that used the common httpconv.ServerRequest path and collected http.method or http.user_agent without bounding cardinality.

Fixed versions​

The OpenTelemetry‑Go Contrib maintainers addressed the issue in the release that introduced more restrictive collection of method values and the removal of other high‑cardinality attributes from default metrics. In public advisories and vulnerability entries, the fix is associated with version 0.44.0 (and corresponding downstream releases that include the change). Administrators are advised to upgrade any affected contrib packages to 0.44.0 or later to receive the fix.

Severity and CVSS​

Several vulnerability databases and vendor bulletins assign CVSS v3.x scores in the High range (commonly 7.5) for CVE‑2023‑45142, reflecting a remote, easily exploitable denial‑of‑service that can cause complete availability loss without requiring authentication. The score captures the network attack vector, low complexity, and impact to availability. While the vulnerability does not directly allow code execution or data exfiltration, the operational consequences are severe for services that rely on the affected instrumentation.

---

Technical analysis: what changed in the fix​

Limiting method values and removing high‑cardinality attributes​

The fix implemented by the OpenTelemetry‑Go maintainers took a pragmatic approach: reduce the attack surface by limiting which HTTP method values are recorded and by removing other attributes that present high cardinality risks.

• Method restriction: instead of recording every observed method string, the instrumentation now restricts method labels to a known, small set of well‑known HTTP methods (for example, GET, POST, PUT, DELETE, etc.). Unknown or malformed method tokens are mapped to a generic bucket, preventing an explosion of distinct label values.
• User‑Agent / other attributes: high cardinality attributes that were previously enabled by default were removed from the set of exported metric labels, or their collection was made opt‑in or filterable, so operators must explicitly enable collection if they choose to accept the risk.

These changes are consistent with best practice: metrics labels should be reserved for attributes with limited cardinality, while per‑request dynamic fields are better captured in traces, logs, or sampled attributes rather than as labels that drive time‑series cardinality.

Why library authors cannot reliably predict cardinality​

A library-level decision to include or exclude attributes must account for the fact that instrumented code runs in diverse environments. A seemingly innocuous attribute such as User‑Agent may be low cardinality in some deployments and catastrophically high in others (for example, when a public endpoint receives synthetic traffic from many unique client agents or when an attacker intentionally randomizes the header).
Because library maintainers cannot forecast every deployment’s exposure model, conservative defaults — restricting high‑cardinality labels and offering configuration knobs for operators — are a safer approach. The fix in 0.44.0 reflects that conservative posture.

---

Exploitation scenarios and real‑world risk​

Typical exploitation pattern​

• Target: any publicly reachable HTTP endpoint that uses otelhttp.NewHandler (or the affected framework adapters) and has a metrics pipeline enabled.
• Prerequisite: the application does not perform upstream filtering (for example, via CDN, load balancer, WAF, or middleware) that blocks or normalizes unusual HTTP methods or User‑Agent headers.
• Attack: send a large number of HTTP requests with unique header values (e.g., policed User‑Agent strings or purposely malformed method tokens) over a short time or in a sustained stream.
• Objective: cause in‑process metrics storage or the metrics exporter to allocate and retain state for each unique label combination until memory is exhausted or the metrics system trips, producing degraded performance or complete crash.

Multiple advisories confirm that exploitation is practical and requires only network access and basic HTTP tooling, making public endpoints the highest‑risk targets.

Who’s at greatest risk​

• Public-facing APIs and microservices that rely on the default otelhttp wrappers for quick instrumentation and do not perform request normalization.
• Services running with constrained memory (container limits, FaaS environments) where incremental allocations have outsized impact.
• Deployments that export metrics in‑process to a long‑running aggregator or where exporters keep a long‑lived registry of metric label sets.
• Systems where the metrics pipeline is used directly for alerting and autoscaling: metric label explosion can create noisy alerts and trigger automation that amplifies disruption.

Supply‑chain implications​

Because OpenTelemetry Contrib sits in many vendors’ and distributions’ dependency graphs, downstream products can inherit the vulnerability. Security bulletins from enterprise vendors (for example, platform and distribution vendors) frequently reference CVE‑2023‑45142 when confirming the impact of OpenTelemetry in their products. This reinforces the need for software bill‑of‑materials (SBOM) awareness and updating dependencies beyond only the direct application code.

---

Detection and monitoring: how to spot an ongoing attack​

Early detection helps avoid catastrophic failure. The following indicators of compromise (IoCs) and telemetry signals are good starting points:

• Sudden rise in distinct label cardinality for HTTP‑related metrics: a growing count of unique dimension values for a given metric is a direct sign.
• Memory growth correlated with metrics ingestion or exporter activity: if memory increases tightly in step with request traffic that sets new header values, metrics labels are a plausible cause.
• Unusual HTTP methods observed in server access logs: strings that are extremely long, contain randomized tokens, or are otherwise nonstandard should be treated as suspicious.
• Spike in metric creation events (if your metrics exporter records when new time series are registered).
• Errors or crashes in the metrics exporter or SDK, such as out‑of‑memory termination (OOM) in a containerized process.

Operators should instrument their observability pipeline to record the number of active time series and watch for anomalous increases. In environments with multi‑tenant concerns, add quotas or rate limits at the metrics ingestion layer to avoid one service exhausting shared resources.

---

Remediation and mitigation guidance​

Immediate steps (fast mitigations)​

• If you cannot immediately upgrade to a fixed release, disable or scope metrics collection for the affected instrumentation:
• Configure the meter provider to a no‑op or to a provider that filters labels.
• Use otelhttp.WithFilter() to prevent recording metrics for suspicious requests, or to exclude headers from being turned into metrics labels. Note that filtering requires care: dropping metrics wholesale may remove useful visibility.
• Implement upstream normalization and filtering:
• Use a CDN, WAF, or load balancer to normalize or reject requests with malformed or nonstandard HTTP methods and to limit excessive variability in User‑Agent headers.
• Apply rate limits for public endpoints to increase the cost of large‑scale exploitation.
• Enforce memory and process limits:
• For containerized workloads, set realistic memory limits and ensure robust restart policies. While limits do not prevent the attack, they provide predictable failure modes and avoid noisy neighbors in multi‑tenant nodes.
• Add metric cardinality guards:
• If your metrics backend supports it, impose per‑metric or per‑label cardinality limits and reject or collapse new label values when thresholds are exceeded.

Upgrading to fixed releases​

• Upgrade all affected go.opentelemetry.io/contrib modules to 0.44.0 or later, ensuring that the transitive dependencies used by your application carry the fixed code path. Because Go modules may be vendored or implicit in dependency graphs, confirm the exact module versions in your build (for example, by reviewing go.sum, go.mod, or the SBOM produced for your build).

Configuring long‑term safe defaults​

• Make high‑cardinality attributes opt‑in and document the tradeoffs. If you truly need User‑Agent analytics at high fidelity, consider collecting a sampled subset and send the raw values only to a log stream rather than to time‑series metrics.
• Use coarse buckets for attributes derived from free‑form text (for example, hash or map User‑Agent strings into small groups like “browser”, “bot”, “client‑X”).
• Prefer traces for per‑request context; traces are designed to capture unique values at scale (but still require sampling to control volume).

---

Detection, incident response, and post‑mortem recommendations​

• When a suspected attack occurs, capture a forensic snapshot: metrics cardinality, exporter logs, process memory state, and recent HTTP access logs. These artifacts help determine whether the cause is instrumentation‑driven cardinality or another issue.
• If memory exhaustion is confirmed, apply fast mitigations (filtering, disabling metrics, or updating the meter provider) before restarting services. Rapid restarts without addressing the root cause risk immediate re‑exhaustion.
• After recovery, perform a root cause analysis that identifies where the vulnerable instrumentation was introduced (direct dependency vs. transitive) and update build processes to enforce safe versions.
• Expand SBOM and dependency scanning to track OpenTelemetry contrib versions across the estate and integrate CVE‑based alerts into patching workflows. Enterprise vendors’ advisories often identify affected products when they embed the vulnerable libraries — use those signals to prioritize patches in downstream products.

---

Risk assessment and broader implications​

Why this class of vulnerability matters beyond a single CVE​

Unbounded metric cardinality is conceptually similar to many resource‑exhaustion attacks: the attacker does not exploit memory safety bugs or code injection; instead, they abuse telemetry or state‑tracking features that maintain ever‑growing registries. These issues strike at a deeper tension:

• Observability wants to be comprehensive; production systems demand boundedness and predictability.
• Libraries that default to “collect everything” simplify developer experience but shift critical security decisions from operators to maintainers who cannot predict every deployment’s exposure model.
• The easiest time to exploit these conditions is in the gap between adoption (where quick, default instrumentation is attractive) and maturity (where operators harden, filter, and tune labels).

This CVE should be a reminder that safety‑first defaults, or at least prominent opt‑outs, are required in observability libraries.

Supply chain and enterprise scale​

Because OpenTelemetry Contrib is reused across many projects and often embedded by upstream vendors, the vulnerability has wide ripples. Enterprise customers should:

• Treat OpenTelemetry contrib versions as first‑class patchable artifacts in their SBOM and vulnerability management systems.
• Coordinate with platform vendors for vendor‑supplied updates or mitigations, as vendors may patch at the product level on a different cadence than upstream libraries. Product advisories sometimes list specific affected product versions that embed vulnerable modules.

---

Practical checklist for engineering teams​

• Inventory all Go services for go.opentelemetry.io/contrib usage (including transitive dependencies) and record exact module versions.
• Prioritize any services that are internet‑facing or handle untrusted input for immediate mitigation and upgrade.
• Upgrade affected modules to 0.44.0 or later in a controlled rollout; test metrics and alerting behavior because label changes can alter telemetry cardinality and historical dashboards.
• If immediate upgrade is not possible, apply otelhttp.WithFilter() to drop suspicious traffic or switch to a metrics provider that enforces safe label cardinality.
• Add automated tests that fail builds if an unsafe OpenTelemetry Contrib version is introduced (CI gating using module checks or SBOM scans).
• Adopt monitoring for total time‑series counts and per‑metric cardinality, and set alerts for unexpected growth.

---

Critical evaluation: strengths, shortcomings, and recommended policy changes​

Notable strengths in the response​

• The maintainers’ remediation — bounding method values and removing high‑cardinality defaults — aligns with long‑standing observability best practices. This preserves useful defaults for typical deployments while protecting operators from a trivial attack vector.
• Public vulnerability databases and vendor advisories provided rapid, coordinated signals that allowed downstream vendors to issue product advisories and mitigations. The availability of multiple independent writeups (CVE registries, Snyk, Wiz, IBM) increased situational awareness.

Remaining risks and shortcomings​

• Backwards compatibility and telemetry semantics: changing labels can silently alter dashboards and alerts. Operators who upgrade without reviewing dashboards may get false positives or lose critical signals. Any upgrade must be accompanied by observability validation.
• Transitive dependencies and vendor lag: many organizations consume OpenTelemetry through higher‑level frameworks or vendor products; those supply chains may delay updates. A robust SBOM and dependency management practice is required to close the window of exposure.
• Configuration complexity: the available mitigations (filters, alternate meter providers) work, but they push complexity onto operators. Libraries should offer safer defaults and clearer guidance for common deployment models.

Policy recommendations for maintainers and vendors​

• Default to safe telemetry: instrumentations should enable only low‑cardinality labels by default and require explicit opt‑in for high‑cardinality attributes.
• Provide a visible compatibility policy: maintainers should publish clear migration notes for label changes and offer guidance for migrating dashboards and alerts.
• Promote runtime safeguards: offer built‑in label cardinality limits or simple configuration flags that let operators set hard caps without deep SDK changes.

---

Conclusion​

CVE‑2023‑45142 is an instructive example of how observability features — intended to help operators — can become attack surfaces when defaults are permissive. The flaw in OpenTelemetry‑Go Contrib’s otelhttp and related adapters allowed unbounded label cardinality for HTTP method and User‑Agent attributes, enabling a straightforward memory‑exhaustion denial‑of‑service if left unpatched. The maintainers addressed the issue by restricting recorded method values and removing or gating high‑cardinality attributes; operators should upgrade to version 0.44.0 or later, or apply careful mitigations such as filtering, upstream normalization, and cardinality limits until an upgrade is feasible.
Beyond the immediate patch, organizations must treat observability components as security‑sensitive parts of their stack: enforce SBOM tracking for contrib modules, alert on unusual metric cardinality growth, and prefer safe defaults in telemetry libraries. Doing so turns observability back into a tool for resilience rather than a liability that can be weaponized to strip an organization’s ability to see or serve its users.

---

Source: MSRC Security Update Guide - Microsoft Security Response Center