fix: close idle OTLP connections before the collector does
otelsetup / test (push) Skipped
otelsetup / vulnerabilities (push) Skipped
pre-commit / pre-commit (push) Skipped
otelsetup / vulnerabilities (pull_request) Successful in 1m0s
otelsetup / test (pull_request) Successful in 1m21s
pre-commit / pre-commit (pull_request) Successful in 3m36s
otelsetup / test (push) Skipped
otelsetup / vulnerabilities (push) Skipped
pre-commit / pre-commit (push) Skipped
otelsetup / vulnerabilities (pull_request) Successful in 1m0s
otelsetup / test (pull_request) Successful in 1m21s
pre-commit / pre-commit (pull_request) Successful in 3m36s
Metrics are pushed every 60s and Alloy's OTLP receiver closes connections idle for 1m, while the exporters keep them for 90s. A push could reuse a connection the collector was closing and fail with EOF or connection reset; the data was dropped (no retry for a POST or for transport errors). In prod that was ~3 failed metric pushes per pod per hour, plus occasional trace batches. Both exporters now use an HTTP client that closes idle connections after 30s. A custom client makes the exporters ignore the OTLP timeout and certificate env vars; none are set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rfQ5HJ7zwfuYQcWXd3Mm3
This commit is contained in:
3 files changed
+56
-2
No files matched your search
@@ -42,3 +42,4 @@ http.Handle("/", otelsetup.Handler(myHandler))
|
||||
|
||||
- `OTEL_RESOURCE_ATTRIBUTES` - Auto-set if not provided (service.name, service.version, service.environment)
|
||||
- Standard OTLP environment variables for endpoint configuration
|
||||
- The trace and metric exporters use our own HTTP client (idle connections closed after 30s, before the collector's 1m idle timeout), so `OTEL_EXPORTER_OTLP_*TIMEOUT` and the `OTEL_EXPORTER_OTLP_*CERTIFICATE` / `*CLIENT_*` TLS variables are ignored
|
||||
Reference in new issue
Block a user