Compare commits

...
4 Commits
Author SHA1 Message Date
releaser 92db06698c chore(release): prepare for v0.6.1 (#185)
otelsetup / test (push) Skipped
otelsetup / vulnerabilities (push) Skipped
Unbound Release / Check Preconditions (push) Successful in 25s
Unbound Release / Create Tag (push) Skipped
Unbound Release / Generate Changelog and Handle PR (push) Successful in 26s
Unbound Release / Create Release (push) Successful in 28s
Release / release (push) Successful in 1m13s
pre-commit / pre-commit (push) Successful in 3m24s
## [0.6.1] - 2026-09-19

### 🐛 Bug Fixes

- Close idle OTLP connections before the collector does (#184)

<!-- generated by git-cliff -->

---

**Note:** Please use **Squash Merge** when merging this PR.

Reviewed-on: https://gitea.unbound.se/shiny/otelsetup/pulls/185
Co-authored-by: Unbound Releaser <releaser@unbound.se>
2026-09-19 13:58:49 +00:00
argoyle 8b3d3110fe fix: close idle OTLP connections before the collector does (#184)
Unbound Release / Check Preconditions (push) Successful in 27s
Unbound Release / Create Tag (push) Skipped
otelsetup / test (push) Skipped
otelsetup / vulnerabilities (push) Skipped
Unbound Release / Generate Changelog and Handle PR (push) Successful in 39s
Unbound Release / Create Release (push) Successful in 29s
pre-commit / pre-commit (push) Successful in 3m27s
Release / release (push) Successful in 1m14s
2026-09-19 13:51:36 +00:00
releaser 80d749ab92 chore(release): prepare for v0.6.0 (#183)
Unbound Release / Check Preconditions (push) Successful in 22s
Unbound Release / Create Tag (push) Skipped
Unbound Release / Generate Changelog and Handle PR (push) Successful in 24s
otelsetup / test (push) Skipped
otelsetup / vulnerabilities (push) Skipped
Unbound Release / Create Release (push) Successful in 24s
pre-commit / pre-commit (push) Successful in 2m53s
Release / release (push) Successful in 1m10s
## [0.6.0] - 2026-09-17

### 🚀 Features

- *(metrics)* Record eventsourced pg outbox publishes and retries (#182)

<!-- generated by git-cliff -->

---

**Note:** Please use **Squash Merge** when merging this PR.

Reviewed-on: https://gitea.unbound.se/shiny/otelsetup/pulls/183
Co-authored-by: Unbound Releaser <releaser@unbound.se>
2026-09-17 12:01:33 +00:00
argoyle 88dde56687 feat(metrics): record eventsourced pg outbox publishes and retries (#182)
Unbound Release / Check Preconditions (push) Successful in 22s
Unbound Release / Create Tag (push) Skipped
Unbound Release / Generate Changelog and Handle PR (push) Successful in 37s
pre-commit / pre-commit (push) Successful in 2m45s
Unbound Release / Create Release (push) Successful in 23s
otelsetup / test (push) Successful in 58s
otelsetup / vulnerabilities (push) Successful in 53s
Release / release (push) Successful in 1m2s
## Why
authz-service now publishes through pg/v2's transactional outbox, and nothing measured it: `NewEventsourcedMetrics` ignored the `pg.Outbox*` metrics. Without them a failing or abandoned publish (a lost privilege revocation for running authz_clients) can't be alerted on.

## What
- `pg.OutboxPublish` → histogram `eventsourced.outbox.publish.duration` (`event.type`, `success`)
- `pg.OutboxRetry` → counter `eventsourced.outbox.retries` (`event.type`, `permanent`)
- `pg.OutboxBatch` / `pg.OutboxCleanup` stay ignored.

otelsetup now imports `codeberg.org/eventsourced/pg/v2` v2.1.1 (`go` directive becomes 1.26.0; grpc patch bump via MVS). All current consumers already depend on pg/v2; a release raises their minimum to v2.1.1.

## Tests
`go test -race ./...` green; the contract test records both new metrics.

## Review
Go Backend expert reviewed: no Critical/High. Noted: the test checks instrument names only, not attributes; the pg/v2 dependency is worth a line in the release notes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_013DJouG8ZZZxKtodF9kDvzj
Reviewed-on: https://gitea.unbound.se/shiny/otelsetup/pulls/182
2026-09-17 11:54:05 +00:00
9 changed files with 149 additions and 13 deletions

No files matched your search

+1 -1
View File
@@ -1,3 +1,3 @@
{
"version": "v0.5.0"
"version": "v0.6.1"
}
+13 -1
View File
@@ -2,7 +2,19 @@
All notable changes to this project will be documented in this file.
## [0.5.0] - 2026-09-11
## [0.6.1] - 2026-09-19
### 🐛 Bug Fixes
- Close idle OTLP connections before the collector does (#184)
## [0.6.0] - 2026-09-17
### 🚀 Features
- *(metrics)* Record eventsourced pg outbox publishes and retries (#182)
## [0.5.0] - 2026-09-14
### 🚀 Features
+1
View File
@@ -42,3 +42,4 @@ http.Handle("/", otelsetup.Handler(myHandler))
- `OTEL_RESOURCE_ATTRIBUTES` - Auto-set if not provided (service.name, service.version, service.environment)
- Standard OTLP environment variables for endpoint configuration
- The trace and metric exporters use our own HTTP client (idle connections closed after 30s, before the collector's 1m idle timeout), so `OTEL_EXPORTER_OTLP_*TIMEOUT` and the `OTEL_EXPORTER_OTLP_*CERTIFICATE` / `*CLIENT_*` TLS variables are ignored
+10 -3
View File
@@ -1,9 +1,10 @@
module gitea.unbound.se/shiny/otelsetup
go 1.26
go 1.26.0
require (
codeberg.org/eventsourced/eventsourced v1.24.0
codeberg.org/eventsourced/pg/v2 v2.1.1
github.com/99designs/gqlgen v0.17.95
go.opentelemetry.io/otel v1.46.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.46.0
@@ -24,17 +25,23 @@ require (
github.com/go-logr/stdr v1.2.2 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0 // indirect
github.com/jmoiron/sqlx v1.4.0 // indirect
github.com/lib/pq v1.12.3 // indirect
github.com/mfridman/interpolate v0.0.2 // indirect
github.com/pressly/goose/v3 v3.28.0 // indirect
github.com/sethvargo/go-retry v0.4.0 // indirect
github.com/sosodev/duration v1.4.0 // indirect
github.com/vektah/gqlparser/v2 v2.5.37 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.46.0 // indirect
go.opentelemetry.io/proto/otlp v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/net v0.58.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.41.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260819154853-08b0e4226688 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260819154853-08b0e4226688 // indirect
google.golang.org/grpc v1.83.1 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260831171406-18b4a7587f8a // indirect
google.golang.org/grpc v1.83.2 // indirect
google.golang.org/protobuf v1.36.12 // indirect
)
+47 -4
View File
@@ -1,18 +1,30 @@
codeberg.org/eventsourced/eventsourced v1.24.0 h1:yNB+74JNB/NHo7D8ZR/yQcTVDYrWnjFf0xUKLgViqRc=
codeberg.org/eventsourced/eventsourced v1.24.0/go.mod h1:uy0HxejDdNFpI+Yp7VOEvjfEUFJQX1VkCZ4e4JyDp7Q=
codeberg.org/eventsourced/pg/v2 v2.1.1 h1:4OHmMU7v93mXUY0ldfz18lkqDC7UGCtDee4wMLIi43I=
codeberg.org/eventsourced/pg/v2 v2.1.1/go.mod h1:y/3Rul9NTcq7ScL5Z44cdDQrrOk6drrudGLoUYjrIns=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/99designs/gqlgen v0.17.95 h1:882h7F5iJImgtyUVttc4MOK2NbzbMYc2oyNeHqkjpP4=
github.com/99designs/gqlgen v0.17.95/go.mod h1:kHYPrpwOXDU1OQyxIg3Z7nVXSnlUoHVWBY7CMJCAM4M=
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/agnivade/levenshtein v1.2.1 h1:EHBY3UOn1gwdy/VbFwgo4cxecRznFk7fKWN1KOX7eoM=
github.com/agnivade/levenshtein v1.2.1/go.mod h1:QVVI16kDrtSuwcpd0p1+xMC6Z/VfhtCyDIjcwga4/DU=
github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=
github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -21,6 +33,27 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0 h1:/Tnpcb2E0Pz/tN9s3bfEY2Q8ePCEX9iuS+cneUwncnw=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0/go.mod h1:zOBXOsUaBSjKgmH4OGzV1esUpR3oUSCPYVd2cUBjKYY=
github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o=
github.com/jmoiron/sqlx v1.4.0/go.mod h1:ZrZ7UsYB/weZdl2Bxg6jCRO9c3YHl8r3ahlKmRT4JLY=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mfridman/interpolate v0.0.2 h1:pnuTK7MQIxxFz1Gr+rjSIx9u7qVjf5VOoM/u6BbAxPY=
github.com/mfridman/interpolate v0.0.2/go.mod h1:p+7uk6oE07mpE/Ik1b8EckO0O4ZXiGAfshKBWLUM9Xg=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pressly/goose/v3 v3.28.0 h1:D2M+iL31GmpZxSHOhX8mqyqAT3CXnokUmm0eKoSP+Vc=
github.com/pressly/goose/v3 v3.28.0/go.mod h1:v26MOuB8bL3kzzrt3Vqhb3R0PRVsl8hFQKdrht/L6Rk=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/sanity-io/litter v1.5.8 h1:uM/2lKrWdGbRXDrIq08Lh9XtVYoeGtcQxk9rtQ7+rYg=
github.com/sanity-io/litter v1.5.8/go.mod h1:9gzJgR2i4ZpjZHsKvUXIRQVk7P+yM3e+jAF7bU2UI5U=
github.com/sethvargo/go-retry v0.4.0 h1:9qy1OoIAxBL+gBYnkTnTnWle5wlfsXQlwRzIbbpdqPw=
github.com/sethvargo/go-retry v0.4.0/go.mod h1:tvsjdKG6xfiCx4LSiUZ06kcv38xvdVQwv8R6/VnnVWg=
github.com/sosodev/duration v1.4.0 h1:35ed0KiVFriGHHzZZJaZLgmTEEICIyt8Sx0RQfj9IjE=
github.com/sosodev/duration v1.4.0/go.mod h1:RQIBBX0+fMLc/D9+Jb/fwvVmo0eZvDDEERAikUR6SDg=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
@@ -59,6 +92,8 @@ go.opentelemetry.io/proto/otlp v1.11.0 h1:5rrYs0Ykyj50sdU/JU0x8etU+LubXWb+gED6Tb
go.opentelemetry.io/proto/otlp v1.11.0/go.mod h1:SmVizdCOAm3XBtG1g1NnOdhW6jtddT72hLMhv8VwA8E=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
@@ -73,9 +108,17 @@ gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/genproto/googleapis/api v0.0.0-20260819154853-08b0e4226688 h1:ax2KzoSRIZU/M0cIxri3pKxy99vniH1PVxWC6si/eZI=
google.golang.org/genproto/googleapis/api v0.0.0-20260819154853-08b0e4226688/go.mod h1:1RJ9BQGyNdZwkGc1eTqkErfRZ6RJyYPHZo73BZ1vQqI=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260819154853-08b0e4226688 h1:cYNAzI2sUwhmCcoj9TxvihSrqsxt6uIkj3rDRhSDmW4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260819154853-08b0e4226688/go.mod h1:DjtHYE8FKJLivXcBEjGwndXfIC23G0VpXiXKqG179uA=
google.golang.org/grpc v1.83.1 h1:HIO0+BEtBP6soyqvqC8sNUjZ7bTs+0hFQuFF+RAy++Y=
google.golang.org/grpc v1.83.1/go.mod h1:kDyl6SKsiHKt0uylY5gtn5cEjkrIOhQOGDgIc4JGwzQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260831171406-18b4a7587f8a h1:3Dnd1cDaZlB68lziofO+bJXpjOy8UfRv8Unt+yH8tQ4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260831171406-18b4a7587f8a/go.mod h1:DjtHYE8FKJLivXcBEjGwndXfIC23G0VpXiXKqG179uA=
google.golang.org/grpc v1.83.2 h1:EManeRomTObA0BU7I8vXgg/78uE5MJ9M8B39EX2WscU=
google.golang.org/grpc v1.83.2/go.mod h1:YPI1hK3kDked6iHvgX3tR0y+nX/qpMFKhPgFsokw1S8=
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
modernc.org/libc v1.75.6 h1:yKk8qo+Di4gkmvRboK8ocCqH22FiUCR6jRy2OwtCRus=
modernc.org/libc v1.75.6/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/sqlite v1.57.0 h1:qNQP6xnx5M0ISNtlnxoOX0+cD5bJ0/gr9aMmndFczzg=
modernc.org/sqlite v1.57.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
+17 -2
View File
@@ -5,6 +5,7 @@ import (
"errors"
"codeberg.org/eventsourced/eventsourced"
"codeberg.org/eventsourced/pg/v2"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/metric"
@@ -40,6 +41,8 @@ type eventsourcedMetrics struct {
snapshotStoreDur metric.Float64Histogram
snapshotLoadDur metric.Float64Histogram
idempotencyCheck metric.Int64Counter
outboxPublishDur metric.Float64Histogram
outboxRetries metric.Int64Counter
}
// NewEventsourcedMetrics builds an eventsourced.MetricsRecorder that records to
@@ -78,6 +81,8 @@ func NewEventsourcedMetrics() (eventsourced.MetricsRecorder, error) {
snapshotStoreDur: hist("eventsourced.snapshot.store.duration", "Time taken to persist a snapshot."),
snapshotLoadDur: hist("eventsourced.snapshot.load.duration", "Time taken to load a snapshot."),
idempotencyCheck: counter("eventsourced.idempotency.checks", "Number of command idempotency lookups."),
outboxPublishDur: hist("eventsourced.outbox.publish.duration", "Time taken to publish an event from the outbox."),
outboxRetries: counter("eventsourced.outbox.retries", "Failed outbox publishes; permanent ones are never retried."),
}
if err := errors.Join(errs...); err != nil {
return nil, err
@@ -86,8 +91,8 @@ func NewEventsourcedMetrics() (eventsourced.MetricsRecorder, error) {
}
// Record implements eventsourced.MetricsRecorder. Metric types the adapter does
// not recognise (for example pg outbox metrics when the outbox is not enabled)
// are ignored.
// not recognise are ignored, and so are pg.OutboxBatch and pg.OutboxCleanup: a failed
// publish already shows in the publish and retry instruments.
func (e *eventsourcedMetrics) Record(ctx context.Context, raw eventsourced.Metric) {
switch m := raw.(type) {
case eventsourced.CommandDuration:
@@ -119,5 +124,15 @@ func (e *eventsourcedMetrics) Record(ctx context.Context, raw eventsourced.Metri
attribute.String("aggregate.type", m.AggregateType),
attribute.Bool("hit", m.Hit),
))
case pg.OutboxPublish:
e.outboxPublishDur.Record(ctx, m.Duration.Seconds(), metric.WithAttributes(
attribute.String("event.type", m.EventType),
attribute.Bool("success", m.Success),
))
case pg.OutboxRetry:
e.outboxRetries.Add(ctx, 1, metric.WithAttributes(
attribute.String("event.type", m.EventType),
attribute.Bool("permanent", m.Permanent),
))
}
}
+5
View File
@@ -7,6 +7,7 @@ import (
"time"
"codeberg.org/eventsourced/eventsourced"
"codeberg.org/eventsourced/pg/v2"
"go.opentelemetry.io/otel"
sdkmetric "go.opentelemetry.io/otel/sdk/metric"
"go.opentelemetry.io/otel/sdk/metric/metricdata"
@@ -33,6 +34,8 @@ func TestNewEventsourcedMetrics_RecordsContract(t *testing.T) {
eventsourced.SnapshotStored{AggregateType: "Entry", Duration: time.Millisecond, Success: true},
eventsourced.SnapshotLoaded{AggregateType: "Entry", Found: false, Duration: time.Millisecond},
eventsourced.IdempotencyCheck{AggregateType: "Entry", Hit: true},
pg.OutboxPublish{EventType: "EntryAdded", Success: false, Duration: time.Millisecond},
pg.OutboxRetry{EventType: "EntryAdded", RetryCount: 1},
unknownMetric{},
} {
r.Record(context.Background(), m)
@@ -57,6 +60,8 @@ func TestNewEventsourcedMetrics_RecordsContract(t *testing.T) {
"eventsourced.snapshot.store.duration",
"eventsourced.snapshot.load.duration",
"eventsourced.idempotency.checks",
"eventsourced.outbox.publish.duration",
"eventsourced.outbox.retries",
}
var missing []string
for _, w := range want {
+33 -2
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"net"
"net/http"
"os"
"time"
@@ -65,7 +66,7 @@ func SetupOTelSDK(ctx context.Context, enabled bool, serviceName, buildVersion,
)
otel.SetTextMapPropagator(prop)
traceExporter, err := otlptracehttp.New(ctx)
traceExporter, err := otlptracehttp.New(ctx, otlptracehttp.WithHTTPClient(otlpHTTPClient()))
if err != nil {
return handleErr(err)
}
@@ -88,7 +89,7 @@ func SetupOTelSDK(ctx context.Context, enabled bool, serviceName, buildVersion,
global.SetLoggerProvider(logProvider)
shutdownFuncs = append(shutdownFuncs, logProvider.Shutdown)
exp, err := otlpmetrichttp.New(ctx)
exp, err := otlpmetrichttp.New(ctx, otlpmetrichttp.WithHTTPClient(otlpHTTPClient()))
if err != nil {
return handleErr(err)
}
@@ -99,6 +100,36 @@ func SetupOTelSDK(ctx context.Context, enabled bool, serviceName, buildVersion,
return shutdown, err
}
// otlpHTTPClient is the OTLP exporters' default client, except that it drops idle
// connections after 30s. The collector (Alloy's OTLP receiver, idle_timeout 1m) closes
// connections idle for a minute, and metrics are pushed every 60s. With the exporters'
// default of 90s, a push could reuse a connection the collector was closing. The push
// then failed (EOF, connection reset) and was dropped: net/http doesn't retry a POST and
// the exporters don't retry transport errors. Traces hit the same race after a quiet
// spell. Closing first means such a push dials fresh.
//
// The transport mirrors the exporters' own (otlpmetrichttp/otlptracehttp v1.46.0
// ourTransport). A custom client makes the exporters ignore OTEL_EXPORTER_OTLP_*TIMEOUT
// and the OTEL_EXPORTER_OTLP_*CERTIFICATE/CLIENT_* TLS variables; nothing sets them.
func otlpHTTPClient() *http.Client {
return &http.Client{
Transport: &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,
IdleConnTimeout: 30 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
},
// The exporters' default.
Timeout: 10 * time.Second,
}
}
func Handler(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := otel.GetTextMapPropagator().Extract(r.Context(), propagation.HeaderCarrier(r.Header))
+22
View File
@@ -0,0 +1,22 @@
package otelsetup
import (
"net/http"
"testing"
"time"
)
func TestOTLPHTTPClient_ClosesIdleConnectionsBeforeTheCollector(t *testing.T) {
c := otlpHTTPClient()
tr, ok := c.Transport.(*http.Transport)
if !ok {
t.Fatalf("transport is %T, want *http.Transport", c.Transport)
}
// Alloy's OTLP receiver closes connections idle for 1m; the client must close first.
if tr.IdleConnTimeout <= 0 || tr.IdleConnTimeout >= time.Minute {
t.Errorf("IdleConnTimeout = %v, want in (0, 1m)", tr.IdleConnTimeout)
}
if c.Timeout != 10*time.Second {
t.Errorf("Timeout = %v, want 10s (the exporters' default)", c.Timeout)
}
}