Issue migrating from v1.42.1 to >=1.44.0 #7185

psagithub · 2025-03-07T03:48:43Z

psagithub
Mar 7, 2025

I'm getting following error while trying to migrate OTel in my spring boot application from OTel version 1.42.1 (no issues publishing the metrics) to 1.44.1. But, getting the following error.

Mar 07, 2025 3:27:11 AM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
SEVERE: Failed to export spans. The request could not be executed. Error message: expected 4136 bytes but received 4153
java.net.ProtocolException: expected 4136 bytes but received 4153
	at okhttp3.internal.connection.Exchange$RequestBodySink.write(Exchange.kt:219)
	at okio.RealBufferedSink.close(RealBufferedSink.kt:280)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:63)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at io.opentelemetry.exporter.sender.okhttp.internal.RetryInterceptor.intercept(RetryInterceptor.java:91)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Lib Versions:

Java => 21
Helm chart DD version: 3.102.0
DD Agent on AWS (configured through helm chart): v7.63.2
org.springframework.boot => 3.4.3 (webflux project)

mavenBom("io.opentelemetry:opentelemetry-bom:1.44.1")
mavenBom("io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom:2.10.0")

// Prometheus/metrics
implementation "org.springframework.boot:spring-boot-starter-actuator:3.4.2"
implementation "io.micrometer:micrometer-registry-prometheus"
implementation 'io.micrometer:micrometer-registry-datadog'
implementation 'io.micrometer:micrometer-registry-statsd:1.14.3'

// opentelemetry
implementation("io.opentelemetry.instrumentation:opentelemetry-spring-boot-starter")
implementation("io.opentelemetry.contrib:opentelemetry-samplers:1.41.0-alpha")
implementation("io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java17:2.10.0-alpha")

OTel config:

otel.traces.exporter=otlp
otel.metrics.exporter=otlp
otel.metric.export.interval=20000
otel.logs.exporter=none
otel.exporter.otlp.protocol=grpc (I even tried to use http/protobuf, but getting the same issue)
otel.exporter.otlp.endpoint=${OTEL_EXPORTER_OTLP_ENDPOINT:http://localhost:4317}
otel.instrumentation.logback-appender.enabled=false

psagithub · 2025-03-07T03:55:01Z

psagithub
Mar 7, 2025
Author

I'm also getting the ArrayIndexOutOfBoundsException error intermittently.

Mar 07, 2025 3:35:01 AM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
SEVERE: Failed to export spans. The request could not be executed. Error message: canceled due to java.lang.ArrayIndexOutOfBoundsException: Index 128 out of bounds for length 128
java.io.IOException: canceled due to java.lang.ArrayIndexOutOfBoundsException: Index 128 out of bounds for length 128
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:530)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.ArrayIndexOutOfBoundsException: Index 128 out of bounds for length 128
		at io.opentelemetry.exporter.internal.marshal.MarshalerContext.getSize(MarshalerContext.java:79)
		at io.opentelemetry.exporter.internal.marshal.Serializer.serializeMessageWithContext(Serializer.java:293)
		at io.opentelemetry.exporter.internal.otlp.traces.SpanStatelessMarshaler.writeTo(SpanStatelessMarshaler.java:63)
		at io.opentelemetry.exporter.internal.otlp.traces.SpanStatelessMarshaler.writeTo(SpanStatelessMarshaler.java:22)
		at io.opentelemetry.exporter.internal.marshal.ProtoSerializer.serializeRepeatedMessageWithContext(ProtoSerializer.java:239)
		at io.opentelemetry.exporter.internal.otlp.traces.InstrumentationScopeSpansStatelessMarshaler.writeTo(InstrumentationScopeSpansStatelessMarshaler.java:39)
		at io.opentelemetry.exporter.internal.otlp.traces.InstrumentationScopeSpansStatelessMarshaler.writeTo(InstrumentationScopeSpansStatelessMarshaler.java:21)
		at io.opentelemetry.exporter.internal.marshal.Serializer$RepeatedElementPairWriter.accept(Serializer.java:606)
		at java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1375)
		at io.opentelemetry.exporter.internal.marshal.Serializer.serializeRepeatedMessageWithContext(Serializer.java:513)
		at io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansStatelessMarshaler.writeTo(ResourceSpansStatelessMarshaler.java:46)
		at io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansStatelessMarshaler.writeTo(ResourceSpansStatelessMarshaler.java:28)
		at io.opentelemetry.exporter.internal.marshal.Serializer$RepeatedElementPairWriter.accept(Serializer.java:606)
		at java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1375)
		at io.opentelemetry.exporter.internal.marshal.Serializer.serializeRepeatedMessageWithContext(Serializer.java:513)
		at io.opentelemetry.exporter.internal.otlp.traces.LowAllocationTraceRequestMarshaler.writeTo(LowAllocationTraceRequestMarshaler.java:73)
		at io.opentelemetry.exporter.internal.marshal.Marshaler.writeBinaryTo(Marshaler.java:23)
		at io.opentelemetry.exporter.sender.okhttp.internal.GrpcRequestBody.writeTo(GrpcRequestBody.java:68)
		at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:62)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
		at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
		at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
		at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
		at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
		at io.opentelemetry.exporter.sender.okhttp.internal.RetryInterceptor.intercept(RetryInterceptor.java:91)
		at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                 at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
		at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
		... 3 more

Exception in thread "okhttp-dispatch-3" java.lang.ArrayIndexOutOfBoundsException: Index 128 out of bounds for length 128
	at io.opentelemetry.exporter.internal.marshal.MarshalerContext.getSize(MarshalerContext.java:79)
	at io.opentelemetry.exporter.internal.marshal.Serializer.serializeMessageWithContext(Serializer.java:293)
	at io.opentelemetry.exporter.internal.otlp.traces.SpanStatelessMarshaler.writeTo(SpanStatelessMarshaler.java:63)
	at io.opentelemetry.exporter.internal.otlp.traces.SpanStatelessMarshaler.writeTo(SpanStatelessMarshaler.java:22)
	at io.opentelemetry.exporter.internal.marshal.ProtoSerializer.serializeRepeatedMessageWithContext(ProtoSerializer.java:239)
	at io.opentelemetry.exporter.internal.otlp.traces.InstrumentationScopeSpansStatelessMarshaler.writeTo(InstrumentationScopeSpansStatelessMarshaler.java:39)
	at io.opentelemetry.exporter.internal.otlp.traces.InstrumentationScopeSpansStatelessMarshaler.writeTo(InstrumentationScopeSpansStatelessMarshaler.java:21)
	at io.opentelemetry.exporter.internal.marshal.Serializer$RepeatedElementPairWriter.accept(Serializer.java:606)
	at java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1375)
	at io.opentelemetry.exporter.internal.marshal.Serializer.serializeRepeatedMessageWithContext(Serializer.java:513)
	at io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansStatelessMarshaler.writeTo(ResourceSpansStatelessMarshaler.java:46)
	at io.opentelemetry.exporter.internal.otlp.traces.ResourceSpansStatelessMarshaler.writeTo(ResourceSpansStatelessMarshaler.java:28)
	at io.opentelemetry.exporter.internal.marshal.Serializer$RepeatedElementPairWriter.accept(Serializer.java:606)
	at java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1375)
	at io.opentelemetry.exporter.internal.marshal.Serializer.serializeRepeatedMessageWithContext(Serializer.java:513)
	at io.opentelemetry.exporter.internal.otlp.traces.LowAllocationTraceRequestMarshaler.writeTo(LowAllocationTraceRequestMarshaler.java:73)
	at io.opentelemetry.exporter.internal.marshal.Marshaler.writeBinaryTo(Marshaler.java:23)
	at io.opentelemetry.exporter.sender.okhttp.internal.GrpcRequestBody.writeTo(GrpcRequestBody.java:68)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:62)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at io.opentelemetry.exporter.sender.okhttp.internal.RetryInterceptor.intercept(RetryInterceptor.java:91)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

2 replies

jack-berg Mar 7, 2025
Maintainer

Hi @psagithub thanks for the report.

Couple of questions:

Have you tried upgrading to 1.47.0? Is possible that this is fixed in a later version.
Does this error happen reliably or intermittently?
Wondering if you can share some sort of reproduction of the issue. For example, if you could set otel.traces.exporter=otlp,logging-otlp, then we could see a JSON encoding of the data in the logs that corresponds to the failed serialization used to export over OTLP.

In the short term, you should be able to get around this error by setting otel.java.exporter.memory_mode=immutable_data (docs here). But we definitely want to identify the root cause of this and fix it. 🙂

jack-berg Mar 7, 2025
Maintainer

Have you tried upgrading to 1.47.0? Is possible that this is fixed in a later version.

See this similar issue which was fixed in 1.47.0: #7019

psagithub · 2025-03-07T17:05:44Z

psagithub
Mar 7, 2025
Author

@jack-berg Thanks for taking a look.

Have you tried upgrading to 1.47.0? Is possible that this is fixed in a later version.

Yes, that was my first choice. But got the same issue. So, wanted to try one version at a time. v1.43.0 worked fine. The problem started with 1.44.0. I'll now take everything latest and add logs.

mavenBom("io.opentelemetry:opentelemetry-bom:1.47.0")
mavenBom("io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom:2.13.3")
implementation("io.opentelemetry.contrib:opentelemetry-samplers:1.44.0-alpha")
implementation("io.opentelemetry.instrumentation:opentelemetry-runtime-telemetry-java17:2.13.3-alpha")

Does this error happen reliably or intermittently?

Reliably/consistently.

you should be able to get around this error by setting otel.java.exporter.memory_mode=immutable_data...

I tried that as well. But that didn't help.

Wondering if you can share some sort of reproduction of the issue....

Got to change some of the values in the log for privacy reasons.

{"ts":"2025-03-07T16:36:01.314Z","level":"INFO","comp":"com.optum.ce.cspa.sample.CspasampleApplication","tin":1,"tn":"main","msg":"Starting CspasampleApplication using Java 21.0.6 with PID 1 (/app/classes started by root in /)"}
{"ts":"2025-03-07T16:36:01.315Z","level":"INFO","comp":"com.optum.ce.cspa.sample.CspasampleApplication","tin":1,"tn":"main","msg":"The following 1 profile is active: \"stage\""}
{"ts":"2025-03-07T16:36:01.373Z","level":"INFO","comp":"io.awspring.cloud.secretsmanager.AwsSecretsManagerPropertySources","tin":1,"tn":"main","msg":"Loading secrets from AWS Secret Manager secret with name: stage/cspa/datadog/api/key, optional: true"}
{"ts":"2025-03-07T16:36:01.373Z","level":"INFO","comp":"io.awspring.cloud.secretsmanager.AwsSecretsManagerPropertySources","tin":1,"tn":"main","msg":"Loading secrets from AWS Secret Manager secret with name: staging/cspa/sample, optional: true"}
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Mar 07, 2025 4:36:03 PM io.micrometer.core.instrument.push.PushMeterRegistry start
INFO: publishing metrics for DatadogMeterRegistry every 1m
{"ts":"2025-03-07T16:36:04.464Z","level":"INFO","comp":"org.flywaydb.core.internal.license.VersionPrinter","tin":1,"tn":"main","msg":"Flyway Community Edition 9.16.0 by Redgate"}
{"ts":"2025-03-07T16:36:04.466Z","level":"INFO","comp":"org.flywaydb.core.internal.license.VersionPrinter","tin":1,"tn":"main","msg":"See release notes here: https://rd.gt/416ObMi"}
{"ts":"2025-03-07T16:36:04.467Z","level":"INFO","comp":"org.flywaydb.core.internal.license.VersionPrinter","tin":1,"tn":"main","msg":""}
{"ts":"2025-03-07T16:36:04.675Z","level":"INFO","comp":"org.flywaydb.core.internal.database.base.BaseDatabaseType","tin":1,"tn":"main","msg":"Database: jdbc:postgresql://postgres-rds-staging.rds.amazonaws.com:5432/sample (PostgreSQL 15.7)"}
{"ts":"2025-03-07T16:36:04.743Z","level":"INFO","comp":"org.flywaydb.core.internal.command.DbValidate","tin":1,"tn":"main","msg":"Successfully validated 6 migrations (execution time 00:00.045s)"}
{"ts":"2025-03-07T16:36:04.805Z","level":"INFO","comp":"org.flywaydb.core.internal.command.DbMigrate","tin":1,"tn":"main","msg":"Current version of schema \"public\": 1.6"}
{"ts":"2025-03-07T16:36:04.806Z","level":"INFO","comp":"org.flywaydb.core.internal.command.DbMigrate","tin":1,"tn":"main","msg":"Schema \"public\" is up to date. No migration necessary."}
{"ts":"2025-03-07T16:36:06.311Z","level":"WARN","comp":"org.springframework.web.reactive.resource.ResourceHandlerUtils","tin":1,"tn":"main","msg":"Appended trailing slash to static resource location: classpath:/META-INF/resources/webjars/swagger-ui/5.18.3/"}
{"ts":"2025-03-07T16:36:06.311Z","level":"WARN","comp":"org.springframework.web.reactive.resource.ResourceHandlerUtils","tin":1,"tn":"main","msg":"Appended trailing slash to static resource location: classpath:/META-INF/resources/webjars/swagger-ui/5.18.3/"}
{"ts":"2025-03-07T16:36:07.186Z","level":"INFO","comp":"com.optum.ce.cspa.sample.CspasampleApplication","tin":1,"tn":"main","msg":"Started CspasampleApplication in 7.843 seconds (process running for 9.186)"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="Metaspace", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'profiled nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger log
WARNING: Too many log messages detected. Will only log once per minute from now on.
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="Metaspace", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'profiled nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger log
WARNING: Too many log messages detected. Will only log once per minute from now on.
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'profiled nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger log
WARNING: Too many log messages detected. Will only log once per minute from now on.
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="Compressed Class Space", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=false, jvm.thread.state="waiting"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=false, jvm.thread.state="timed_waiting"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="waiting"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=false, jvm.thread.state="runnable"}
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger log
WARNING: Too many log messages detected. Will only log once per minute from now on.
Mar 07, 2025 4:36:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="timed_waiting"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:36:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Eden Space", jvm.memory.type="heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger log
WARNING: Too many log messages detected. Will only log once per minute from now on.
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Survivor Space", jvm.memory.type="heap"}
Mar 07, 2025 4:37:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:37:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:37:43 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:37:53 PM io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter write
INFO: {"resource":{"attributes":[{"key":"container.id","value":{"stringValue":"ad80d452f2424a1d27edc42354d2c4f8b2889f92bcd64458b17aa36623abef6e"}},{"key":"deployment.environment","value":{"stringValue":"staging"}},{"key":"git.commit.sha","value":{"stringValue":"ed78275dd19501b8d6b07df4f71789f36245bd37"}},{"key":"git.repository_url","value":{"stringValue":"https://github.com/"}},{"key":"host.arch","value":{"stringValue":"amd64"}},{"key":"host.name","value":{"stringValue":"sample-rollout-b96597558-6wqfg"}},{"key":"os.description","value":{"stringValue":"Linux 5.10.233-224.894.amzn2.x86_64"}},{"key":"os.type","value":{"stringValue":"linux"}},{"key":"process.runtime.description","value":{"stringValue":"Eclipse Adoptium OpenJDK 64-Bit Server VM 21.0.6+7-LTS"}},{"key":"process.runtime.name","value":{"stringValue":"OpenJDK Runtime Environment"}},{"key":"process.runtime.version","value":{"stringValue":"21.0.6+7-LTS"}},{"key":"service.instance.id","value":{"stringValue":"a5c52ed2-2463-47cf-bb80-aa18d740831d"}},{"key":"service.name","value":{"stringValue":"sample"}},{"key":"service.version","value":{"stringValue":"v2025-03-07-ed78275-SNAPSHOT"}},{"key":"telemetry.distro.name","value":{"stringValue":"opentelemetry-spring-boot-starter"}},{"key":"telemetry.distro.version","value":{"stringValue":"2.13.3"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.47.0"}}]},"scopeSpans":[{"scope":{"name":"io.opentelemetry.r2dbc-1.0","version":"2.13.3-alpha","attributes":[]},"spans":[{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"c52fcbd336086c65","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"299c5d36dba90889","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472947418246","endTimeUnixNano":"1741365472957761585","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"44674e8bc5415492","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"70eb7c672c4d4af9","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472948585033","endTimeUnixNano":"1741365472958946604","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"d82c2b47a1d63f0f","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"013c0bc4db6e86b6","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472927393132","endTimeUnixNano":"1741365472961876027","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"ce73499edb4222cf","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"93cc9fbdbf6c44ba","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472966594131","endTimeUnixNano":"1741365472974954467","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"3dde92f7252f9083","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"2bb53729c32a58a8","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473007231493","endTimeUnixNano":"1741365473039184823","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"a0059a5619dde6c1","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473194132149","endTimeUnixNano":"1741365473197247191","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"1bc4041332aab995","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473230147420","endTimeUnixNano":"1741365473235680679","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"240e79f7811eab1d","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473233147968","endTimeUnixNano":"1741365473237846638","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"cfe6fdab47cb2a3c","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473242221457","endTimeUnixNano":"1741365473243975748","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"413b54aaa34f31c8","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473240876696","endTimeUnixNano":"1741365473244717424","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"b6e8180446b2e7ed","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473227036327","endTimeUnixNano":"1741365473248095744","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"8235f0a748cc48dc","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"b85c7f52252d0cc7","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473465152194","endTimeUnixNano":"1741365473470305873","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257}]},{"scope":{"name":"io.opentelemetry.spring-webflux-5.3","version":"2.13.3-alpha","attributes":[]},"spans":[{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"70eb7c672c4d4af9","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"815c7942a438407d","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822401504","endTimeUnixNano":"1741365473004169914","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"56575"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"recommendations"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/recommendations/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"299c5d36dba90889","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"0b40adf45794ee8d","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822422621","endTimeUnixNano":"1741365473005874829","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"41932"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"healthandwellness"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/healthandwellness/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"93cc9fbdbf6c44ba","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"db6f361024cd7afa","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472824396951","endTimeUnixNano":"1741365473025500108","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"7078"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"claims"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/claims/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"013c0bc4db6e86b6","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"dd5be42975d5f835","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822193322","endTimeUnixNano":"1741365473052594246","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"40342"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"uhcdigitalv2"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/uhcdigitalv2/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"2bb53729c32a58a8","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"f2406296738013e3","name":"GET /sample/v1/sample-categories/{category-name}/active-sample-list","kind":2,"startTimeUnixNano":"1741365472970788623","endTimeUnixNano":"1741365473055521085","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"37706"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/active-sample-list"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/active-sample-list"}},{"key":"http.request.method","value":{"stringValue":"GET"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"3f6010f926e7619b","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"8050928ff9a32835","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473166816230","endTimeUnixNano":"1741365473248374104","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"26671"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"dental"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/dental/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"0180a323ec97f0a6","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"1208321c5e7ea927","name":"GET /sample/v1/sample-categories/{category-name}/active-sample-list","kind":2,"startTimeUnixNano":"1741365473339847982","endTimeUnixNano":"1741365473349569289","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"40342"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/active-sample-list"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/active-sample-list"}},{"key":"http.request.method","value":{"stringValue":"GET"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"cbbecd508a4aa91b","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"f137b13b1b3d7708","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473439980397","endTimeUnixNano":"1741365473455375738","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"56575"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"b85c7f52252d0cc7","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"f4302aab6e9d9a53","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473443209034","endTimeUnixNano":"1741365473493427944","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"7078"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"card"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/card/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.24.0"}
Mar 07, 2025 4:37:53 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
SEVERE: Failed to export spans. The request could not be executed. Error message: expected 16901 bytes but received 17021
java.net.ProtocolException: expected 16901 bytes but received 17021
	at okhttp3.internal.connection.Exchange$RequestBodySink.write(Exchange.kt:219)
	at okio.RealBufferedSink.close(RealBufferedSink.kt:280)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:63)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at io.opentelemetry.exporter.sender.okhttp.internal.RetryInterceptor.intercept(RetryInterceptor.java:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:38:03 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.committed has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.limit has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used_after_last_gc has recorded multiple values for the same attributes: {jvm.memory.pool.name="Tenured Gen", jvm.memory.type="heap"}
Mar 07, 2025 4:38:23 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.thread.count has recorded multiple values for the same attributes: {jvm.thread.daemon=true, jvm.thread.state="runnable"}

I was going through all the changes b/w 1.42.1 and 1.44.1 and I'm wondering if this change has something to do with this "byte" error - #6686.

4 replies

jack-berg Mar 13, 2025
Maintainer

I was going through all the changes b/w 1.42.1 and 1.44.1 and I'm wondering if this change has something to do with this "byte" error - #6686.

I don't think that's the problem. That is new code only used in the experimental profile serialization.

@jack-berg Did you get a chance to look into the logs?

I spent the last couple of hours trying to recreate the error from the JSON encoded message from your log output, but was unsuccessful:

INFO: {"resource":{"attributes":[{"key":"container.id","value":{"stringValue":"ad80d452f2424a1d27edc42354d2c4f8b2889f92bcd64458b17aa36623abef6e"}},{"key":"deployment.environment","value":{"stringValue":"staging"}},{"key":"git.commit.sha","value":{"stringValue":"ed78275dd19501b8d6b07df4f71789f36245bd37"}},{"key":"git.repository_url","value":{"stringValue":"https://github.com/"}},{"key":"host.arch","value":{"stringValue":"amd64"}},{"key":"host.name","value":{"stringValue":"sample-rollout-b96597558-6wqfg"}},{"key":"os.description","value":{"stringValue":"Linux 5.10.233-224.894.amzn2.x86_64"}},{"key":"os.type","value":{"stringValue":"linux"}},{"key":"process.runtime.description","value":{"stringValue":"Eclipse Adoptium OpenJDK 64-Bit Server VM 21.0.6+7-LTS"}},{"key":"process.runtime.name","value":{"stringValue":"OpenJDK Runtime Environment"}},{"key":"process.runtime.version","value":{"stringValue":"21.0.6+7-LTS"}},{"key":"service.instance.id","value":{"stringValue":"a5c52ed2-2463-47cf-bb80-aa18d740831d"}},{"key":"service.name","value":{"stringValue":"sample"}},{"key":"service.version","value":{"stringValue":"v2025-03-07-ed78275-SNAPSHOT"}},{"key":"telemetry.distro.name","value":{"stringValue":"opentelemetry-spring-boot-starter"}},{"key":"telemetry.distro.version","value":{"stringValue":"2.13.3"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.47.0"}}]},"scopeSpans":[{"scope":{"name":"io.opentelemetry.r2dbc-1.0","version":"2.13.3-alpha","attributes":[]},"spans":[{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"c52fcbd336086c65","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"299c5d36dba90889","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472947418246","endTimeUnixNano":"1741365472957761585","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"44674e8bc5415492","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"70eb7c672c4d4af9","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472948585033","endTimeUnixNano":"1741365472958946604","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"d82c2b47a1d63f0f","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"013c0bc4db6e86b6","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472927393132","endTimeUnixNano":"1741365472961876027","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"ce73499edb4222cf","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"93cc9fbdbf6c44ba","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365472966594131","endTimeUnixNano":"1741365472974954467","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"3dde92f7252f9083","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"2bb53729c32a58a8","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473007231493","endTimeUnixNano":"1741365473039184823","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"a0059a5619dde6c1","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473194132149","endTimeUnixNano":"1741365473197247191","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"1bc4041332aab995","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473230147420","endTimeUnixNano":"1741365473235680679","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"240e79f7811eab1d","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473233147968","endTimeUnixNano":"1741365473237846638","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"cfe6fdab47cb2a3c","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473242221457","endTimeUnixNano":"1741365473243975748","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"413b54aaa34f31c8","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473240876696","endTimeUnixNano":"1741365473244717424","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"b6e8180446b2e7ed","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"3f6010f926e7619b","name":"SELECT sample.sample_contains_value","kind":3,"startTimeUnixNano":"1741365473227036327","endTimeUnixNano":"1741365473248095744","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.sql.table","value":{"stringValue":"sample_contains_value"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":"SELECT * FROM sample_contains_value WHERE sample_category_name = $1 AND sample_name = $2 AND input_key_name = $3 AND LOWER(value) = LOWER($4) AND effective_at <= $5 AND (deleted_at IS NULL OR deleted_at > $5) "}}],"events":[],"links":[],"status":{},"flags":257},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"8235f0a748cc48dc","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"b85c7f52252d0cc7","name":"SELECT sample","kind":3,"startTimeUnixNano":"1741365473465152194","endTimeUnixNano":"1741365473470305873","attributes":[{"key":"db.operation","value":{"stringValue":"SELECT"}},{"key":"server.port","value":{"intValue":"5432"}},{"key":"db.name","value":{"stringValue":"sample"}},{"key":"db.user","value":{"stringValue":"master"}},{"key":"db.connection_string","value":{"stringValue":"postgresql://postgres-rds-staging.rds.amazonaws.com:5432"}},{"key":"server.address","value":{"stringValue":"postgres-rds-staging.rds.amazonaws.com"}},{"key":"db.system","value":{"stringValue":"postgresql"}},{"key":"db.statement","value":{"stringValue":" SELECT sample_list.* FROM sample_list INNER JOIN sample_category ON sample_list.sample_category_id = sample_category.id WHERE sample_category.name = $1 AND sample_category.deleted_at IS NULL AND sample_list.effective_at <= $2 AND sample_list.active = true AND sample_list.deleted_at IS NULL ORDER BY sample_list.effective_at DESC, sample_list.created_at DESC LIMIT ? "}}],"events":[],"links":[],"status":{},"flags":257}]},{"scope":{"name":"io.opentelemetry.spring-webflux-5.3","version":"2.13.3-alpha","attributes":[]},"spans":[{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"70eb7c672c4d4af9","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"815c7942a438407d","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822401504","endTimeUnixNano":"1741365473004169914","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"56575"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"recommendations"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/recommendations/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"299c5d36dba90889","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"0b40adf45794ee8d","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822422621","endTimeUnixNano":"1741365473005874829","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"41932"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"healthandwellness"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/healthandwellness/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"93cc9fbdbf6c44ba","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"db6f361024cd7afa","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472824396951","endTimeUnixNano":"1741365473025500108","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"7078"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"claims"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/claims/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"013c0bc4db6e86b6","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"dd5be42975d5f835","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365472822193322","endTimeUnixNano":"1741365473052594246","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"40342"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"uhcdigitalv2"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/uhcdigitalv2/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"2bb53729c32a58a8","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"f2406296738013e3","name":"GET /sample/v1/sample-categories/{category-name}/active-sample-list","kind":2,"startTimeUnixNano":"1741365472970788623","endTimeUnixNano":"1741365473055521085","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"37706"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/active-sample-list"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/active-sample-list"}},{"key":"http.request.method","value":{"stringValue":"GET"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e000000000282a2a4e326fc5c2","spanId":"3f6010f926e7619b","traceState":"dd=s:1;p:267a38e36a2059a3;t.dm:-1;t.tid:67cb20e000000000","parentSpanId":"8050928ff9a32835","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473166816230","endTimeUnixNano":"1741365473248374104","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"26671"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"dental"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/dental/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"0180a323ec97f0a6","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"1208321c5e7ea927","name":"GET /sample/v1/sample-categories/{category-name}/active-sample-list","kind":2,"startTimeUnixNano":"1741365473339847982","endTimeUnixNano":"1741365473349569289","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"40342"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/active-sample-list"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/active-sample-list"}},{"key":"http.request.method","value":{"stringValue":"GET"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"cbbecd508a4aa91b","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"f137b13b1b3d7708","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473439980397","endTimeUnixNano":"1741365473455375738","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"56575"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"rewards"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/rewards/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769},{"traceId":"67cb20e100000000563eb49c34f57ff1","spanId":"b85c7f52252d0cc7","traceState":"dd=s:1;p:2dacc7c9ea2ad466;t.dm:-1;t.tid:67cb20e100000000","parentSpanId":"f4302aab6e9d9a53","name":"POST /sample/v1/sample-categories/{category-name}/evaluate","kind":2,"startTimeUnixNano":"1741365473443209034","endTimeUnixNano":"1741365473493427944","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"7078"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"categoryName","value":{"stringValue":"card"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"url.path","value":{"stringValue":"/sample/v1/sample-categories/card/evaluate"}},{"key":"network.peer.address","value":{"stringValue":"10.68.12.143"}},{"key":"server.address","value":{"stringValue":"internal-k8s-lb-staging.elb.amazonaws.com"}},{"key":"client.address","value":{"stringValue":"10.68.12.143"}},{"key":"user_agent.original","value":{"stringValue":"ReactorNetty/1.1.12"}},{"key":"http.route","value":{"stringValue":"/sample/v1/sample-categories/{category-name}/evaluate"}},{"key":"http.request.method","value":{"stringValue":"POST"}}],"events":[],"links":[],"status":{},"flags":769}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.24.0"}

See a commit with my attempt to recreate here. Steps I took:

Write deserialization logic to convert from ResourceSpans to the List<SpanData> opentelemetry-java
Deserialize the JSON payload from your logs to List<SpanData>
Re-serialize List<SpanData> using both LowAllocationTraceRequestMarshaler and TraceRequestMarshaler, confirming that that:
JSON encoding outputs match each other
Binary encodings match each other, and getBinarySerializedSize() matches the actual size of the payload

Any chance you could post a minimal sample app that recreates the issue?

psagithub Mar 14, 2025
Author

@jack-berg I've created a sample project that publishes the traces through datadog agent (Same version as the one that is running in our AWS) running in local machine. The traces are publishing fine with the latest OTel version (with some warnings on runtime metrics).

Project Repo: https://github.com/psagithub/otel-sample
DD agent:

docker pull datadog/agent:7.57.2

docker run -d --cgroupns host --pid host --name dd-agent -v /var/run/docker.sock:/var/run/docker.sock:ro \
  -v /proc/:/host/proc/:ro -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro -e DD_API_KEY=<DD API KEY> \
  -e DD_SITE=datadoghq.com -e DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT=0.0.0.0:4317 -p 4317:4317 datadog/agent:7.57.2
  
docker logs dd-agent -f

Logs:

Mar 13, 2025 11:11:31 PM io.opentelemetry.exporter.logging.otlp.internal.writer.LoggerJsonWriter write
INFO: {"resource":{"attributes":[{"key":"deployment.environment","value":{"stringValue":"otel-sample-local"}},{"key":"git.commit.sha","value":{}},{"key":"git.repository_url","value":{"stringValue":"https://github.com/psagithub/otel-sample"}},{"key":"host.arch","value":{"stringValue":"aarch64"}},{"key":"host.name","value":{"stringValue":"LAMU0MQHH2GXPGY.uhc.com."}},{"key":"os.description","value":{"stringValue":"Mac OS X 15.3.1"}},{"key":"os.type","value":{"stringValue":"darwin"}},{"key":"process.runtime.description","value":{"stringValue":"Eclipse Adoptium OpenJDK 64-Bit Server VM 21.0.4+7-LTS"}},{"key":"process.runtime.name","value":{"stringValue":"OpenJDK Runtime Environment"}},{"key":"process.runtime.version","value":{"stringValue":"21.0.4+7-LTS"}},{"key":"service.instance.id","value":{"stringValue":"167333a5-c3e0-4e3a-a35a-dfd0cbaf63d7"}},{"key":"service.name","value":{"stringValue":"otel-sample"}},{"key":"service.version","value":{}},{"key":"telemetry.distro.name","value":{"stringValue":"opentelemetry-spring-boot-starter"}},{"key":"telemetry.distro.version","value":{"stringValue":"2.13.3"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.47.0"}}]},"scopeSpans":[{"scope":{"name":"io.opentelemetry.spring-webflux-5.3","version":"2.13.3-alpha","attributes":[]},"spans":[{"traceId":"e6e9b4a1b2bd314fc95eef26a278d10b","spanId":"e7d66b48929eb442","name":"GET /otel/home","kind":2,"startTimeUnixNano":"1741921889623298000","endTimeUnixNano":"1741921889676950250","attributes":[{"key":"server.port","value":{"intValue":"8080"}},{"key":"network.peer.port","value":{"intValue":"52739"}},{"key":"url.scheme","value":{"stringValue":"http"}},{"key":"http.response.status_code","value":{"intValue":"200"}},{"key":"server.address","value":{"stringValue":"localhost"}},{"key":"client.address","value":{"stringValue":"0:0:0:0:0:0:0:1"}},{"key":"user_agent.original","value":{"stringValue":"PostmanRuntime/7.29.0"}},{"key":"network.peer.address","value":{"stringValue":"0:0:0:0:0:0:0:1"}},{"key":"url.path","value":{"stringValue":"/otel/home"}},{"key":"http.request.method","value":{"stringValue":"GET"}},{"key":"http.route","value":{"stringValue":"/otel/home"}}],"events":[],"links":[],"status":{},"flags":257}]}],"schemaUrl":"https://opentelemetry.io/schemas/1.24.0"}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.loaded has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.unloaded has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.class.count has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.time has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.count has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.cpu.recent_utilization has recorded multiple values for the same attributes: {}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="CodeHeap 'non-nmethods'", jvm.memory.type="non_heap"}
Mar 13, 2025 11:11:36 PM io.opentelemetry.sdk.internal.ThrottlingLogger doLog
WARNING: Instrument jvm.memory.used has recorded multiple values for the same attributes: {jvm.memory.pool.name="Metaspace", jvm.memory.type="non_heap"}

Helm Chart DD Config (on AWS):

locals {
  dd_default_tags = tolist([for tagk, tagv in var.tags : "${tagk}:${tagv}"])
}

resource "helm_release" "datadog_agent" {
  name       = "datadog-agent"
  chart      = "datadog"
  repository = "https://helm.datadoghq.com"
  version    = "3.102.0"
  namespace  = "datadog"
  timeout    = 600

  create_namespace = true

  values = [<<YAML
    datadog:
      env:
        - name: DD_EC2_PREFER_IMDSV2
          value: "true"
    clusterAgent:
      env:
        - name: DD_EC2_PREFER_IMDSV2
          value: "true"
  YAML
  ]

  set_list {
    name  = "datadog.tags"
    value = local.dd_default_tags
  }

  set_sensitive {
    name  = "datadog.apiKey"
    value = var.datadog_api_key
  }

  set_sensitive {
    name  = "datadog.appKey"
    value = var.datadog_app_key
  }

  set {
    name  = "datadog.apm.enabled"
    value = true
  }

  set {
    name  = "datadog.apm.portEnabled"
    value = "true"
  }

  set {
    name  = "datadog.site"
    value = "datadoghq.com"
  }

  set {
    name  = "datadog.logs.enabled"
    value = false
  }

  set {
    name  = "datadog.logs.containerCollectAll"
    value = false
  }

  set {
    name  = "datadog.leaderElection"
    value = true
  }

  set {
    name  = "datadog.collectEvents"
    value = true
  }

  set {
    name  = "datadog.containerImageCollection.enabled"
    value = true
  }

  set {
    name  = "clusterAgent.enabled"
    value = true
  }

  set {
    name  = "clusterAgent.metricsProvider.enabled"
    value = true
  }

  # This is needed for events collection (datadog.collectEvents) using cluster agent.
  set {
    name  = "clusterAgent.rbac.create"
    value = true
  }

  set {
    name  = "datadog.clusterName"
    value = module.eks.cluster_name
  }

  set {
    name  = "datadog.networkMonitoring.enabled"
    value = true
  }

  set {
    name  = "systemProbe.enableTCPQueueLength"
    value = true
  }

  set {
    name  = "systemProbe.enableOOMKill"
    value = true
  }

  set {
    name  = "securityAgent.runtime.enabled"
    value = true
  }

  set {
    name  = "datadog.hostVolumeMountPropagation"
    value = "HostToContainer"
  }

  set {
    name  = "clusterAgent.replicas"
    value = 2
  }

  set {
    name  = "clusterAgent.createPodDisruptionBudget"
    value = true
  }

  # https://docs.datadoghq.com/opentelemetry/interoperability/otlp_ingest_in_the_agent/?tab=kubernetes#enabling-otlp-ingestion-on-the-datadog-agent
  set {
    name  = "datadog.otlp.receiver.protocols.grpc.enabled"
    value = true
  }
}

jack-berg Mar 17, 2025
Maintainer

The traces are publishing fine with the latest OTel version

So are you saying the problem is resolved in the latest version?

psagithub Mar 17, 2025
Author

Not sure what's happening here. When I run the DD agent in my local, the metrics are published fine to DataDog (except the runtime metrics related warnings). But, getting the error when publishing it to the agent created by helm-chart.

psagithub · 2025-03-13T14:22:40Z

psagithub
Mar 13, 2025
Author

@jack-berg Did you get a chance to look into the logs?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Issue migrating from v1.42.1 to >=1.44.0 #7185

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments 6 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Issue migrating from v1.42.1 to >=1.44.0 #7185

Uh oh!

psagithub Mar 7, 2025

Replies: 3 comments · 6 replies

Uh oh!

psagithub Mar 7, 2025 Author

Uh oh!

jack-berg Mar 7, 2025 Maintainer

Uh oh!

jack-berg Mar 7, 2025 Maintainer

Uh oh!

psagithub Mar 7, 2025 Author

Uh oh!

jack-berg Mar 13, 2025 Maintainer

Uh oh!

psagithub Mar 14, 2025 Author

Uh oh!

jack-berg Mar 17, 2025 Maintainer

Uh oh!

psagithub Mar 17, 2025 Author

Uh oh!

psagithub Mar 13, 2025 Author

psagithub
Mar 7, 2025

Replies: 3 comments 6 replies

psagithub
Mar 7, 2025
Author

jack-berg Mar 7, 2025
Maintainer

jack-berg Mar 7, 2025
Maintainer

psagithub
Mar 7, 2025
Author

jack-berg Mar 13, 2025
Maintainer

psagithub Mar 14, 2025
Author

jack-berg Mar 17, 2025
Maintainer

psagithub Mar 17, 2025
Author

psagithub
Mar 13, 2025
Author