-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[ENH] Add load service to local dev tiltfile #4397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
config.yaml: | | ||
load_service: | ||
service_name: "chroma-load" | ||
otel_endpoint: "http://otel-collector.chroma:4317" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.chroma
... It's always DNS 🙃
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
classic
.with_http() | ||
.with_http_client(client) | ||
let tracing_span_exporter = opentelemetry_otlp::SpanExporter::builder() | ||
.with_tonic() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so this was mis-configured to use HTTP instead of gRPC too? Is that what the with_tonic()
also changes?
aside: I wish that this was somehow made our "default" Chroma setup so that it's consistent for all our rust code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only chroma-load is not using our default otel setup. See tracing crate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is because chroma-load was exporting to honeycomb directly. I'd love to see it unified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might want to be careful with "log spew" given that the tracer is now set to INFO
level -- I haven't audited the callsites, but it may be a thing we set dynamically.
abe253e
to
978c418
Compare
Make the trace exporter to export all INFO and above traces (not just ERROR).
978c418
to
339db10
Compare
Description of changes
Make the trace exporter to export all INFO and above traces (not just ERROR).
Also, update load service tracing to use GRPC (to match our other services).
Test plan
Ran locally via
tilt up
. Verified traces appear underchroma-load
service in jaeger.Documentation Changes
n/a