-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[ENH] Configure the cache to have a hostPath and mountPath for log service. #4483
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…rvice. This adds defaults to values.yaml that make k8s instantiate the log with a cache dir mounted.
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
This PR adds cache configuration for the Rust log service in the Kubernetes deployment by introducing hostPath and mountPath parameters in values.yaml. It also bumps the Helm chart version accordingly from 0.1.38 to 0.1.39. This summary was automatically generated by @propel-code-bot |
✅ The Helm chart's version was changed. Your changes to the chart will be published upon merge to |
k8s/distributed-chroma/values.yaml
Outdated
@@ -51,6 +51,10 @@ rustLogService: | |||
image: | |||
repository: 'rust-log-service' | |||
tag: 'latest' | |||
env: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[DataTypeCheck]
The env:
key on line 54 implies an empty map ({}
). However, other services in this file (e.g., sysdb
, logService
) define env
as a list of name/value pairs. If Helm templates expect a list (e.g., for iteration with range
), this type mismatch can cause errors or unexpected behavior. For an empty list of environment variables, it should be env: []
to maintain consistency and ensure correct template processing.
env: | |
env: [] |
⚡ Committable suggestion
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with this suggestion. Though, why do we need the env
field here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yaml cargo culting on my part. I'll remove.
@@ -51,6 +51,10 @@ rustLogService: | |||
image: | |||
repository: 'rust-log-service' | |||
tag: 'latest' | |||
env: | |||
cache: | |||
hostPath: '/local/cache/chroma-log-service' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[ArchitectureDecision]
Using hostPath
for the cache volume (line 56) ties the cache data to a specific node. In a multi-node Kubernetes cluster, if the pod is rescheduled to a different node, the cache data will be lost or become inconsistent. For production deployments or multi-node environments requiring persistent cache, consider using PersistentVolumeClaim
(PVC) for more robust and portable cache storage. hostPath
is generally more suitable for single-node clusters or local development setups where node affinity is guaranteed and simplicity is prioritized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also agree with this suggestion, but understand that we have made this trade-off for performance reasons.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you think we can have cache coherency across deploys another way we can make that change. it’s possible to not use hostpath but still mount the local disk. We should probably do that, i think it was just some clunkiness on our deployed env that made it hard
Alternatively if you have a system design that doesn’t need cache coherency would love to hear about it.
Description of changes
This adds defaults to values.yaml that make k8s instantiate the log with a cache dir mounted.
Test plan
CI + eyes.
Documentation Changes
N/A