Skip to content

Refactor Redis config to use redis.Options struct #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 3, 2025
Merged

Refactor Redis config to use redis.Options struct #37

merged 1 commit into from
Jun 3, 2025

Conversation

relyt0925
Copy link
Contributor

Simplified Redis configuration by consolidating individual parameters (address, password, and DB) into the redis.Options struct. This improves code readability, reduces redundancy, and aligns with the Redis client API's recommended usage.

@relyt0925 relyt0925 marked this pull request as draft June 1, 2025 03:42
@relyt0925 relyt0925 force-pushed the redis-auth-support branch from b757ac5 to 3e9b3e6 Compare June 1, 2025 03:45
This was referenced Jun 1, 2025
Closed
Closed
@relyt0925
Copy link
Contributor Author

Doing e2e test by doing build of llm-d-inference-scheduler with this change in place and going to ensure

  1. current strategy of specifying redis variable with addresses results in same redis option object
  2. New code enables a secure connection to a redis instance with user/pass over TLS.

Once done will convert from draft to pr.

@relyt0925
Copy link
Contributor Author

E2E test passed in KV Cache manager: saw previously errors specifying redis url could not be read shown in previous pr:

Now inference-scheduler shows clean logs and successful startup of scorers

{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"epp/main.go:146","msg":"Flags processed","flags":{"DestinationEndpointHintMetadataNamespace":"envoy.lb","certPath":"","destinationEndpointHintKey":"x-gateway-destination-endpoint","grpcHealthPort":9003,"grpcPort":9002,"kubeconfig":"","kvCacheUsagePercentageMetric":"vllm:gpu_cache_usage_perc","loraInfoMetric":"vllm:lora_requests_info","metricsPort":9090,"poolName":"llama-3-1-8b-instruct-inference-pool","poolNamespace":"rhibm-models","refreshMetricsInterval":50000000,"refreshPrometheusMetricsInterval":5000000000,"secureServing":true,"totalQueuedRequestsMetric":"vllm:num_requests_waiting","v":4,"zap-devel":true,"zap-encoder":{},"zap-log-level":{},"zap-stacktrace-level":{},"zap-time-encoding":{}}}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"ENABLE_KVCACHE_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"KVCACHE_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"KVCACHE_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"ENABLE_LOAD_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"LOAD_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"LOAD_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"ENABLE_PREFIX_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFIX_AWARE_SCORER_WEIGHT","value":2}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"PREFIX_AWARE_SCORER","weight":2}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"ENABLE_SESSION_AWARE_SCORER","value":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"SESSION_AWARE_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_LEAST_KVCACHE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LEAST_KVCACHE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_LEAST_QUEUE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LEAST_QUEUE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_LORA_AFFINITY_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LORA_AFFINITY_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_LOW_QUEUE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LOW_QUEUE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_SHEDDABLE_CAPACITY_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_SHEDDABLE_CAPACITY_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_KVCACHE_UTILIZATION_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_KVCACHE_UTILIZATION_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_QUEUE_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_QUEUE_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"ENABLE_GIE_PREFIX_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_PREFIX_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_ENABLE_KVCACHE_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_KVCACHE_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"KVCACHE_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_ENABLE_LOAD_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_LOAD_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"LOAD_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_ENABLE_PREFIX_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_PREFIX_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"PREFIX_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_ENABLE_SESSION_AWARE_SCORER","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PREFILL_SESSION_AWARE_SCORER_WEIGHT","value":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:119","msg":"Initialized plugin","plugin":"SESSION_AWARE_SCORER","weight":1}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_LEAST_KVCACHE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LEAST_KVCACHE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_LEAST_QUEUE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LEAST_QUEUE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_LORA_AFFINITY_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LORA_AFFINITY_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_LOW_QUEUE_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_LOW_QUEUE_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_SHEDDABLE_CAPACITY_FILTER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_SHEDDABLE_CAPACITY_FILTER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_KVCACHE_UTILIZATION_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_KVCACHE_UTILIZATION_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_QUEUE_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_QUEUE_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFILL_ENABLE_GIE_PREFIX_SCORER","defaultValue":"false"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"config/config.go:114","msg":"Skipping plugin creation as it is not enabled","name":"GIE_PREFIX_SCORER"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PD_ENABLED","value":"true"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:29","msg":"Successfully loaded environment variable","key":"PD_PROMPT_LEN_THRESHOLD","value":10}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"PREFIX_SCORER_BLOCK_SIZE","defaultValue":256}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"scorer/prefix_aware.go:154","msg":"Prefix aware scorer cleanup started"}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"LOAD_AWARE_SCORER_QUEUE_THRESHOLD","defaultValue":128}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"env/env.go:18","msg":"Environment variable not set, using default value","key":"LOAD_AWARE_SCORER_QUEUE_THRESHOLD","defaultValue":128}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"setup","caller":"epp/main.go:224","msg":"Controller manager starting"}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"manager/server.go:83","msg":"starting server","name":"metrics","addr":":9090"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"controller-runtime.metrics","caller":"server/server.go:208","msg":"Starting metrics server"}
{"level":"info","ts":"2025-06-01T18:38:47Z","logger":"controller-runtime.metrics","caller":"server/server.go:247","msg":"Serving metrics server","bindAddress":":8080","secure":false}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"runnable/grpc.go:26","msg":"gRPC server starting","name":"health"}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"runnable/grpc.go:34","msg":"gRPC server listening","name":"health","port":9003}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"controller/controller.go:204","msg":"Starting EventSource","controller":"pod","controllerGroup":"","controllerKind":"Pod","source":"kind source: *v1.Pod"}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"controller/controller.go:204","msg":"Starting EventSource","controller":"inferencepool","controllerGroup":"inference.networking.x-k8s.io","controllerKind":"InferencePool","source":"kind source: *v1alpha2.InferencePool"}
{"level":"info","ts":"2025-06-01T18:38:47Z","caller":"controller/controller.go:204","msg":"Starting EventSource","controller":"inferencemodel","controllerGroup":"inference.networking.x-k8s.io","controllerKind":"InferenceModel","source":"kind source: *v1alpha2.InferenceModel"}

@relyt0925 relyt0925 marked this pull request as ready for review June 1, 2025 19:10
@relyt0925
Copy link
Contributor Author

Also note: need to work with LMCache community to support same configuration standard in the LMCache package that is packaged into llm-d itself: PR is in progress but shouldn't block having support in this layer of stack:

https://github.com/LMCache/LMCache/pull/737/files

vMaroon
vMaroon approved these changes Jun 2, 2025
View reviewed changes
Copy link
Member

@vMaroon vMaroon left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thank you for your contribution!
@relyt0925 could you amend and sign the commit?

@relyt0925
Copy link
Contributor Author

yes!

@relyt0925 relyt0925 force-pushed the redis-auth-support branch from 3e9b3e6 to d3ab0b8 Compare June 2, 2025 19:40
@relyt0925
Copy link
Contributor Author

amended and signed @vMaroon

@vMaroon
Copy link
Member

vMaroon commented Jun 2, 2025

@relyt0925 need to also verify signature then ready to go.

@relyt0925
Copy link
Contributor Author

Got it will do in couple hrs

@relyt0925
Refactor Redis config to use redis.Options struct
25384d0 
Simplified Redis configuration by consolidating individual parameters (address, password, and DB) into the redis.Options struct. This improves code readability, reduces redundancy, and aligns with the Redis client API's recommended usage.

Signed-off-by: Tyler Lisowski <[email protected]>
@relyt0925 relyt0925 force-pushed the redis-auth-support branch from d3ab0b8 to 25384d0 Compare June 2, 2025 22:57
@relyt0925
Copy link
Contributor Author

@vMaroon Updated with verified signature!!!

@vMaroon vMaroon merged commit c7f0332 into llm-d:main Jun 3, 2025
1 check passed
@relyt0925 relyt0925 deleted the redis-auth-support branch June 3, 2025 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vMaroon vMaroon vMaroon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@relyt0925 @vMaroon