Pass API token as kubernetes secret #3421

schustmi · 2025-03-17T19:34:57Z

Describe changes

Add option to pass the ZenML server API token as a Kubernetes secret instead of plain environment variable.

Pre-requisites

Please ensure you have done the following:

I have read the CONTRIBUTING.md document.
I have added tests to cover my changes.
I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
IMPORTANT: I made sure that my changes are reflected properly in the following resources:
- ZenML Docs
- Dashboard: Needs to be communicated to the frontend team.
- Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
- Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Other (add details above)

coderabbitai · 2025-03-17T19:35:05Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

schustmi · 2025-03-17T19:35:24Z

@stefannica Are you okay with this approach in general? If yes I'd clean this up a little and write some docs for it

schustmi · 2025-03-18T09:45:35Z

src/zenml/integrations/kubernetes/orchestrators/kubernetes_orchestrator_entrypoint.py

@@ -174,6 +192,20 @@ def run_step_on_kubernetes(step_name: str) -> None:

    logger.info("Orchestration pod completed.")

+    if (


I only run this cleanup in case the orchestration pod exited succesfully. This is in order to prevent accidently deleting the secret while a step pod is still running, but will lead to some secrets being left over if the pipeline failed. @stefannica what do you think about this?

If I'm reading the ThreadedDagRunner implementation correctly, this code is only reached when all of the pods have completed running (i.e. all of the threads have been "joined"). So you don't need this condition at all.

Which condition? I'm not explicitly checking for success anywhere, but implicitly by putting the code where it is right now

Oh, I see... then why don't you just use a try/finally around ThreadedDagRunner.run() ?

That is precisely my question: This would handle also the case where a step is failing. There is however one other option that I've also seen sometimes: The dag runner fails with some random error, even though the step is actually still running. In that case, we would remove the secret already. Now that I'm typing it however, I realize that at this point the token would already be set as environment variable, which means we can delete it without disturbing the (potentially still active) step. I'll adjust it.

* POC: Pass API token as kubernetes secret * Better secret name * Cleanup secrets and docs * Dont regenerate unnecessary tokens * Formatting * Docsrtrings * Always run secret cleanup * Formatting

POC: Pass API token as kubernetes secret

97c3406

github-actions bot added internal To filter out internal PRs and issues enhancement New feature or request labels Mar 17, 2025

schustmi added 2 commits March 17, 2025 20:57

Better secret name

3b3ab21

Cleanup secrets and docs

2d2780d

schustmi marked this pull request as ready for review March 18, 2025 09:43

schustmi commented Mar 18, 2025

View reviewed changes

schustmi requested a review from stefannica March 18, 2025 09:45

schustmi added 5 commits March 18, 2025 11:24

Dont regenerate unnecessary tokens

2bbc6d1

Formatting

d1ea06b

Docsrtrings

104e819

Merge branch 'develop' into feature/pass-api-token-as-kubernetes-secret

122eaa6

Always run secret cleanup

3374e39

stefannica approved these changes Mar 18, 2025

View reviewed changes

Formatting

5efc68b

schustmi merged commit 34d64bb into develop Mar 18, 2025
43 of 45 checks passed

schustmi deleted the feature/pass-api-token-as-kubernetes-secret branch March 18, 2025 15:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Pass API token as kubernetes secret #3421

Pass API token as kubernetes secret #3421

Uh oh!

schustmi commented Mar 17, 2025 •

edited

Loading

Uh oh!

coderabbitai bot commented Mar 17, 2025

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

Uh oh!

schustmi commented Mar 17, 2025

Uh oh!

schustmi Mar 18, 2025

Uh oh!

stefannica Mar 18, 2025

Uh oh!

schustmi Mar 18, 2025

Uh oh!

stefannica Mar 18, 2025

Uh oh!

schustmi Mar 18, 2025

Uh oh!

Uh oh!

Uh oh!

		@@ -174,6 +192,20 @@ def run_step_on_kubernetes(step_name: str) -> None:

		logger.info("Orchestration pod completed.")

		if (

Pass API token as kubernetes secret #3421

Pass API token as kubernetes secret #3421

Uh oh!

Conversation

schustmi commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe changes

Pre-requisites

Types of changes

Uh oh!

coderabbitai bot commented Mar 17, 2025

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

Uh oh!

schustmi commented Mar 17, 2025

Uh oh!

schustmi Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

stefannica Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

schustmi Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

stefannica Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

schustmi Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

schustmi commented Mar 17, 2025 •

edited

Loading