Skip to content

Improve cluster deletion #1864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 6, 2025
Merged

Conversation

mkuratczyk
Copy link
Collaborator

There were 3 issues:

  • the STS could get deleted before deletion label propagated, leading to some pods terminating for a long time, because they didn't skip quorum checks
  • there was an unnecessary explicit STS deletion
  • addRabbitmqDeletionLabel was not filtering by namespace, potentially performing operations on wrong/too many pods (if there were multiple clusters with the same name in different namespaces)

There were 3 issues:
* the STS could get delete before deletion label propagated,
  leading to some pods terminating for a long time, because they
  didn't skip quorum checks
* there was an unnecessary explicit STS deletion
* addRabbitmqDeletionLabel was not filtering by namespace,
  potentially performing operations on wrong/too many pods
  (if there were multiple clusters with the same name in different
  namespaces)
Copy link
Member

@Zerpet Zerpet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to object, the solution looks good to me 👍 Glad you figured out the problem!

Out of curiosity, did you consider using CreateOrUpdate() function in removeFinalizer() ?
CreateOrUpdate does essentially what you are doing of Get prior to Update()

@mkuratczyk
Copy link
Collaborator Author

I had not tried before but I've just pushed a commit which uses CreateOrUpdate and seems to work just as well.
Happy to keep this version if you prefer

@Zerpet Zerpet added this to the 2.12.2 milestone May 6, 2025
@Zerpet
Copy link
Member

Zerpet commented May 6, 2025

Ship it! :shipit:

@mkuratczyk mkuratczyk merged commit fe56e19 into main May 6, 2025
13 checks passed
@mkuratczyk mkuratczyk deleted the wait-for-label-propagation-before-deleting branch May 6, 2025 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants