Skip to content

Ensure to have always only one retry or re-schedule event in place for a CustomResource #601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
csviri opened this issue Oct 13, 2021 · 2 comments · Fixed by #609
Closed
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. scheduling-improvements-epic
Milestone

Comments

@csviri
Copy link
Collaborator

csviri commented Oct 13, 2021

Currently it can happen that there are multiple events scheduled from reSchedule funtionality. This should not happen, there should be at most one, possibly the earliest keeped in such case. In addition to that we have a automated retry functionality, so we have to think about how this two should work together.

Ideally there is always only 1 of these to is scheduled types scheduled. But also in general there should be at most one timed event coming either of these functionalities.

@metacosm
Copy link
Collaborator

The rationale for this is that if you schedule a reconciliation to happen at some later time in the future it's because you don't expect a reconciliation to happen "naturally" because of a cluster-triggered event. It follows, then, that if a reconciliation is triggered before the specified time delay, that scheduled reconciliation should be cancelled.
Similarly, if a reconciliation is scheduled at some future time and a new request for rescheduling comes in, then only the rescheduling request that would trigger an earlier reconciliation should be kept (for the same reason: a rescheduling request is meant to ensure at least one reconciliation happens by the requested delay, so if a reconciliation happens before, then the request should be considered fulfilled).

@csviri
Copy link
Collaborator Author

csviri commented Oct 15, 2021

Just to summarize the behavior:

  1. every successful reconciliation run will cancel the scheduled events (either re-schedule or retry)
  2. if a retry or re-schedule is happening, it will also cancel the previous schedules.

@metacosm I think we should take just the "last instruction" wins. Not taking into account the timing. So even if a new re-schedule happens but there is already one with an earlier execution planned, the new one should be the lead and we should cancel the old one. This makes sense because the new one is set by a more recent reconciliation aware of the current state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. scheduling-improvements-epic
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants