Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #2564
The PR adds a
priority
property to run configurations that allows controlling the order in which the runs are provisioned. Runs with higher priorities take precedence over runs with lower priorities. Previously, submitted jobs were processed in FIFO manner with older jobs processed first. Now, the jobs are first sorted by descending priority.The
priority
property is updatable, so it can be changed for already submitted runs and will take effect:Note that if a high priority run cannot be scheduled, it does NOT block other runs with lower priority from scheduling (aka Best effort FIFO). So dstack will try a higher priority run, and if it fails, moves to a lower priority one. This can lead to a lower priority run "overtaking" higher priority run and is considered ok. An alternative is Strict FIFO which blocks lower priority runs until a higher priority run is scheduled. It would be a worse default since one unschedulable run can block all runs in the project, but we may consider supporting it in the future if requested.