solver: handle not found error on cache export #5999
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tonistiigi
force-pushed
the
cache-export-notfound
branch
from
b1368e2 to
a252c4e
Compare
When a cleanup happens while a build is running it may cause boltdb record to be deleted while that record would have been used for cache export of the running build, causing the cache export to fail with Not Found error. This is possible because when cache is loaded to snapshots then snapshots are reference counted, protecting against them being released or deleted, but when cache key is just intermediary, the query result just gets saved and is used later. It should be safe to ignore this error and skip over the result if it doesn't exist anymore. Note that cache export also gets called on provenance creation because cache chains are used for finding remote descriptors for build steps. This was the more likely codepath for hitting this error. Signed-off-by: Tonis Tiigi <[email protected]>
tonistiigi
force-pushed
the
cache-export-notfound
branch
from
a252c4e to
705a9bc
Compare
4 tasks
AkihiroSuda
approved these changes
Jun 5, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #5784 (hopefully)
possible that this appears more frequently after #5692
When a cleanup happens while a build is running it may cause boltdb record to be deleted while that record would have been used for cache export of the running build, causing the cache export to fail with Not Found error. This is possible because when the cache is loaded to snapshots, then snapshots are reference counted, protecting against them being released or deleted, but when the cache key is just an intermediary, the query result just gets saved and is used later.
It should be safe to ignore this error and skip over the result if it doesn't exist anymore.
Note that cache export also gets called on provenance creation because cache chains are used for finding remote descriptors for build steps. This was the more likely codepath for hitting this error.