Skip to content

Prune inputs #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 19, 2024
Merged

Prune inputs #177

merged 3 commits into from
Mar 19, 2024

Conversation

dergoegge
Copy link
Member

@dergoegge dergoegge commented Mar 18, 2024

As per the usual process to avoid wasted CI resources and timeouts when it runs on large and presumed irrelevant inputs.

Previous: #157

To "reproduce"

Install a fresh VM, as explained in the bash script's doc, and run it:

apt update && apt install wget -y
wget https://raw.githubusercontent.com/bitcoin-core/bitcoin-maintainer-tools/main/delete_nonreduced_fuzz_inputs.sh
bash delete_nonreduced_fuzz_inputs.sh

To "test"

  • Keep an eye on coverage stats, to ensure it doesn't drop
  • Re-run the script, to ensure it is "reproducible" to some extent
  • Anything else you think is important to test or review

@maflcko
Copy link
Contributor

maflcko commented Mar 18, 2024

Please add a link to the previous one, and the steps to reproduce, or a link to it.

@dergoegge
Copy link
Member Author

Please add a link to the previous one, and the steps to reproduce, or a link to it.

Done

@maflcko
Copy link
Contributor

maflcko commented Mar 18, 2024

Looks like the runtime remains roughly the same? So maybe it is not worth it this time?

@dergoegge
Copy link
Member Author

4.4GB vs 1.6GB on this PR with no coverage loss. I don't see the benefit of keeping the redundant inputs around.

@maflcko
Copy link
Contributor

maflcko commented Mar 18, 2024

Ok, fair enough.

@maflcko
Copy link
Contributor

maflcko commented Mar 18, 2024

When I try to reproduce, I get subprocess timed out: Currently only libFuzzer is supported for the second step with the sanitizers.

@maflcko
Copy link
Contributor

maflcko commented Mar 19, 2024

Ok, I re-ran on a different machine and it somehow passed this time. Results:

  • About 100k deleted files
# git diff origin/main..fe4760b23a848101bf6e619bcaef80136951d9f4 --stat | tail -1
 113815 files changed, 574351 deletions(-)
  • Cross-diff of 25k files:
# git -c diff.renameLimit=13303 diff HEAD..fe4760b23a848101bf6e619bcaef80136951d9f4 --stat | tail -1
 24735 files changed, 9163 insertions(+), 285203 deletions(-)

@maflcko
Copy link
Contributor

maflcko commented Mar 19, 2024

lgtm

@dergoegge dergoegge merged commit f488d77 into bitcoin-core:main Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants