git bisecting broken?

[matthiaskrgr]

I was trying to bisect rust-lang/rust#87308
with --start=2021-07-18 --end=2021-07-20

While bisecting the actual nightly seemed to have worked, cargo-bisect-rustc failed to bisect the actual PR

installing nightly-2021-07-18
cargo for x86_64-unknown-linux-gnu: 6.07 MB / 6.07 MB [=================================================================================================================================================================================================================================================] 100.00 % 6.77 MB/s testing...
RESULT: nightly-2021-07-18, ===> No
uninstalling nightly-2021-07-18

installing nightly-2021-07-20
cargo for x86_64-unknown-linux-gnu: 6.07 MB / 6.07 MB [=================================================================================================================================================================================================================================================] 100.00 % 6.76 MB/s testing...
RESULT: nightly-2021-07-20, ===> Yes
uninstalling nightly-2021-07-20

installing nightly-2021-07-19
testing...
RESULT: nightly-2021-07-19, ===> Yes
uninstalling nightly-2021-07-19

searched toolchains nightly-2021-07-18 through nightly-2021-07-20


********************************************************************************
Regression in nightly-2021-07-19
********************************************************************************

fetching https://static.rust-lang.org/dist/2021-07-18/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2021-07-18: 40 B / 40 B [============================================================================================================================================================================================================================================================] 100.00 % 640.76 KB/s converted 2021-07-18 to c7331d65bdbab1187f5a9b8f5b918248678ebdb9
fetching https://static.rust-lang.org/dist/2021-07-19/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2021-07-19: 40 B / 40 B [============================================================================================================================================================================================================================================================] 100.00 % 256.19 KB/s converted 2021-07-19 to 59216858a323978a97593cba22b5ed84350a3783
looking for regression commit between 2021-07-18 and 2021-07-19
cloning rust repository
fetching (via local git) commits from c7331d65bdbab1187f5a9b8f5b918248678ebdb9 to 59216858a323978a97593cba22b5ed84350a3783
opening existing repository at "rust.git"
refreshing repository
looking up first commit
looking up second commit
checking that commits are by bors and thus have ci artifacts...
finding bors merge commits
found 8 bors merge commits in the specified range
  commit[0] 2021-07-17UTC: Auto merge of #87203 - jackh726:logging, r=nikomatsakis
  commit[1] 2021-07-17UTC: Auto merge of #87129 - FabianWolff:issue-75356, r=varkor
  commit[2] 2021-07-18UTC: Auto merge of #85686 - ptrojahn:loop_reinitialize, r=estebank
  commit[3] 2021-07-18UTC: Auto merge of #87071 - inquisitivecrystal:inclusive-range, r=estebank
  commit[4] 2021-07-18UTC: Auto merge of #87242 - JohnTitor:rollup-t9rmwpo, r=JohnTitor
  commit[5] 2021-07-18UTC: Auto merge of #86698 - cjgillot:modc, r=estebank
  commit[6] 2021-07-18UTC: Auto merge of #87252 - RalfJung:miri, r=RalfJung
  commit[7] 2021-07-18UTC: Auto merge of #86950 - tmiasko:personality, r=nagisa
validated commits found, specifying toolchains

installing c7331d65bdbab1187f5a9b8f5b918248678ebdb9
cargo for x86_64-unknown-linux-gnu: 6.07 MB / 6.07 MB [=================================================================================================================================================================================================================================================] 100.00 % 3.42 MB/s testing...
RESULT: c7331d65bdbab1187f5a9b8f5b918248678ebdb9, ===> No
uninstalling c7331d65bdbab1187f5a9b8f5b918248678ebdb9

installing 59216858a323978a97593cba22b5ed84350a3783
cargo for x86_64-unknown-linux-gnu: 6.07 MB / 6.07 MB [=================================================================================================================================================================================================================================================] 100.00 % 3.38 MB/s testing...
RESULT: 59216858a323978a97593cba22b5ed84350a3783, ===> No
uninstalling 59216858a323978a97593cba22b5ed84350a3783

ERROR: the commit at the end of the range (59216858a323978a97593cba22b5ed84350a3783) does not reproduce the regression

[ehuss]

Hm, something seems strange here. When I run the test, I get that nightly-2021-07-19 does not regress. Did you use --regress=ice? Perhaps run with -vv to see why it thinks nightly-2021-07-19 failed?

Successful build with 2021-07-19:

> RUSTFLAGS=-Zunpretty=everybody_loops cargo +bisector-nightly-2021-07-19-x86_64-apple-darwin build
   Compiling z72 v0.1.0 (/Users/eric/Proj/rust/cargo/scratch/z72)
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2018::*;
#[macro_use]
extern crate std;
macro_rules! foo {
    () => { break 'x ; } //~ ERROR use of undeclared label `'x`
}

pub fn main() { loop { } }
    Finished dev [unoptimized + debuginfo] target(s) in 0.16s

[dtolnay]

I've hit this too. I think it is related to #123. It's off-by-one somewhere.

For example in rust-lang/rust#88043, where cargo +nightly-2021-08-14 build is successful while cargo +nightly-2021-08-15 build is regressed, cargo-bisect-rustc has this unexpected behavior:

$ cargo-bisect-rustc --start 2021-08-14 --end 2021-08-15 -- build
checking nightly-2021-08-14 from the currently installed default nightly toolchain as the last failure
installing nightly-2021-08-14
testing...
RESULT: nightly-2021-08-14, ===> Yes
uninstalling nightly-2021-08-14

ERROR: the start of the range (nightly-2021-08-14) must not reproduce the regression

Okay, let's try to offset by one. But it still wrongly concludes the regression is in nightly-2021-08-14 (whereas it's really in nightly-2021-08-15) and then predictably fails to bisect at the commit level.

$ cargo-bisect-rustc --start 2021-08-13 --end 2021-08-15 -- build
installing nightly-2021-08-13
RESULT: nightly-2021-08-13, ===> No
uninstalling nightly-2021-08-13

installing nightly-2021-08-15
RESULT: nightly-2021-08-15, ===> Yes
uninstalling nightly-2021-08-15

installing nightly-2021-08-14
testing...
RESULT: nightly-2021-08-14, ===> Yes
uninstalling nightly-2021-08-14

searched toolchains nightly-2021-08-13 through nightly-2021-08-15


********************************************************************************
Regression in nightly-2021-08-14
********************************************************************************

fetching https://static.rust-lang.org/dist/2021-08-13/channel-rust-nightly-git-commit-hash.txt
fetching https://static.rust-lang.org/dist/2021-08-14/channel-rust-nightly-git-commit-hash.txt
looking for regression commit between 2021-08-13 and 2021-08-14
opening existing repository at "/git/bisect.rustc"
refreshing repository
fetching (via local git) commits from 0fa3190394475a84360b34e074e719d519bc40f1 to 5a19ffe1c2b99d9e09706cc286aad1ec0868eddb
opening existing repository at "/git/bisect.rustc"
refreshing repository
looking up first commit
looking up second commit
checking that commits are by bors and thus have ci artifacts...
finding bors merge commits
found 8 bors merge commits in the specified range
  commit[0] 2021-08-12UTC: Auto merge of #87916 - nbdd0121:black_box, r=nagisa
  commit[1] 2021-08-12UTC: Auto merge of #87980 - Manishearth:rollup-vkuix3y, r=Manishearth
  commit[2] 2021-08-13UTC: Auto merge of #87927 - spastorino:use-def-id-typeckresults, r=oli-obk
  commit[3] 2021-08-13UTC: Auto merge of #87954 - flip1995:clippyup, r=Manishearth
  commit[4] 2021-08-13UTC: Auto merge of #87956 - m-ou-se:closure-migration-macro-body, r=Aaron1011
  commit[5] 2021-08-13UTC: Auto merge of #88009 - GuillaumeGomez:rollup-f194yyk, r=GuillaumeGomez
  commit[6] 2021-08-13UTC: Auto merge of #87984 - m-ou-se:closure-lint-wording, r=Aaron1011
  commit[7] 2021-08-13UTC: Auto merge of #86492 - hyd-dev:no-mangle-method, r=petrochenkov
validated commits found, specifying toolchains

installing 0fa3190394475a84360b34e074e719d519bc40f1
RESULT: 0fa3190394475a84360b34e074e719d519bc40f1, ===> No
uninstalling 0fa3190394475a84360b34e074e719d519bc40f1

installing 5a19ffe1c2b99d9e09706cc286aad1ec0868eddb
RESULT: 5a19ffe1c2b99d9e09706cc286aad1ec0868eddb, ===> No
uninstalling 5a19ffe1c2b99d9e09706cc286aad1ec0868eddb

ERROR: the commit at the end of the range (5a19ffe1c2b99d9e09706cc286aad1ec0868eddb) does not reproduce the regression

[ehuss]

Oh! I have a guess as to what is wrong. Which version of cargo-bisect-rustc are you using? I always use latest master (actually, I use a local branch with several other fixes). The last published release was a long time ago, and there was an off-by-one fix done in the meantime (#113), among many other fixes. Is that maybe the case here?

[camelid]

I think I was using the latest released version, 0.6.0, in #123.

[ehuss]

@spastorino Would you be able to publish a new version? Or, if you would like people to help, I can publish if you invite me (though I'm not sure who has write access here). It would also be helpful if someone could take a look at #130 and #131, as the current master has issues for me.

[spastorino]

@ehuss sorry that I've missed your message. I've added you on crates.io. Let me know if you're able to publish.

[ehuss]

I have access on crates.io now, but I'll need to be able to push to this repo to update the version and such. @Mark-Simulacrum would it be alright to give me write access to this repo?

[Mark-Simulacrum]

Yeah, that makes sense to me. We should have some discussion about where the maintenance of this crate/repository best connects -- release team may make sense...

[ehuss]

I just published 0.6.1. I'm going to close as I'm going to assume this was caused by #113 or maybe #125. Feel free to open a new issue if things don't seem resolved with the new version.