-
Notifications
You must be signed in to change notification settings - Fork 13.4k
Fix linking statics on Arm64EC #140176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix linking statics on Arm64EC #140176
Conversation
r? @Nadrieril rustbot has assigned @Nadrieril. Use |
Some changes occurred in compiler/rustc_codegen_ssa |
#
#
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look good to me. Is it possible to write a regression test? (Either a full-scale integration test that links an exported static from one crate to another or a test that imports a static and dumps the symbol name or something?)
r? @wesleywiser |
@rustbot author |
Reminder, once the PR becomes ready for a review, use |
This PR modifies cc @jieyouxu |
Good call, found another bug: we need to correctly export variables as @rustbot ready |
#
This comment has been minimized.
This comment has been minimized.
Thanks @dpaoliello! @bors r+ rollup Also nominating for potential beta backport as this fixes a serious issue for the Tier 2 |
Hmm, while that fixes our build of cargo, we're now hitting the same warning when linking Chromium, and we definitely don't want to disable this warning globally. This seems like a regression. |
@bjorn3 is this something you can help with? There are two ways that this could be fixed without needing a warning suppression for the linker:
I don't know enough about the architecture of the compiler to know which of these approaches is plausible, though I would argue that the second is more "correct" (i.e., we are no longer lying to the compiler and telling it that |
If it takes a while to figure out a good way forward, would it be possible to revert to green in the meantime? |
We're also running into this issue on Fuchsia trying to build our stage0 windows rustc. If we do a revert, could we also cut a new stage0 as well? |
The beta backport nomination was rejected in the compiler async backport thread. @rustbot label: -beta-nominated |
The timing here is a bit unfortunate because most team members are N/A this week due to RustWeek / All Hands. I think it's safer to attempt a revert at this time, before we figure out a better solution for EDIT: revert PR is #141024. |
Unfortunately, multiple people are reporting linker warnings related to `__rust_no_alloc_shim_is_unstable` after this change. The solution isn't quite clear yet, let's revert to green for now, and try a reland with a determined solution for `__rust_no_alloc_shim_is_unstable`. This reverts commit c8b7f32, reversing changes made to 667247d.
Revert "Fix linking statics on Arm64EC rust-lang#140176" This reverts PR rust-lang#140176. Unfortunately, this will reopen rust-lang#138541 (re-breaking the `arm64ec-pc-windows-msvc` target). Unfortunately, multiple people are [reporting linker warnings related to `__rust_no_alloc_shim_is_unstable`](rust-lang#140176 (comment)) after this change in `x86_64-pc-windows-msvc` as well. The solution isn't quite clear yet, let's revert to avoid the linker warnings on the Tier 1 MSVC target for now[^timing], and try a reland with a determined solution for `__rust_no_alloc_shim_is_unstable`. Judging from [people reporting that they are observing this also when bootstrapping w/ stage0 rustc](rust-lang#140176 (comment)), we may have to cut a new beta and then repoint stage0 against that newer beta? cc `@dpaoliello` `@wesleywiser` r? `@wesleywiser` (or compiler) [^timing]: Note that it's still RustWeek this week, so most team members are N/A.
Revert "Fix linking statics on Arm64EC #140176" This reverts PR #140176. Unfortunately, this will reopen rust-lang/rust#138541 (re-breaking the `arm64ec-pc-windows-msvc` target). Unfortunately, multiple people are [reporting linker warnings related to `__rust_no_alloc_shim_is_unstable`](rust-lang/rust#140176 (comment)) after this change in `x86_64-pc-windows-msvc` as well. The solution isn't quite clear yet, let's revert to avoid the linker warnings on the Tier 1 MSVC target for now[^timing], and try a reland with a determined solution for `__rust_no_alloc_shim_is_unstable`. Judging from [people reporting that they are observing this also when bootstrapping w/ stage0 rustc](rust-lang/rust#140176 (comment)), we may have to cut a new beta and then repoint stage0 against that newer beta? cc `@dpaoliello` `@wesleywiser` r? `@wesleywiser` (or compiler) [^timing]: Note that it's still RustWeek this week, so most team members are N/A.
Unfortunately, multiple people are reporting linker warnings related to `__rust_no_alloc_shim_is_unstable` after this change. The solution isn't quite clear yet, let's revert to green for now, and try a reland with a determined solution for `__rust_no_alloc_shim_is_unstable`. This reverts commit c8b7f32, reversing changes made to 667247d. (cherry picked from commit 734a5b1)
[beta] backports and stage0 bump - bump stage0 - Update the edition guide for let chains rust-lang#140852 - Fix download of GCC from CI on non-nightly channels rust-lang#140901 - Revert "Fix linking statics on Arm64EC rust-lang#140176" rust-lang#141024 - [win][arm64] Remove 'Arm64 Hazard' undocumented MSVC option and instead disable problematic test rust-lang#141045 - Do not call name() on rpitit assoc_item rust-lang#141308 r? cuviper
[beta] backports and stage0 bump - bump stage0 to 1.87.0 - Update the edition guide for let chains #140852 - Fix download of GCC from CI on non-nightly channels #140901 - Revert "Fix linking statics on Arm64EC #140176" #141024 - [win][arm64] Remove 'Arm64 Hazard' undocumented MSVC option and instead disable problematic test #141045 - Do not call name() on rpitit assoc_item #141308 - Temporarily use Windows Server 2022 instead of Windows Server 2025 images #141023 r? cuviper
[beta] backports and stage0 bump - bump stage0 to 1.87.0 - Update the edition guide for let chains #140852 - Fix download of GCC from CI on non-nightly channels #140901 - Revert "Fix linking statics on Arm64EC #140176" #141024 - [win][arm64] Remove 'Arm64 Hazard' undocumented MSVC option and instead disable problematic test #141045 - Do not call name() on rpitit assoc_item #141308 - Temporarily use Windows Server 2022 instead of Windows Server 2025 images #141023 - Use Docker cache from the current repository #141280 - Move dist-x86_64-linux CI job to GitHub temporarily #141388 - ci: prepare aws access keys for migration #141389 r? cuviper
[beta] backports and stage0 bump - bump stage0 to 1.87.0 - Update the edition guide for let chains #140852 - Fix download of GCC from CI on non-nightly channels #140901 - Revert "Fix linking statics on Arm64EC #140176" #141024 - [win][arm64] Remove 'Arm64 Hazard' undocumented MSVC option and instead disable problematic test #141045 - Do not call name() on rpitit assoc_item #141308 - Temporarily use Windows Server 2022 instead of Windows Server 2025 images #141023 - Use Docker cache from the current repository #141280 - Move dist-x86_64-linux CI job to GitHub temporarily #141388 - ci: prepare aws access keys for migration #141389 - Add bors environment to CI #141323 r? cuviper
[beta] backports and stage0 bump - bump stage0 to 1.87.0 - Update the edition guide for let chains #140852 - Fix download of GCC from CI on non-nightly channels #140901 - Revert "Fix linking statics on Arm64EC #140176" #141024 - [win][arm64] Remove 'Arm64 Hazard' undocumented MSVC option and instead disable problematic test #141045 - Do not call name() on rpitit assoc_item #141308 - Temporarily use Windows Server 2022 instead of Windows Server 2025 images #141023 - Use Docker cache from the current repository #141280 - Move dist-x86_64-linux CI job to GitHub temporarily #141388 - ci: prepare aws access keys for migration #141389 - Add bors environment to CI #141323 - ci: split dist-arm-linux job #141078 r? cuviper
To include beta backport of revert rust-lang#141024 which should undo linker warnings during bootstrapping of Windows MSVC targets due to rust-lang#140176.
To include beta backport of revert <rust-lang#141024> which should undo linker warnings during bootstrapping of Windows MSVC targets due to <rust-lang#140176>.
…troalbini Bump master `stage0` compiler To include beta backport of revert rust-lang#141024 which should undo linker warnings during bootstrapping of Windows MSVC targets due to rust-lang#140176. Closes rust-lang#141395. r? `@Mark-Simulacrum` (or release)
Bump master `stage0` compiler To include beta backport of revert rust-lang/rust#141024 which should undo linker warnings during bootstrapping of Windows MSVC targets due to rust-lang/rust#140176. Closes rust-lang/rust#141395. r? `@Mark-Simulacrum` (or release)
Change __rust_no_alloc_shim_is_unstable to be a function This fixes a long sequence of issues: 1. A customer reported that building for Arm64EC was broken: rust-lang#138541 2. This was caused by a bug in my original implementation of Arm64EC support, namely that only functions on Arm64EC need to be decorated with `#` but Rust was decorating statics as well. 3. Once I corrected Rust to only decorate functions, I started linking failures where the linker couldn't find statics exported by dylib dependencies. This was caused by the compiler not marking exported statics in the generated DEF file with `DATA`, thus they were being exported as functions not data. 4. Once I corrected the way that the DEF files were being emitted, the linker started failing saying that it couldn't find `__rust_no_alloc_shim_is_unstable`. This is because the MSVC linker requires the declarations of statics imported from other dylibs to be marked with `dllimport` (whereas it will happily link to functions imported from other dylibs whether they are marked `dllimport` or not). 5. I then made a change to ensure that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport`, but the MSVC linker started emitting warnings that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport` but was declared in an obj file. This is a harmless warning which is a performance hint: anything that's marked `dllimport` must be indirected via an `__imp` symbol so I added a linker arg in the target to suppress the warning. 6. A customer then reported a similar warning when using `lld-link` (<rust-lang#140176 (comment)>). I don't think it was an implementation difference between the two linkers but rather that, depending on the obj that the declaration versus uses of `__rust_no_alloc_shim_is_unstable` landed in we would get different warnings, so I suppressed that warning as well: rust-lang#140954. 7. Another customer reported that they weren't using the Rust compiler to invoke the linker, thus these warnings were breaking their build: <rust-lang#140176 (comment)>. At that point, my original change was reverted (rust-lang#141024) leaving Arm64EC broken yet again. Taking a step back, a lot of these linker issues arise from the fact that `__rust_no_alloc_shim_is_unstable` is marked as `extern "Rust"` in the standard library and, therefore, assumed to be a foreign item from a different crate BUT the Rust compiler may choose to generate it either in the current crate, some other crate that will be statically linked in OR some other crate that will by dynamically imported. Worse yet, it is impossible while building a given crate to know if `__rust_no_alloc_shim_is_unstable` will statically linked or dynamically imported: it might be that one of its dependent crates is the one with an allocator kind set and thus that crate (which is compiled later) will decide depending if it has any dylib dependencies or not to import `__rust_no_alloc_shim_is_unstable` or generate it. Thus, there is no way to know if the declaration of `__rust_no_alloc_shim_is_unstable` should be marked with `dllimport` or not. There is a simple fix for all this: there is no reason `__rust_no_alloc_shim_is_unstable` must be a static. It needs to be some symbol that must be linked in; thus, it could easily be a function instead. As a function, there is no need to mark it as `dllimport` when dynamically imported which avoids the entire mess above. There may be a perf hit for changing the `volatile load` to be a `tail call`, so I'm happy to change that part back (although I question what the codegen of a `volatile load` would look like, and if the backend is going to try to use load-acquire semantics). Build with this change applied BEFORE rust-lang#140176 was reverted to demonstrate that there are no linking issues with either MSVC or MinGW: <https://github.com/rust-lang/rust/actions/runs/15078657205> Incidentally, I fixed `tests/run-make/no-alloc-shim` to work with MSVC as I needed it to be able to test locally (FYI for rust-lang#128602) r? `@bjorn3` cc `@jieyouxu`
Change __rust_no_alloc_shim_is_unstable to be a function This fixes a long sequence of issues: 1. A customer reported that building for Arm64EC was broken: rust-lang#138541 2. This was caused by a bug in my original implementation of Arm64EC support, namely that only functions on Arm64EC need to be decorated with `#` but Rust was decorating statics as well. 3. Once I corrected Rust to only decorate functions, I started linking failures where the linker couldn't find statics exported by dylib dependencies. This was caused by the compiler not marking exported statics in the generated DEF file with `DATA`, thus they were being exported as functions not data. 4. Once I corrected the way that the DEF files were being emitted, the linker started failing saying that it couldn't find `__rust_no_alloc_shim_is_unstable`. This is because the MSVC linker requires the declarations of statics imported from other dylibs to be marked with `dllimport` (whereas it will happily link to functions imported from other dylibs whether they are marked `dllimport` or not). 5. I then made a change to ensure that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport`, but the MSVC linker started emitting warnings that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport` but was declared in an obj file. This is a harmless warning which is a performance hint: anything that's marked `dllimport` must be indirected via an `__imp` symbol so I added a linker arg in the target to suppress the warning. 6. A customer then reported a similar warning when using `lld-link` (<rust-lang#140176 (comment)>). I don't think it was an implementation difference between the two linkers but rather that, depending on the obj that the declaration versus uses of `__rust_no_alloc_shim_is_unstable` landed in we would get different warnings, so I suppressed that warning as well: rust-lang#140954. 7. Another customer reported that they weren't using the Rust compiler to invoke the linker, thus these warnings were breaking their build: <rust-lang#140176 (comment)>. At that point, my original change was reverted (rust-lang#141024) leaving Arm64EC broken yet again. Taking a step back, a lot of these linker issues arise from the fact that `__rust_no_alloc_shim_is_unstable` is marked as `extern "Rust"` in the standard library and, therefore, assumed to be a foreign item from a different crate BUT the Rust compiler may choose to generate it either in the current crate, some other crate that will be statically linked in OR some other crate that will by dynamically imported. Worse yet, it is impossible while building a given crate to know if `__rust_no_alloc_shim_is_unstable` will statically linked or dynamically imported: it might be that one of its dependent crates is the one with an allocator kind set and thus that crate (which is compiled later) will decide depending if it has any dylib dependencies or not to import `__rust_no_alloc_shim_is_unstable` or generate it. Thus, there is no way to know if the declaration of `__rust_no_alloc_shim_is_unstable` should be marked with `dllimport` or not. There is a simple fix for all this: there is no reason `__rust_no_alloc_shim_is_unstable` must be a static. It needs to be some symbol that must be linked in; thus, it could easily be a function instead. As a function, there is no need to mark it as `dllimport` when dynamically imported which avoids the entire mess above. There may be a perf hit for changing the `volatile load` to be a `tail call`, so I'm happy to change that part back (although I question what the codegen of a `volatile load` would look like, and if the backend is going to try to use load-acquire semantics). Build with this change applied BEFORE rust-lang#140176 was reverted to demonstrate that there are no linking issues with either MSVC or MinGW: <https://github.com/rust-lang/rust/actions/runs/15078657205> Incidentally, I fixed `tests/run-make/no-alloc-shim` to work with MSVC as I needed it to be able to test locally (FYI for rust-lang#128602) r? `@bjorn3` cc `@jieyouxu`
Change __rust_no_alloc_shim_is_unstable to be a function This fixes a long sequence of issues: 1. A customer reported that building for Arm64EC was broken: rust-lang#138541 2. This was caused by a bug in my original implementation of Arm64EC support, namely that only functions on Arm64EC need to be decorated with `#` but Rust was decorating statics as well. 3. Once I corrected Rust to only decorate functions, I started linking failures where the linker couldn't find statics exported by dylib dependencies. This was caused by the compiler not marking exported statics in the generated DEF file with `DATA`, thus they were being exported as functions not data. 4. Once I corrected the way that the DEF files were being emitted, the linker started failing saying that it couldn't find `__rust_no_alloc_shim_is_unstable`. This is because the MSVC linker requires the declarations of statics imported from other dylibs to be marked with `dllimport` (whereas it will happily link to functions imported from other dylibs whether they are marked `dllimport` or not). 5. I then made a change to ensure that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport`, but the MSVC linker started emitting warnings that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport` but was declared in an obj file. This is a harmless warning which is a performance hint: anything that's marked `dllimport` must be indirected via an `__imp` symbol so I added a linker arg in the target to suppress the warning. 6. A customer then reported a similar warning when using `lld-link` (<rust-lang#140176 (comment)>). I don't think it was an implementation difference between the two linkers but rather that, depending on the obj that the declaration versus uses of `__rust_no_alloc_shim_is_unstable` landed in we would get different warnings, so I suppressed that warning as well: rust-lang#140954. 7. Another customer reported that they weren't using the Rust compiler to invoke the linker, thus these warnings were breaking their build: <rust-lang#140176 (comment)>. At that point, my original change was reverted (rust-lang#141024) leaving Arm64EC broken yet again. Taking a step back, a lot of these linker issues arise from the fact that `__rust_no_alloc_shim_is_unstable` is marked as `extern "Rust"` in the standard library and, therefore, assumed to be a foreign item from a different crate BUT the Rust compiler may choose to generate it either in the current crate, some other crate that will be statically linked in OR some other crate that will by dynamically imported. Worse yet, it is impossible while building a given crate to know if `__rust_no_alloc_shim_is_unstable` will statically linked or dynamically imported: it might be that one of its dependent crates is the one with an allocator kind set and thus that crate (which is compiled later) will decide depending if it has any dylib dependencies or not to import `__rust_no_alloc_shim_is_unstable` or generate it. Thus, there is no way to know if the declaration of `__rust_no_alloc_shim_is_unstable` should be marked with `dllimport` or not. There is a simple fix for all this: there is no reason `__rust_no_alloc_shim_is_unstable` must be a static. It needs to be some symbol that must be linked in; thus, it could easily be a function instead. As a function, there is no need to mark it as `dllimport` when dynamically imported which avoids the entire mess above. There may be a perf hit for changing the `volatile load` to be a `tail call`, so I'm happy to change that part back (although I question what the codegen of a `volatile load` would look like, and if the backend is going to try to use load-acquire semantics). Build with this change applied BEFORE rust-lang#140176 was reverted to demonstrate that there are no linking issues with either MSVC or MinGW: <https://github.com/rust-lang/rust/actions/runs/15078657205> Incidentally, I fixed `tests/run-make/no-alloc-shim` to work with MSVC as I needed it to be able to test locally (FYI for rust-lang#128602) r? ``@bjorn3`` cc ``@jieyouxu``
Arm64EC builds recently started to fail due to the linker not finding a symbol:
It turns out that
EMPTY_PANIC
is a new static variable that was being exported then imported from the standard library, but when exporting LLVM didn't prepend the name with#
(as only functions are prefixed with this character), whereas Rust was prefixing with#
when attempting to import it.The fix is to have Rust not prefix statics with
#
when importing.Adding tests discovered another issue: we need to correctly mark static exported from dylibs with
DATA
, otherwise MSVC's linker assumes they are functions and complains that there is no exit thunk for them.CI found another bug: we only apply
DllImport
to non-local statics that aren't foreign items (i.e., in anextern
block), that is we want to useDllImport
for statics coming from other Rust crates. However,__rust_no_alloc_shim_is_unstable
is a static generated by the Rust compiler if required, but downstream crates consider it a foreign item since it is declared in anextern "Rust"
block, thus they do not applyDllImport
to it and so fails to link if it is exported by the previous crate asDATA
. The fix is to applyDllImport
to foreign items that are marked with therustc_std_internal_symbol
attribute (i.e., we assume they aren't actually foreign and will be in some Rust crate).Fixes #138541
try-job: dist-aarch64-msvc
try-job: dist-x86_64-msvc
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2