Skip to content

manylinux_2_24: Use gcc-9 & binutils 2.36.1 #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

mayeut
Copy link
Contributor

@mayeut mayeut commented Apr 10, 2021

c.f. #54 & pypa/manylinux#1012 (comment)

Use gcc-9 from ubuntu-toolchain-r/test PPA:
A gcc toolchain recent enough is required to build OpenBLAS aarch64 code.
Use "official" backport of gcc 9.3.0 toolchain for Xenial instead of default 6.3.0 toolchain on Debian 9.
No symbol conflicts observed for OpenBLAS & its dependencies when using this toolchain.

Use binutils 2.36.1 rebuilt from source:
A recent enough assembler is needed to build OpenBLAS code.
Replaces the 2.28 binutils on Debian 9

@mattip
Copy link
Collaborator

mattip commented Apr 19, 2021

There is still a problem on ppc64, dynamic_power.c:206:6: error: cpu ‘power10’ is an invalid argument to builtin ‘__builtin_cpu_is’. See PR OpenMathLib/OpenBLAS#2994 which caused the problem, and issue OpenMathLib/OpenBLAS#3074 which found it. This was fixed in b94dab525 for OpenBLAS. We have OpenBLAS pinned to an earlier commit af2b0d0. So maybe try modifying the .travis.yml and .github/workflows/*.yml files to use the later changeset.

@mattip mattip changed the title manylinux_2_24 gcc-9 manylinux_2_24 gcc-10 Apr 20, 2021
@mattip
Copy link
Collaborator

mattip commented Apr 20, 2021

All checks passed 🎉. Do you want to remove the "Draft" status?

@mayeut
Copy link
Contributor Author

mayeut commented Apr 20, 2021

@mattip,

Sorry but I kind of hi-jacked this PR to see what could be done with custom PPA...
The answer is PPA are probably the way to go to get newer tools in manylinux_2_24 image. However, the quality of the "http://ppa.launchpad.net/mayeut-github/test*" PPA used in this PR is not good at all (I merely tried stuff to get builds passing without much knowledge regarding what I was actually changing in the package source...). I'll try to sum-up what I tried in pypa/manylinux#1012 and hopefully, someone with knowledge of debian packaging can take over providing a PPA with at least binutils. If the gcc-10 can also be patched with compatibility for stdc++ like devtoolset is on RH, then it would certainly be interesting to include this in the stock manylinux images.

I do feel comfortable using the gcc-9 PPA from the debian/ubuntu folks and rebuilding binutils from source as was once tried in this PR. As you noted, it will require bumping OpenBLAS to get past the power10 issue on ppc64le. I will update the PR this way.

@mattip
Copy link
Collaborator

mattip commented Apr 20, 2021

I will update the PR this way.

Thanks!

@mayeut mayeut force-pushed the manylinux224-gcc9 branch from 0d1f526 to 2b103ce Compare April 20, 2021 16:52
mayeut added 2 commits April 24, 2021 08:51
Use gcc-9 from ubuntu-toolchain-r/test PPA:
A gcc toolchain recent enough is required to build OpenBLAS aarch64 code.
Use "official" backport of gcc 9.3.0 toolchain for Xenial instead of default 6.3.0 toolchain on Debian 9.
No symbol conflicts observed for OpenBLAS & its dependencies when using this toolchain.

Use binutils 2.36.1 rebuilt from source:
A recent enough assembler is needed to build OpenBLAS code.
Replaces the 2.28 binutils on Debian 9
Fix build issue with gcc-9 on ppc64le
@mayeut mayeut force-pushed the manylinux224-gcc9 branch from bd4f9b6 to bd2abcf Compare April 24, 2021 06:58
@mayeut mayeut changed the title manylinux_2_24 gcc-10 manylinux_2_24: Use gcc-9 & binutils 2.36.1 Apr 24, 2021
@mayeut mayeut marked this pull request as ready for review April 24, 2021 07:50
@mayeut
Copy link
Contributor Author

mayeut commented Apr 24, 2021

I cleaned this up a bit.
There are 2 errored jobs on ppc64le. I'm confident they'll pass given the outcome of the other jobs.
Seems GitHub changed the way Actions are triggered and thus those workflows weren't triggered with the last update.

@tylerjereddy
Copy link
Collaborator

Seems GitHub changed the way Actions are triggered

I think it is a protection against cryptocurrency mining (seriously). Anyway, I approved the CI to run--I think this only happens first time for new contributor.

@mattip
Copy link
Collaborator

mattip commented Apr 30, 2021

I think this should be good to go. I am not sure what to do about the g++ compatibility issue, it will be a blocker for actually releasing manylinux2_24 wheels, since SciPy, and soon NumPy, use c++ code.

@mattip
Copy link
Collaborator

mattip commented Mar 16, 2022

Closing. the g++ issue prevents manylinux2_24 wheels from being viable.

@mattip mattip closed this Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants