Skip to content

Add ppc64le support #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

Add ppc64le support #165

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a792964
Add ppc64le support
clnperez Mar 7, 2023
e87f5da
fix ppc64le support
ninehills Mar 8, 2023
26ab732
Merge pull request #1 from ninehills/ppc64le-support
clnperez Mar 8, 2023
8fe40af
fix ppc64le validation
ninehills Mar 9, 2023
9d5bb23
add ppc64le release
ninehills Mar 9, 2023
14f038f
Merge branch 'clnperez:ppc64le-support' into ppc64le-support
ninehills Mar 9, 2023
eeff0c1
Merge pull request #2 from ninehills/ppc64le-support
clnperez Mar 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,37 @@ jobs:
py: 'cpython-3.11'
optimizations: 'lto'

# Cross-compiles can't do PGO and require Python 3.9.
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.9'
optimizations: 'debug'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.9'
optimizations: 'noopt'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.9'
optimizations: 'lto'

- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.10'
optimizations: 'debug'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.10'
optimizations: 'noopt'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.10'
optimizations: 'lto'

- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.11'
optimizations: 'debug'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.11'
optimizations: 'noopt'
- target_triple: 'ppc64le-unknown-linux-gnu'
py: 'cpython-3.11'
optimizations: 'lto'

# We don't publish noopt builds when PGO is available.
- target_triple: 'x86_64-unknown-linux-gnu'
py: 'cpython-3.8'
Expand Down
3 changes: 3 additions & 0 deletions cpython-unix/build-cpython.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -677,6 +677,9 @@ mipsel-unknown-linux-gnu)
mips64el-unknown-linux-gnuabi64)
PYTHON_ARCH="mips64el-linux-gnuabi64"
;;
ppc64le-unknown-linux-gnu)
PYTHON_ARCH="powerpc64le-linux-gnu"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#manylinux / PEP 599, I think this should be ppc64le.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it should be as well, and i see NixOS/nixpkgs#168083 referencing PEP600 and coming to the same conclusion (setting pythonHostPlatform and therefore _PYTHON_HOST_PLATFORM to ppc64le : https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/python/cpython/default.nix#L209) . But... the cpython configure.ac that @ninehills linked does have powerpc64le- instead

I see that you're using PYTHON_ARCH to create a symlink. If we change this will this impact that at all or be fine? I was trying to poke around and see where things land as far as directory structure on my dev VM running ppc64le -- but it's currently unavailable so I just have to bother you with questions.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed PYTHON_ARCH from powerpc64le-linux-gnu to ppc64le-linux-gnu, then built failed:

Github action: powerpc64le-linux-gnu

It seems that CPython has hard-coded this value. Unless we patch the CPython source code, it's best to keep the value of PYTHON_ARCH as powerpc64le-linux-gnu.

build error message:

Traceback (most recent call last):
  File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1088, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1071, in main
    build_cpython(
  File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 803, in build_cpython
    fh.write(build_env.get_output_archive("python"))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/buildenv.py", line 116, in get_output_archive
    data = normalize_tar_archive(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/utils.py", line 338, in normalize_tar_archive
    filedata = tf.extractfile(ti)
               ^^^^^^^^^^^^^^^^^^
File "/ opt /hostedtoolcache/Python/3.11.2/x64/lib/python3.11/tarfile.py", line 2143, in extractfile
    return self.extractfile(self._find_link_target(tarinfo))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ opt /hostedtoolcache/Python/3.11.2/x64/lib/python3.11/tarfile.py", line 2452, in _find_link_target
    raise KeyError("linkname %r not found" % linkname)
KeyError: "linkname 'python/install/lib/python3.9/config-3.9d-ppc64le-linux-gnu/libpython3.9d.a' not found"
make: *** [Makefile:278: /home/runner/work/python-build-standalone/python-build-standalone/build/cpython-3.9.16-ppc64le-unknown-linux-gnu-debug.tar] Error 1
Error: Process completed with exit code 2.

;;
s390x-unknown-linux-gnu)
PYTHON_ARCH="s390x-linux-gnu"
;;
Expand Down
3 changes: 3 additions & 0 deletions cpython-unix/build-libX11.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ if [ -n "${CROSS_COMPILING}" ]; then
mips64el-unknown-linux-gnuabi64)
EXTRA_FLAGS="${EXTRA_FLAGS} --enable-malloc0returnsnull"
;;
ppc64le-unknown-linux-gnu)
EXTRA_FLAGS="${EXTRA_FLAGS} --enable-malloc0returnsnull"
;;
s390x-unknown-linux-gnu)
EXTRA_FLAGS="${EXTRA_FLAGS} --enable-malloc0returnsnull"
;;
Expand Down
2 changes: 2 additions & 0 deletions cpython-unix/build.cross.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,13 @@ RUN apt-get install \
gcc-mips-linux-gnu \
gcc-mips64el-linux-gnuabi64 \
gcc-mipsel-linux-gnu \
gcc-powerpc64le-linux-gnu \
gcc-s390x-linux-gnu \
libc6-dev-arm64-cross \
libc6-dev-armel-cross \
libc6-dev-armhf-cross \
libc6-dev-mips-cross \
libc6-dev-mips64el-cross \
libc6-dev-mipsel-cross \
libc6-dev-ppc64el-cross \
libc6-dev-s390x-cross
1 change: 1 addition & 0 deletions cpython-unix/extension-modules.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ _decimal:
- define: CONFIG_64=1
targets:
- aarch64-.*
- ppc64le-unknown-linux.*
- s390x-unknown-linux-.*
- x86_64.*
links:
Expand Down
37 changes: 37 additions & 0 deletions cpython-unix/targets.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,43 @@ mipsel-unknown-linux-gnu:
- zlib
openssl_target: linux-mips32

ppc64le-unknown-linux-gnu:
host_platforms:
- linux64
pythons_supported:
- '3.9'
- '3.10'
- '3.11'
docker_image_suffix: .cross
host_cc: /usr/bin/x86_64-linux-gnu-gcc
host_cxx: /usr/bin/x86_64-linux-gnu-g++
target_cc: /usr/bin/powerpc64le-linux-gnu-gcc
needs:
- autoconf
- bdb
- binutils
- bzip2
- expat
- libedit
- libffi
- libX11
- libXau
- libxcb
- m4
- mpdecimal
- ncurses
- openssl
- patchelf
- sqlite
- tcl
- tk
- tix
- uuid
- xorgproto
- xz
- zlib
openssl_target: linux-ppc64le

s390x-unknown-linux-gnu:
host_platforms:
- linux64
Expand Down
1 change: 1 addition & 0 deletions docs/building.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ As are various other targets::
$ ./build-linux.py --target armv7-unknown-linux-gnueabihf
$ ./build-linux.py --target mips-unknown-linux-gnu
$ ./build-linux.py --target mipsel-unknown-linux-gnu
$ ./build-linux.py --target ppc64le-unknown-linux-gnu
$ ./build-linux.py --target s390x-unknown-linux-gnu

macOS
Expand Down
9 changes: 9 additions & 0 deletions src/release.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,15 @@ pub static RELEASE_TRIPLES: Lazy<BTreeMap<&'static str, TripleRelease>> = Lazy::
},
);

h.insert(
"ppc64le-unknown-linux-gnu",
TripleRelease {
suffixes: linux_suffixes_nopgo.clone(),
install_only_suffix: "lto",
python_version_requirement: None,
},
);

h.insert(
"i686-unknown-linux-gnu",
TripleRelease {
Expand Down
8 changes: 8 additions & 0 deletions src/validation.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ const RECOGNIZED_TRIPLES: &[&str] = &[
"mips-unknown-linux-gnu",
"mipsel-unknown-linux-gnu",
"mips64el-unknown-linux-gnuabi64",
"ppc64le-unknown-linux-gnu",
"s390x-unknown-linux-gnu",
"thumbv7k-apple-watchos",
"x86_64-apple-darwin",
Expand Down Expand Up @@ -155,6 +156,10 @@ static GLIBC_MAX_VERSION_BY_TRIPLE: Lazy<HashMap<&'static str, version_compare::
"mips64el-unknown-linux-gnuabi64",
version_compare::Version::from("2.19").unwrap(),
);
versions.insert(
"ppc64le-unknown-linux-gnu",
version_compare::Version::from("2.17").unwrap(),
);
versions.insert(
"s390x-unknown-linux-gnu",
version_compare::Version::from("2.17").unwrap(),
Expand Down Expand Up @@ -212,6 +217,7 @@ static ELF_ALLOWED_LIBRARIES_BY_TRIPLE: Lazy<HashMap<&'static str, Vec<&'static
("mips-unknown-linux-gnu", vec!["ld.so.1"]),
("mipsel-unknown-linux-gnu", vec!["ld.so.1"]),
("mips64el-unknown-linux-gnuabi64", vec![]),
("ppc64le-unknown-linux-gnu", vec!["ld64.so.1"]),
("s390x-unknown-linux-gnu", vec!["ld64.so.1"]),
("x86_64-unknown-linux-gnu", vec!["ld-linux-x86-64.so.2"]),
("x86_64_v2-unknown-linux-gnu", vec!["ld-linux-x86-64.so.2"]),
Expand Down Expand Up @@ -426,6 +432,7 @@ static PLATFORM_TAG_BY_TRIPLE: Lazy<HashMap<&'static str, &'static str>> = Lazy:
("mips-unknown-linux-gnu", "linux-mips"),
("mipsel-unknown-linux-gnu", "linux-mipsel"),
("mips64el-unknown-linux-gnuabi64", "todo"),
("ppc64le-unknown-linux-gnu", "linux-powerpc64le"),
("s390x-unknown-linux-gnu", "linux-s390x"),
("x86_64-apple-darwin", "macosx-10.9-x86_64"),
("x86_64-apple-ios", "iOS-x86_64"),
Expand Down Expand Up @@ -777,6 +784,7 @@ fn validate_elf<'data, Elf: FileHeader<Endian = Endianness>>(
"mips-unknown-linux-gnu" => object::elf::EM_MIPS,
"mipsel-unknown-linux-gnu" => object::elf::EM_MIPS,
"mips64el-unknown-linux-gnuabi64" => 0,
"ppc64le-unknown-linux-gnu" => object::elf::EM_PPC64,
"s390x-unknown-linux-gnu" => object::elf::EM_S390,
"x86_64-unknown-linux-gnu" => object::elf::EM_X86_64,
"x86_64_v2-unknown-linux-gnu" => object::elf::EM_X86_64,
Expand Down