Add support for maccatalyst #43

asavva-2016 · 2025-03-08T08:29:49Z

This is to add Mac Catalyst support to the cpython-apple-source-deps.

Added a new target MacCatalyst.

PR Checklist:

All new features have been tested
All new features have been documented
I have read the CONTRIBUTING.md file
I will abide by the code of conduct

Add maccatalyst support

…talyst

freakboy3742

Thanks for the PR!

The core of this PR looks really good; I've left a couple of broad comments inline about the specifics. Those comments are mostly related to the format of the additional patches; the one big issue is how the MacCatalyst build is configured.

My immediate inclination is that even thought the compiler triple used by MacCatalyst is arm64-apple-ios-macabi, we might not want to handle this strictly as an "iOS" build. A separate TARGETS-MacCatalyst et al would give much more flexibility to define catalyst-specific flags (which seems necessary if only for the VERSION_MIN). It also gives the flexibility to build only the MacCatalyst artefacts (or not build them if you're working on iOS).

Three final notes:

The ideal end state is that we won't need patches at all. As the libFFI patch is a substantial change in behavior, it should be submitted upstream to libFFI for inclusion there.
mpdecimal and xz are a little harder, as the actual patch is to config.sub, which is part of autoconf, and the autoconf maintainers have been very resistant to adding the -simulator part of the patch. I've tried multiple times, and had no response - not even a "acknowledgement of receipt". To that end, our current approach is to keep the patch to the bare minimum needed on top of the version of config.sub in the project. I'd encourage you to try to submit the patch (although even the submission process is... baroque), but don't be surprised if it goes nowhere.
I've just added a CI workflow to this repo so that changes in the build can be easily evaluated. To test that this PR actually works, you'll need to merge that update into this PR, and add the extra configuration to build MacCatalyst as part of the build matrix.

patch/mpdecimal.patch

patch/xz.patch

Makefile

freakboy3742

I think this mostly makes sense; a couple of minor patch cleanups flagged, and one request for more details about the specifics of macCatalyst itself.

patch/mpdecimal.patch

freakboy3742 · 2025-03-10T05:49:23Z

patch/libffi.patch

+		;;
+ 	*-eabi*- | *-gnueabi*-)
+		;;
+	ios*-macabi- )


From a consistency point of view - why not include this as part of the the simulator group on L16?

Relocate to L16

... it has been? I'm not seeing any change here. Have you pushed this update?

Updated now

freakboy3742 · 2025-03-10T05:50:50Z

patch/libffi.patch

+
+ class Platform(object):
+-    pass
+    abi = None


Based on the usage in this PR - what's the difference between the sdk and the abi? Why can't MacCatalyst builds be referenced as sdk="macabi"?

I posted a comment above

... where is this comment? I'm not sure I follow what you're referring to.

The macosx is the sdk and the macabi instructs the compiler to use the Mac Catalyst ABI. It allows the iOS(iPad) version of the app to run on Mac. The Mac Catalyst version only supports the iOS APIs (no access to the MacOSX APIs) and is therefore closely aligned with iOS and for all intents and purposes behaves as such.

The purpose of Mac Catalyst is to allow taking an iPad app and using all the iPad apis in a Mac app.

I understand what Mac Catalyst is conceptually - I'm asking why it's necessary to separate sdk and abi in this configuration. What functional purpose is met by adding an entirely new configuration flag? It's a flag that is only used in the macabi situation, and is literally blank everywhere else. It's mutually exclusive with sdk="iphoneos". You're not making any other use of abi as a logic differentiator... so why make the distinction? Or, if you do want to keep the two separate - why isn't that separation followed through to the rest of the SDK/ABI handling (i.e., every platform has an SDK and an ABI - so why not define them an exploit that to simplify other logic)?

patch/mpdecimal.patch

freakboy3742 · 2025-03-10T05:51:47Z

patch/xz.patch

 	none--*)
 		# None (no kernel, i.e. freestanding / bare metal),
+ 		# can be paired with an machine code file format


Again - same weird addition...

These appear to be contextual diff lines. I can clean them up manually but it was just generated through diff -Naur on the original vs patched version of the directories.

Update the patch in line with the above

Makefile

freakboy3742

Looks like you may have missed pushing some updates - your comments are referencing changes that don't appear to have been made.

Also - this still requires updates to the CI configuration to generate catalyst builds.

freakboy3742 · 2025-03-12T23:44:57Z

patch/libffi.patch

+
+ class Platform(object):
+-    pass
+    abi = None


I understand what Mac Catalyst is conceptually - I'm asking why it's necessary to separate sdk and abi in this configuration. What functional purpose is met by adding an entirely new configuration flag? It's a flag that is only used in the macabi situation, and is literally blank everywhere else. It's mutually exclusive with sdk="iphoneos". You're not making any other use of abi as a logic differentiator... so why make the distinction? Or, if you do want to keep the two separate - why isn't that separation followed through to the rest of the SDK/ABI handling (i.e., every platform has an SDK and an ABI - so why not define them an exploit that to simplify other logic)?

freakboy3742 · 2025-03-12T23:46:09Z

.github/workflows/release.yaml

@@ -16,7 +16,7 @@ jobs:
      BUILD_NUMBER: ${{ steps.build-vars.outputs.BUILD_NUMBER }}
    strategy:
      matrix:
-        target: [ "iOS", "tvOS", "watchOS" ]
+        target: [ "iOS", "tvOS", "watchOS", "MacCatalyst" ]


This is the release workflow - it doesn't address the CI workflow that I added a couple of days ago to make it possible to test this PR. You may need to merge with main to get that update.

freakboy3742 · 2025-03-16T01:19:31Z

@asavva-2016 Can I ask why you've closed this PR (and the related PRs)? It felt like it was on the right track and getting fairly close to being merged - there were just a few details that needed to be clarified.

asavva-2016 · 2025-03-16T09:13:22Z

@freakboy3742 I was doing this in my spare time and it was just consuming too much of my time.

asavva-2016 · 2025-03-16T09:19:44Z

I'm asking why it's necessary to separate sdk and abi in this configuration. What functional purpose is met by adding an entirely new configuration flag? It's a flag that is only used in the macabi situation, and is literally blank everywhere else. It's mutually exclusive with sdk="iphoneos". You're not making any other use of abi as a logic differentiator... so why make the distinction? Or, if you do want to keep the two separate - why isn't that separation followed through to the rest of the SDK/ABI handling (i.e., every platform has an SDK and an ABI - so why not define them an exploit that to simplify other logic)?

Re: this @freakboy3742. It is required for Mac Catalyst builds because the SDK is macosx and in this case we need to pass through the --build flag here:

-                [] if platform.sdk == "macosx" else [f"--build={os.uname().machine}-apple-darwin"]
+                [] if platform.sdk == "macosx" and platform.abi != "macabi" else [f"--build={os.uname().machine}-apple-darwin"]

Without the abi flag the --build flag isn't passed through and it doesn't compile for the correct target.

Maybe there was a different way to effect that change but that was the simplest that I could determine in my time constraints.

In case I wasn't clear elsewhere the Mac Catalyst build uses the macosx SDK with a different ABI. Therefore this check for the macosx SDK is not sufficient anymore.

freakboy3742 · 2025-03-16T22:34:53Z

@freakboy3742 I was doing this in my spare time and it was just consuming too much of my time.

Totally understood. Thanks for the time you did spend; at the very least, this would be a good starting point if anyone else wants to pick up the development effort where you've left off.

For the sake of posterity (and anyone else that might pick up where you left off):

I'm asking why it's necessary to separate sdk and abi in this configuration. What functional purpose is met by adding an entirely new configuration flag?

Re: this @freakboy3742. It is required for Mac Catalyst builds because the SDK is macosx and in this case we need to pass through the --build flag here:
...
In case I wasn't clear elsewhere the Mac Catalyst build uses the macosx SDK with a different ABI. Therefore this check for the macosx SDK is not sufficient anymore.

Yes... but you don't need two flags to differentiate MacCatalyst from macOS. If you know the ABI is macabi, you also know the SDK is macosx. You can unambiguously determine one from the other. Yes, this means changes in logic in various places, but all the information you need is available.

johnzhou721 · 2025-04-12T20:26:15Z

Can't we just do this with macOS regularly, and since the last time I checked, without an XCFramework, dynamic libraries built for macOS could be linked against Mac Catalyst, just link it?

Or is there some API that CPython or any of those projects uses that are available in Mac Catalyst but not macOS?

freakboy3742 · 2025-04-13T01:19:46Z

@johnzhou721 I haven't done a deep dive to confirm if there's any APIs in use that would be a problem - so it may well be possible to use macOS libraries without special handling. However, an explicitly Catalyst-linked build will always be safer as you know that they're safe from an API usage perspective.

johnzhou721 · 2025-04-13T03:00:50Z

About half a year ago I tried to do this, there was an issues with a header file, but after I tweak it a bit it was okay.

Also upstream report:
python/cpython#98297

johnzhou721 · 2025-04-13T03:02:33Z

In addition should I try to revive this?

freakboy3742 · 2025-04-13T03:22:08Z

In addition should I try to revive this?

I'll gladly merge a PR adding macCatalyst support if someone revives it. It might even make sense to roll this into the #50, given the changes will be almost exact parallels of each other.

johnzhou721 · 2025-04-13T03:27:27Z

Hi! I just opened beeware/Python-Apple-support#269 which can't even initialize the interpreter at the moment. Must have tons of bugs. If you have time, would you please take a look? Thank you!

Let me fix stuff in #50 now

asavva-2016 added 9 commits January 2, 2025 10:40

Add support for MacCatalyst support

1ad87d0

Rename to maccatalyst

b914c33

Fix for the libFFI build on maccatalyst

6d57ba7

Merge pull request #1 from asavva-2016/add-maccatalyst-support

bf3fe44

Add maccatalyst support

Merge remote-tracking branch 'upstream/main'

653c6df

Tidy up the Mac Catalyst support

29648d4

Tidy up the Mac Catalyst support

821324e

Tidy up the Mac Catalyst support

7f4b45b

Merge remote-tracking branch 'origin/main' into add-support-for-macca…

2019023

…talyst

freakboy3742 requested changes Mar 9, 2025

View reviewed changes

patch/mpdecimal.patch Outdated Show resolved Hide resolved

patch/xz.patch Outdated Show resolved Hide resolved

patch/xz.patch Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

asavva-2016 added 5 commits March 9, 2025 17:38

Add MacCatalyst target

5a757de

Fix the libffi patch, revert incidental changes

ff2c07c

Revert mpdecimal patches

a05125c

Revert the xz.patch

84b0045

Minumal xz patch

d5a17e5

freakboy3742 requested changes Mar 10, 2025

View reviewed changes

Fix the xz and libffi patches

ba3c97b

freakboy3742 requested changes Mar 11, 2025

View reviewed changes

asavva-2016 added 2 commits March 12, 2025 18:50

Update the libffi patch

ad0f881

Add MacCatalyst to the workflow

1a7a51c

freakboy3742 requested changes Mar 12, 2025

View reviewed changes

asavva-2016 closed this by deleting the head repository Mar 15, 2025

freakboy3742 mentioned this pull request Mar 16, 2025

Catalyst Support beeware/Python-Apple-support#117

Open

johnzhou721 mentioned this pull request Apr 23, 2025

Support for Mac Catalyst #55

Closed

4 tasks

Uh oh!

Add support for maccatalyst #43

Add support for maccatalyst #43

Uh oh!

Conversation

asavva-2016 commented Mar 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Checklist:

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freakboy3742 commented Mar 16, 2025

Uh oh!

asavva-2016 commented Mar 16, 2025

Uh oh!

asavva-2016 commented Mar 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

freakboy3742 commented Mar 16, 2025

Uh oh!

johnzhou721 commented Apr 12, 2025

Uh oh!

freakboy3742 commented Apr 13, 2025

Uh oh!

johnzhou721 commented Apr 13, 2025

Uh oh!

johnzhou721 commented Apr 13, 2025

Uh oh!

freakboy3742 commented Apr 13, 2025

Uh oh!

johnzhou721 commented Apr 13, 2025

Uh oh!

Uh oh!

asavva-2016 commented Mar 8, 2025 •

edited

Loading

asavva-2016 commented Mar 16, 2025 •

edited

Loading