pre-compile is failing with `aenum` package

[aignas]

The part of error log:

pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/__pycache__/_py2.cpython-311.pyc

It seems that the wheel that I am using is https://files.pythonhosted.org/packages/d0/fa/ca0c66b388624ba9dbbf35aab3a9f326bfdf5e56a7237fe8f1b600da6864/aenum-3.1.15-py3-none-any.whl#sha256=e0dfaeea4c2bd362144b87377e2c61d91958c5ed0b4daf89cb6f45ae23af6288

The precompile config setting was set to enabled and I was using the latest rules_python 1.3.0rc1 on a mac host.

One of the errors:

ERROR: /private/var/tmp/_bazel_ignas.anikevicius/b76e964649685d10d1a2df959fae69ae/external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/BUILD.bazel:5:20: Python precompiling site-packages/aenum/_py2.py into external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/__pycache__/_py2.cpython-311.pyc failed: (Exit 3): python3 failed: error executing PyCompile command (from target @@rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea//:pkg) bazel-out/darwin_x86_64-opt-exec-ST-d57f47055a04/bin/external/rules_python+/python/private/python3 ... (remaining 2 arguments skipped)
Unhandled error:
Request: {'arguments': ['--invalidation_mode', 'checked_hash', '--src', 'external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/_py2.py', '--src_name', '../rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/_py2.py', '--pyc', 'bazel-out/darwin_x86_64-fastbuild-ST-5be17bb44133/bin/external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/__pycache__/_py2.cpython-311.pyc', '--optimize', '0', '--python_version', '3.11'], 'inputs': [{'path': 'external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/_py2.py', 'digest': 'NjEyMWE3MDRjZWRmZWVhNTkwNmRkNjc5NGQ4ODE1YzE3N2VhNDI0Nzc1MWIxNWJhNTQ1NzcxZWI1Y2RmNmQ2ZA=='}, {'path': 'bazel-out/darwin_x86_64-opt-exec-ST-d57f47055a04/bin/external/rules_python+/python/private/python3', 'digest': 'MDE5MTZjMzRmYTg4ZjA1MjIwNTg2NDJjZTBkYTI0YzdhZTExNjlhMTljZjIwZTlhMGE0NGIxMDAzYzY5MDVjNA=='}, {'path': 'bazel-out/darwin_x86_64-opt-exec-ST-d57f47055a04/bin/external/rules_python+/tools/precompiler/precompiler', 'digest': 'MzUyYWNhMDkzMjljOTVmY2UxMTQzOTI2NmM0YzhiODgwMmU4YTcyNDYxOWFjMzk0Y2M3MGZjODIxYjhlY2FhNg=='}, {'path': 'bazel-out/darwin_x86_64-fastbuild-ST-5be17bb44133/bin/external/rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/__pycache__/_py2.cpython-311.pyc-0.params', 'digest': 'YjlkM2VkZTUwNDAwZTEyMjUzZDQyYmUyYTAwNWE2ZTk3MzUyZjQ3YzJiNmVjODk3M2JmNmI5MTY4NDRmYmI4NQ=='}], 'requestId': 3315}
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_ignas.anikevicius/b76e964649685d10d1a2df959fae69ae/external/rules_python++python+python_3_11_x86_64-apple-darwin/lib/python3.11/py_compile.py", line 144, in compile
    code = loader.source_to_code(source_bytes, dfile or file,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 1004, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "../rules_python++pip+pypi_311_aenum_py3_none_any_e0dfaeea/site-packages/aenum/_py2.py", line 5
    raise exc, None, tb
             ^
SyntaxError: invalid syntax

And the contents of this file are:

from inspect import getargspec

def raise_with_traceback(exc, tb):
    raise exc, None, tb

__all__ = ['_div_', 'getargspec', 'raise_with_traceback']

I understand that I can patch the wheel to exclude the file, but I am curious what the user is expected to do in this case.

[rickeylev]

This is a tough one. It's tempting to dismiss this because it's py2, but new syntax does occasionally get introduced; a conditional import isn't an unreasonable way to handle it.

Normally one would use a select() to omit the the file from srcs, but since this is coming from a pypi package, I don't think that'll be doable.

The first idea that comes to mind is to have precompiling, when applied to srcs from pypi-generated stuff, ignore errors. Bazel requires the action to produce the outputs, though. Maybe write an empty pyc, or one with a -1 timestamp or bogus hash?

A directory output is the only other way I can think of to have an optional output. Maybe, for a pypi package, produce all the pyc's in one invocation with a directory output?

The repo-phase could sniff the files and inject the select() condition.

[aignas]

I think we may need to also be able to exclude certain third party packages from being pre-compiled. Something like precompile-ignore=[]. Maybe this could be passed in the pip.override?

[rickeylev]

passed in via pip.override

+1. I generally like the idea of pip.override to customize the pypi generation stuff on a per-package basis from the outside.