Provide a public API

[agronholm]

Several third party libraries broke when the internal API of wheel was changed as the result of a major refactoring effort. Since there is clearly a need for wheel to function as a library as well, a public API should be defined and documented.

Here's what I plan to include in the API:

• The WheelFile class

[matthew-brett]

Just a request to put something in the WheelFile class or other API that will:

• read and write the information in the WHEEL package info - currently read_pkg_info and write_pkg_info
• report the tags with which a particular wheel is compatible - e.g. https://github.com/pypa/wheel/blob/0.31.1/wheel/install.py#L122
• read and write the RECORD file (read / write CSV as tuples or something).

Thanks for considering.

[wimglenn]

PyPA define a wheel class in at least 6 different places (!)

• here in pip
• here in wheel
• here in distlib
• here in setuptools
• here in twine
• here in installer

To make matters worse, pip also imports wheel and vendors distlib + setuptools, so it gets 4 different implementations at once. I think there would be great benefit to pypa/wheel providing a public API for other projects to use, and reduce this fragmentation. It's important core infrastructure for packaging and should have some public surface area.

[agronholm]

To comment @matthew-brett 's list:

• read and write the information in the WHEEL package info - currently read_pkg_info and write_pkg_info
  • this sounds like a better fit for a more generic packaging library as it is not wheel specific
• report the tags with which a particular wheel is compatible - e.g. https://github.com/pypa/wheel/blob/0.31.1/wheel/install.py#L122
  • doable, although I would still prefer a shared pep425 implementation if possible
• read and write the RECORD file (read / write CSV as tuples or something).
  • not specific to wheel, so maybe should be put in a more generic packaging library too?

[matthew-brett]

Is the idea then, to have two copies of the code, one in wheel and another in a generic packaging library? Or would wheel depend on this library? Is it really worth building a new generic packaging library for this, given that the code is pretty small?

[pradyunsg]

Is the plan to essentially document the WheelFile as it sits today as a supported API?

If so, I have one major-ish concern with that - it has a baked-in assumption about the wheel format always being a flat zip. It's likely not going to be possible to transparently add support for newer versions of the wheel format, which don't operate on ".data files as members of same zip". There's been a lot of discussion about nested archives for wheels' .data directory, which allows for smaller overall wheel files on https://discuss.python.org/t/3810/ and we shouldn't lock ourselves out of that enhancement due to this API.

Obviously, we can't support that kind of wheel format just yet because those aren't even close to being fully specified (!). However, I think it'd be really nice to have an API that can accommodate for such changes in a wheel 2.0 format, without needing significant user-side code change. It'd be nice to be able to gain support for reading from newer versions of wheel files by upgrading wheel (the package), without needing to update the code that handles the wheel files using the API.

In other words, I think we should avoid baking in the assumption in a supported API that wheel files' .data directory will be files directly in the same zip. It's OK if we don't do it now, but we'd be creating more design work, for when a wheel 2.0 format is specified.

---

I'd like it if we clearly from separate them from the supported API (so that pip can "easily" trim those files when vendoring). It'd be even nicer to have a separate library with the reusable code, so that pip doesn't have to do any trimming.

---

In terms of how pip uses wheel (the package) today, pip tries to import it to check if it is even installed, and that's all. :)

Beyond that, there's 2 main ways we deal with wheel files:

• unpacking an existing wheel file (installation)
• creating a wheel file, from a bunch of info we have at runtime (in tests)

I don't expect/want wheel to have any installation logic exposed (that's what github.com/pradyunsg/installer is for!).

In terms of "reading" a wheel, I think having the capability to perform random access on dist_info files, and only having sequential access to data files is all that's needed/can be guaranteed if we wanna be a bit more future-proof.

In terms of "writing" a wheel, I don't have a feel for what the API should look like - pip's use case is "dumb". We have a helper function that takes a dictionary of file_name: contents with the assumption that the contents is always a UTF-8 string. That's not OK for anything that is doing something more relevant to actual users (eg: dealing with large data files, or compiled code).

(typed on my phone, with typo happy thumbs)

[agronholm]

If so, I have one major-ish concern with that - it has a baked-in assumption about the wheel format always being a flat zip.

I was going to refactor WheelFile so it doesn't inherit from ZipFile, to minimize the API surface.

I'd like it if we clearly from separate them from the supported API (so that pip can "easily" trim those files when vendoring). It'd be even nicer to have a separate library with the reusable code, so that pip doesn't have to do any trimming.

I suggested back on discuss that setuptools could absorb the bdist_wheel command which would leave wheel as just a reference implementation and a command line tool. Would this work for the pip maintainers?