-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Add AVIF to wheels using only aomenc and dav1d AVIF codecs for reduced size #8858
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
8b4e66e
a2cbce7
84485e1
ab55e54
8a6e5af
deb1c9d
6d7f396
7aca46c
8bc14e2
621bcd5
1c40ea5
331354a
8cac43f
ce01f19
8ab93db
a005e28
6c918a0
5573242
8a174f5
f40eed2
8905480
f20ce67
5a8313c
a3cba6a
01daa3d
3ffce33
7607474
2222237
1cabcba
c993fbc
7f44ca6
124c1a5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -51,6 +51,7 @@ LIBWEBP_VERSION=1.5.0 | |
BZIP2_VERSION=1.0.8 | ||
LIBXCB_VERSION=1.17.0 | ||
BROTLI_VERSION=1.1.0 | ||
LIBAVIF_VERSION=1.3.0 | ||
|
||
function build_pkg_config { | ||
if [ -e pkg-config-stamp ]; then return; fi | ||
|
@@ -98,6 +99,59 @@ function build_harfbuzz { | |
touch harfbuzz-stamp | ||
} | ||
|
||
function build_libavif { | ||
if [ -e libavif-stamp ]; then return; fi | ||
|
||
python3 -m pip install meson ninja | ||
|
||
if [[ "$PLAT" == "x86_64" ]] || [ -n "$SANITIZER" ]; then | ||
build_simple nasm 2.16.03 https://www.nasm.us/pub/nasm/releasebuilds/2.16.03 | ||
fi | ||
|
||
local build_type=MinSizeRel | ||
local lto=ON | ||
|
||
local libavif_cmake_flags | ||
|
||
if [ -n "$IS_MACOS" ]; then | ||
lto=OFF | ||
libavif_cmake_flags=( | ||
-DCMAKE_C_FLAGS_MINSIZEREL="-Oz -DNDEBUG -flto" \ | ||
-DCMAKE_CXX_FLAGS_MINSIZEREL="-Oz -DNDEBUG -flto" \ | ||
-DCMAKE_SHARED_LINKER_FLAGS_INIT="-Wl,-S,-x,-dead_strip_dylibs" \ | ||
) | ||
else | ||
if [[ "$MB_ML_VER" == 2014 ]] && [[ "$PLAT" == "x86_64" ]]; then | ||
build_type=Release | ||
fi | ||
libavif_cmake_flags=(-DCMAKE_SHARED_LINKER_FLAGS_INIT="-Wl,--strip-all,-z,relro,-z,now") | ||
fi | ||
|
||
local out_dir=$(fetch_unpack https://github.com/AOMediaCodec/libavif/archive/refs/tags/v$LIBAVIF_VERSION.tar.gz libavif-$LIBAVIF_VERSION.tar.gz) | ||
# CONFIG_AV1_HIGHBITDEPTH=0 is a flag for libaom (included as a subproject | ||
# of libavif) that disables support for encoding high bit depth images. | ||
(cd $out_dir \ | ||
&& cmake \ | ||
-DCMAKE_INSTALL_PREFIX=$BUILD_PREFIX \ | ||
-DCMAKE_INSTALL_LIBDIR=$BUILD_PREFIX/lib \ | ||
-DCMAKE_INSTALL_NAME_DIR=$BUILD_PREFIX/lib \ | ||
-DBUILD_SHARED_LIBS=ON \ | ||
-DAVIF_LIBSHARPYUV=LOCAL \ | ||
-DAVIF_LIBYUV=LOCAL \ | ||
-DAVIF_CODEC_AOM=LOCAL \ | ||
-DCONFIG_AV1_HIGHBITDEPTH=0 \ | ||
-DAVIF_CODEC_AOM_DECODE=OFF \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Frankie: Is it possible to build with the libaom and dav1d packages in Linux distributions (i.e., There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The shared libraries linked from the pillow extension are all included in the distributed binary, so that people installing Pillow don't need to install the libraries themselves. This is why libjpeg-turbo, libpng, libtiff, and other libraries are included in the wheel files on PyPI. Nothing would be gained by using libaom from the linux distribution. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If someone were to build Pillow from source, they could use libavif from the distro, or build libavif against system libraries. The binary wheels are tagged for python version, system architecture, and ABI compatibility (musl and two different versions of glibc). They are intended to be installed in any compatible environment without any other dependencies. |
||
-DAVIF_CODEC_DAV1D=LOCAL \ | ||
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=$lto \ | ||
-DCMAKE_C_VISIBILITY_PRESET=hidden \ | ||
-DCMAKE_CXX_VISIBILITY_PRESET=hidden \ | ||
-DCMAKE_BUILD_TYPE=$build_type \ | ||
"${libavif_cmake_flags[@]}" \ | ||
. \ | ||
&& make install) | ||
touch libavif-stamp | ||
} | ||
|
||
function build { | ||
build_xz | ||
if [ -z "$IS_ALPINE" ] && [ -z "$SANITIZER" ] && [ -z "$IS_MACOS" ]; then | ||
|
@@ -132,6 +186,7 @@ function build { | |
build_tiff | ||
fi | ||
|
||
build_libavif | ||
build_libpng | ||
build_lcms2 | ||
build_openjpeg | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
Copyright (c) 2016, Alliance for Open Media. All rights reserved. | ||
|
||
Redistribution and use in source and binary forms, with or without | ||
modification, are permitted provided that the following conditions | ||
are met: | ||
|
||
1. Redistributions of source code must retain the above copyright | ||
notice, this list of conditions and the following disclaimer. | ||
|
||
2. Redistributions in binary form must reproduce the above copyright | ||
notice, this list of conditions and the following disclaimer in | ||
the documentation and/or other materials provided with the | ||
distribution. | ||
|
||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS | ||
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE | ||
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, | ||
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, | ||
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | ||
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN | ||
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE | ||
POSSIBILITY OF SUCH DAMAGE. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
Copyright © 2018-2019, VideoLAN and dav1d authors | ||
All rights reserved. | ||
|
||
Redistribution and use in source and binary forms, with or without | ||
modification, are permitted provided that the following conditions are met: | ||
|
||
1. Redistributions of source code must retain the above copyright notice, this | ||
list of conditions and the following disclaimer. | ||
|
||
2. Redistributions in binary form must reproduce the above copyright notice, | ||
this list of conditions and the following disclaimer in the documentation | ||
and/or other materials provided with the distribution. | ||
|
||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND | ||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | ||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | ||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR | ||
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | ||
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | ||
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | ||
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | ||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you mean to change
BUILD_SHARED_LIBS
toON
? It is stillOFF
in winbuild/build_prepare.py.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did. Using the linker to generate the shared library file allows it to remove unused objects. There is a mechanism for including shared library dependencies of a python extension inside a wheel file on macOS and linux (delocate and auditwheel, respectively) but there is no such standard mechanism for doing the same on windows. So the windows library has to be built statically.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The savings are modest, about 0.5 MB, but there isn't any cost to switching to a shared library.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When you say 0.5mb, I presume that's per wheel. Which wheel(s) were you using to check that?