我编写了一个依赖于某些C扩展的Python模块。这些C扩展又依赖于几个已编译的C库。我希望能够分发与所有依赖项捆绑在一起的该模块。
我整理了一个最小的示例（可以在GitHub上找到全部示例）。
目录结构为：
$ tree . . ├── README.md ├── poc │ ├── __init__.py │ ├── cython_extensions │ │ ├── __init__.py │ │ ├── cvRoberts_dns.c │ │ ├── cvRoberts_dns.h │ │ ├── helloworld.c │ │ ├── helloworld.pxd │ │ ├── helloworld.pyx │ │ ├── test.c │ │ └── test.h │ ├── do_stuff.c │ └── do_stuff.pyx └── setup.py
setup.py构建扩展，和链接对必要的库（libsundials_cvode，libsundials_nvectorserial在这种情况下）：
libsundials_cvode
libsundials_nvectorserial
from setuptools import setup, find_packages from setuptools.extension import Extension from Cython.Build import cythonize ext_module_dostuff = Extension( 'poc.do_stuff', ['poc/do_stuff.pyx'], ) ext_module_helloworld = Extension( 'poc.cython_extensions.helloworld', ['poc/cython_extensions/helloworld.pyx', 'poc/cython_extensions/test.c', 'poc/cython_extensions/cvRoberts_dns.c'], include_dirs = ['/usr/local/include'], libraries = ['m', 'sundials_cvodes', 'sundials_nvecserial'], library_dirs = ['/usr/local/lib'], ) cython_ext_modules = [ ext_module_dostuff, ext_module_helloworld ] setup ( name = "poc", ext_modules = cythonize(cython_ext_modules), packages=['poc', 'poc.cython_extensions'], )
一切都很好，但确实需要最终用户先安装sundials（在实际情况下，还需要安装其他几个非常难以启动和运行的库）。
理想情况下，我希望只能在开发计算机上进行设置，创建包含适当共享库的发行版，并提供某种捆绑包。
到目前为止，我已经找到了各种教程，示例和SO帖子。我被引导相信自己在正确的道路上。但是，还有一些最后的步骤，我只是没有刻板。
任何帮助表示赞赏：-）。
您可能知道，建议使用分发格式来分发带有已编译组件的Python模块。似乎没有任何标准的跨平台方式将第三方本机库捆绑到一个轮子中。但是，有用于此目的的特定于平台的工具。
auditwheel
auditwheel修改现有的Linux wheel文件以添加基本的“ manylinux ”标准中未包含的任何第三方库。以下是在全新安装的Ubuntu 17.10上如何将其与项目一起使用的演练：
首先，安装基本的Python开发工具，以及带有其标头的第三方库：
root@ubuntu-17:~# apt-get install cython python-pip unzip root@ubuntu-17:~# apt-get install libsundials-serial-dev
然后将您的项目构建到wheel文件中：
root@ubuntu-17:~# cd cython-example/ root@ubuntu-17:~/cython-example# python setup.py bdist_wheel [...] root@ubuntu-17:~/cython-example# cd dist/ root@ubuntu-17:~/cython-example/dist# ll total 80 drwxr-xr-x 2 root root 4096 Nov 8 11:28 ./ drwxr-xr-x 7 root root 4096 Nov 8 11:28 ../ -rw-r--r-- 1 root root 70135 Nov 8 11:28 poc-0.0.0-cp27-cp27mu-linux_x86_64.whl root@ubuntu-17:~/cython-example/dist# unzip -l poc-0.0.0-cp27-cp27mu-linux_x86_64.whl Archive: poc-0.0.0-cp27-cp27mu-linux_x86_64.whl Length Date Time Name --------- ---------- ----- ---- 62440 2017-11-08 11:28 poc/do_stuff.so 2 2017-11-08 11:28 poc/__init__.py 116648 2017-11-08 11:28 poc/cython_extensions/helloworld.so 2 2017-11-08 11:28 poc/cython_extensions/__init__.py 10 2017-11-08 11:28 poc-0.0.0.dist-info/DESCRIPTION.rst 211 2017-11-08 11:28 poc-0.0.0.dist-info/metadata.json 4 2017-11-08 11:28 poc-0.0.0.dist-info/top_level.txt 105 2017-11-08 11:28 poc-0.0.0.dist-info/WHEEL 167 2017-11-08 11:28 poc-0.0.0.dist-info/METADATA 793 2017-11-08 11:28 poc-0.0.0.dist-info/RECORD --------- ------- 180382 10 files
wheel文件现在可以在本地安装并经过测试：
root@ubuntu-17:~/cython-example/dist# pip install poc-0.0.0-cp27-cp27mu-linux_x86_64.whl [...] root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()" hello cython 0.841470984808 trying to load the sundials program 3-species kinetics problem At t = 2.6391e-01 y = 9.899653e-01 3.470564e-05 1.000000e-02 rootsfound[] = 0 1 At t = 4.0000e-01 y = 9.851641e-01 3.386242e-05 1.480205e-02 [...]
现在我们安装该auditwheel工具。它需要Python 3，但是能够处理Python 2或3的轮子。
root@ubuntu-17:~/cython-example/dist# apt-get install python3-pip root@ubuntu-17:~/cython-example/dist# pip3 install auditwheel
auditwheel使用另一个称为“patchelf工作”的工具。不幸的是，patchelfUbuntu 17.10随附的版本缺少一个错误修正，没有该修正，auditwheel将无法运行。因此，我们必须从源代码构建脚本（脚本取自manylinux Docker映像）：
patchelf
root@ubuntu-17:~# apt-get install autoconf root@ubuntu-17:~# PATCHELF_VERSION=6bfcafbba8d89e44f9ac9582493b4f27d9d8c369 root@ubuntu-17:~# curl -sL -o patchelf.tar.gz https://github.com/NixOS/patchelf/archive/$PATCHELF_VERSION.tar.gz root@ubuntu-17:~# tar -xzf patchelf.tar.gz root@ubuntu-17:~# (cd patchelf-$PATCHELF_VERSION && ./bootstrap.sh && ./configure && make && make install)
现在，我们可以检查方向盘需要哪些第三方库：
root@ubuntu-17:~/cython-example/dist# auditwheel show poc-0.0.0-cp27-cp27mu-linux_x86_64.whl poc-0.0.0-cp27-cp27mu-linux_x86_64.whl is consistent with the following platform tag: "linux_x86_64". The wheel references external versioned symbols in these system- provided shared libraries: libc.so.6 with versions {'GLIBC_2.4', 'GLIBC_2.2.5', 'GLIBC_2.3.4'} The following external shared libraries are required by the wheel: { "libblas.so.3": "/usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1", "libc.so.6": "/lib/x86_64-linux-gnu/libc-2.26.so", "libgcc_s.so.1": "/lib/x86_64-linux-gnu/libgcc_s.so.1", "libgfortran.so.4": "/usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0", "liblapack.so.3": "/usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1", "libm.so.6": "/lib/x86_64-linux-gnu/libm-2.26.so", "libpthread.so.0": "/lib/x86_64-linux-gnu/libpthread-2.26.so", "libquadmath.so.0": "/usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0", "libsundials_cvodes.so.2": "/usr/lib/libsundials_cvodes.so.2.0.0", "libsundials_nvecserial.so.0": "/usr/lib/libsundials_nvecserial.so.0.0.2" } In order to achieve the tag platform tag "manylinux1_x86_64" the following shared library dependencies will need to be eliminated: libblas.so.3, libgfortran.so.4, liblapack.so.3, libquadmath.so.0, libsundials_cvodes.so.2, libsundials_nvecserial.so.0
并创建一个捆绑它们的新轮子：
root@ubuntu-17:~/cython-example/dist# auditwheel repair poc-0.0.0-cp27-cp27mu-linux_x86_64.whl Repairing poc-0.0.0-cp27-cp27mu-linux_x86_64.whl Grafting: /usr/lib/libsundials_nvecserial.so.0.0.2 -> poc/.libs/libsundials_nvecserial-42b4120e.so.0.0.2 Grafting: /usr/lib/libsundials_cvodes.so.2.0.0 -> poc/.libs/libsundials_cvodes-50fde5ee.so.2.0.0 Grafting: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 -> poc/.libs/liblapack-549933c4.so.3.7.1 Grafting: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 -> poc/.libs/libblas-52fa99c8.so.3.7.1 Grafting: /usr/lib/x86_64-linux-gnu/libgfortran.so.4.0.0 -> poc/.libs/libgfortran-2df4b07d.so.4.0.0 Grafting: /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 -> poc/.libs/libquadmath-0d7c3070.so.0.0.0 Setting RPATH: poc/cython_extensions/helloworld.so to "$ORIGIN/../.libs" Previous filename tags: linux_x86_64 New filename tags: manylinux1_x86_64 Previous WHEEL info tags: cp27-cp27mu-linux_x86_64 New WHEEL info tags: cp27-cp27mu-manylinux1_x86_64 Fixed-up wheel written to /root/cython-example/dist/wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl root@ubuntu-17:~/cython-example/dist# unzip -l wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl Archive: wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl Length Date Time Name --------- ---------- ----- ---- 167 2017-11-08 11:28 poc-0.0.0.dist-info/METADATA 4 2017-11-08 11:28 poc-0.0.0.dist-info/top_level.txt 10 2017-11-08 11:28 poc-0.0.0.dist-info/DESCRIPTION.rst 211 2017-11-08 11:28 poc-0.0.0.dist-info/metadata.json 1400 2017-11-08 12:08 poc-0.0.0.dist-info/RECORD 110 2017-11-08 12:08 poc-0.0.0.dist-info/WHEEL 62440 2017-11-08 11:28 poc/do_stuff.so 2 2017-11-08 11:28 poc/__init__.py 131712 2017-11-08 12:08 poc/cython_extensions/helloworld.so 2 2017-11-08 11:28 poc/cython_extensions/__init__.py 230744 2017-11-08 12:08 poc/.libs/libsundials_cvodes-50fde5ee.so.2.0.0 7005072 2017-11-08 12:08 poc/.libs/liblapack-549933c4.so.3.7.1 264024 2017-11-08 12:08 poc/.libs/libquadmath-0d7c3070.so.0.0.0 2039960 2017-11-08 12:08 poc/.libs/libgfortran-2df4b07d.so.4.0.0 17736 2017-11-08 12:08 poc/.libs/libsundials_nvecserial-42b4120e.so.0.0.2 452432 2017-11-08 12:08 poc/.libs/libblas-52fa99c8.so.3.7.1 --------- ------- 10206026 16 files
如果我们卸载第三方库，则先前安装的滚轮将停止工作：
root@ubuntu-17:~/cython-example/dist# apt-get remove libsundials-serial-dev && apt-get autoremove [...] root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()" Traceback (most recent call last): File "<string>", line 1, in <module> File "poc/do_stuff.pyx", line 1, in init poc.do_stuff ImportError: libsundials_cvodes.so.2: cannot open shared object file: No such file or directory
但是带有捆绑库的轮子可以正常工作：
root@ubuntu-17:~/cython-example/dist# pip uninstall poc [...] root@ubuntu-17:~/cython-example/dist# pip install wheelhouse/poc-0.0.0-cp27-cp27mu-manylinux1_x86_64.whl [...] root@ubuntu-17:~/cython-example/dist# python -c "from poc.do_stuff import hello; hello()" hello cython 0.841470984808 trying to load the sundials program 3-species kinetics problem At t = 2.6391e-01 y = 9.899653e-01 3.470564e-05 1.000000e-02 rootsfound[] = 0 1 At t = 4.0000e-01 y = 9.851641e-01 3.386242e-05 1.480205e-02 [...]
delocate
delocateOSX的工作方式显然与auditwheel。不幸的是，我没有可用于提供演练的OSX机器。
使用这两种工具的一个项目是SciPy。尽管它的名称，此存储库包含所有平台（不仅仅是Mac）的官方SciPy构建过程。具体来说，将Linux构建脚本（使用auditwheel）和OSX构建脚本（使用delocate）进行比较。
要查看此过程的结果，您可能需要从PyPI下载并解压缩一些SciPy轮子。例如，scipy-1.0.0-cp27-cp27m-manylinux1_x86_64.whl包含以下内容：
scipy-1.0.0-cp27-cp27m-manylinux1_x86_64.whl
38513408 2017-10-25 06:02 scipy/.libs/libopenblasp-r0-39a31c03.2.18.so 1023960 2017-10-25 06:02 scipy/.libs/libgfortran-ed201abd.so.3.0.0
虽然scipy-1.0.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl包含以下内容：
scipy-1.0.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
273072 2017-10-25 07:03 scipy/.dylibs/libgcc_s.1.dylib 1550456 2017-10-25 07:03 scipy/.dylibs/libgfortran.3.dylib 279932 2017-10-25 07:03 scipy/.dylibs/libquadmath.0.dylib
