Cleaner install for Python library on Linux

[alice]

Right now, we need to do an extra step to tell ld.so where to find the .so objects when loading the Python library.

Some cursory research suggests that it might be possible to install the Python library such that this step is no longer necessary.

[alice]

Working notes:

The StackOverflow post links to this section in the SWIG manual: https://www.swig.org/Doc4.2/SWIGDocumentation.html#Python_nn6

The preferred approach to building an extension module for python is to compile it with distutils, which comes with all recent versions of python (Distutils Docs).

... A configuration file (conventionally called: setup.py) describes the extension (and related python modules). The distutils will then generate all the right compiler directives to build it for you.

There's an example setup.py which imports setup and Extension from distutils.core, and then configures an Extension object pointing at example_wrap.c and example.c - is example_wrap.c generated by SWIG? Ah, yes, the next paragraph notes "example_wrap.c, generated by swig, and example.c, your original c source." I guess for us it would be ... all (??) of the source files, plus the generated _wrap.cxx.

It then calls setup(), passing in the module defined in the Extension object, and some string metadata.

The distutils page now says that it's deprecated, and points to setuptools instead, which looks like quite a similar (identical?) API.

Getting back to the StackOverflow post, it notes that distutils (and hopefully setuptools too...) can handle .i files directly: https://docs.python.org/2/distutils/setupscript.html#extension-source-files

setuptools note only "Source files may be C, C++, SWIG (.i)", but hopefully that means the same guidance applies.

setup(...,
      ext_modules=[Extension('_foo', ['foo.i'],
                             swig_opts=['-modern', '-I../include'])],
      py_modules=['foo'],
     )

setuptools also has swig_opts, so that should work.

The StackOverflow post then links out to an example the author came up with: https://github.com/danielunderwood/swig-example

That has a library and a "main" which uses the library (I think in our case we only have the "main" library?), a .i file which just %includes the "main" .h file, and then a setup.py.in which is configure_filed in the CMakeLists.txt file to point to the correct filenames to pass to the filenames argument for setuptools' install function.

Their setup.py seems to only do the install part, rather than actually building the library; the building part looks quite similar to what we do. And the install part just copies the files??