How to link C extension correctly

[dtamayo]

Our package REBOUNDx builds a C extension, which needs to link to a C extension in another package REBOUND (without the x!).

How do I configure my pyproject.toml file to do this correctly on different operating systems? In particular, we would strongly prefer for the wheel to not package the REBOUND C extension dependency, but rather have it locate it on the target machine so that different users could have different combinations of REBOUND and REBOUNDx versions that aren't tied to each other.

Is this a misunderstanding of how wheels work? Is the libreboundx.so shared library already pre-built in the wheel (so that there is no hope of setting the correct user-specific paths at wheel installation time)?

I tried using --exclude librebound.so in the wheel repair step, which built the wheel successfully, but when I try to install from it and import reboundx, I get

OSError: dlopen(/Users/dtamayo/opt/miniconda3/envs/pypi/lib/python3.11/site-packages/reboundx/../[libreboundx.cpython-311-darwin.so](http://libreboundx.cpython-311-darwin.so/), 0x0006): symbol not found in flat namespace (_reb_create_ode)

In setup.py, we deal with this by importing rebound and getting the path to where it's installed in a couple ways

class build_ext(_build_ext):
    def finalize_options(self):
        ...
        import rebound
        rebdir = os.path.dirname(inspect.getfile(rebound))
        rebdirsp = sysconfig.get_path('platlib')+'/'

then append these to self.library_dirs, and ext.runtime_library_dirs, extra_link_args for ext in self.extensions.

Any help/pointers would be most appreciated!

[joerick]

It seems like the symbols from rebound haven't been loaded yet. Does it work if when you test you do import rebound (or some import that loads the so module) before import reboundx?

[dtamayo]

Unfortunately not @joerick. Here's what I don't understand conceptually:

In the regular setup.py build, when we build libreboundx.so, we need to set all the link paths and rpaths correctly to where the user has their installation of the librebound.so library. When we build a wheel, are we building the C extension on our own local computer and then packaging that into the wheel? If so, it seems like wheels will not work for this type of project, since we can't know what user-specific paths to set at the time we build the wheel.

Are there any hooks available that lets us write python code that runs at the end of installing the wheel (that we could use to find where librebound.so is installed and set the appropriate paths at the end?). Or is the only option to also package librebound.so inside our wheel for reboundx?

[Czaki]

Wheels is the format for unpack and use. If you need to execute code on installation, you need to use sdist.

If you would like to use wheels you need to ship librebound.so or create a separate specialized package whose purpose is only to determine the path to librebound.so during installation time and export it in a form readable by the main package.

[dtamayo]

Thanks @Czaki, I think that clears everything up. Really appreciate both your help!