Suggestion: follow symlinks to save the macOS use-case

[golddranks]

It's common to install libraries with Homebrew on macOS. However, when linking those libraries with the pkg-config crate, the -L/usr/local/lib flag is passed to Cargo. This may cause conflicts with dynamic resolution: /usr/local/lib gets always searched first, and unrelated libraries may override system libraries causing havoc.

An example: dynamically loaded libjpeg is a transitive dependency of Diesel. However, an incompatible version of libjpeg is also commonly installed to /usr/local/lib as a dependency of some Homebrew application. A dependency of Diesel, libpq used to support finding the native library using the pkg-config crate, but since that passes -L/usr/local/lib to Cargo which breaks libjpeg, the support had to be revoked.

libpq itself now avoids passing /usr/local/lib to Cargo, and instead checks if the library is actually a symlink. In the case of Homebrew, it always is; every library resides in their own directories, like /usr/local/Cellar/jpeg/8d/lib/. Thus, passing that to Cargo doesn't cause conflicts with other libraries.

To be usable in the macOS ecosystem, it would be beneficial if the pkg-config crate would provide this symlink-following behaviour for the users.

[golddranks]

Btw. part of the problem seems to be that Homebrew is inconsistent with the link flags: some .pc files declare the /usr/local/lib directory while some directly declare the "keg" directory (=actual, single-library-exclusive install directory). I filed a GitHub issue to Homebrew that it would be good if they had a general guideline to declare the "keg" dir, since passing an exclusive directory should prevent any conflicts, and that's why they have them, but it would be super nice if pkg-config also had this stop-gap measure to improve the overall robustness of this stuff!

[alexcrichton]

This sounds like it'd almost want to be a pkg-config feature rather than a pkg-config-rs feature? How would we know what symlinks to follow?

[golddranks]

You're right in the sense that it would be ideal if pkg-config always returned the "right" directory. However, it quite simply returns what's stated in the .pc files, and in that sense, this is actually a problem of Homebrew as they provide .pc files with slightly misleading information.

What pkg-config-rs could do, is to check the library file like compilers do. It already knows the -L and -l flags pkg-config returned to it. Let's say that we're on macOS and pkg-config returned the flags -L/usr/local/lib -lpq. That would mean checking whether /usr/local/lib/libpq.dylib is a symlink, and if it is, follow it, and return updated flags, for example: -L/usr/local/Cellar/jpeg/8d/lib/ -lpq instead.

[alexcrichton]

I'd prefer to avoid adding a pseudo-linker in pkg-config-rs, though. All we get is flags, we don't actually know where libs are ourselves. Passing something more specific would require pkg-config-rs to interpret the meaning of flags and start looking in paths and such.

[golddranks]

Yeah, that is totally understandable. I just wonder, what's the next best thing to do? pkg-config as it currently is, is broken on macOS (For Homebrew users, that is. But that's the most popular package manager for macOS. It may be broken for Fink and MacPorts too, but I haven't tested.), and just trying this simple heuristic should fix the problem and make it an it-just-works experience. It's something that isn't pkg-config-rs's responsibility, as it's doing what's expected of a simple wrapper. It just seems the most straightforward way to fix things.

[alexcrichton]

What package is printing out -L/usr/local/lib? A few I've spot-checked I've got install with homebrew all print out -L/usr/local/Cellar/... which seems to me like the best solution here

[golddranks]

Okay, that's great to hear. It's specifically libpq of the postgresql package that prints out -L/usr/local/lib. I checked all of my packages, and found out that most of the packages did print out the Cellar path which is great. postgres-related things didn't, and SDL-related things didn't.

I was under an impression that a larger portion of packages would print out the problematic -L/usr/local/lib, but apparently this was false. So maybe it's possible to fix at the Homebrew side after all, case-by-case.

[alexcrichton]

Oh yeah that'd be ideal if the postgres package could be updated to print out the local version to avoid pollution of picking up other libs by accident.

[golddranks]

For reference, we are having that discussion here: Homebrew/homebrew-core#8472 (comment)