Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install script does not use pg_config's LIBDIR for runtime linking #1356

Open
gtzampanakis opened this issue Sep 22, 2021 · 3 comments · May be fixed by #1357
Open

Install script does not use pg_config's LIBDIR for runtime linking #1356

gtzampanakis opened this issue Sep 22, 2021 · 3 comments · May be fixed by #1357

Comments

@gtzampanakis
Copy link

The install script does not pass pg_config's LDFLAGS to the linker. In my case pg_config outputs the following LDFLAGS:

LDFLAGS = -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-12/lib',--enable-new-dtags

But the psycopg2 install script ignores everything that doesn't start with "-L", therefore it ignores the rpath configuration. Then, when psycopg2 is loaded by a uwsgi executable that has its own rpath the process exits with a "missing name" error because the fact that the executable has its own rpath prevents it from loading the psycopg2 library even though ldconfig has run and has properly configured the system.

To work around this I do

LDFLAGS='-L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,/usr/pgsql-12/lib,--enable-new-dtags' pip install psycopg2

and then psycopg2 seems to work fine.

Please consider loading pg_config's LDFLAGS and passing them to the linker without user intervention.

@dvarrazzo
Copy link
Member

Have you got a MR to suggest?

gtzampanakis pushed a commit to gtzampanakis/psycopg2 that referenced this issue Sep 22, 2021
@gtzampanakis gtzampanakis linked a pull request Sep 22, 2021 that will close this issue
@gtzampanakis
Copy link
Author

See #1357. My original suggestion to pass pg_config's LDFLAGS to the linker was not correct because those flags are the ones that were used to build postgres so it doesn't follow that they should be used to build psycopg2 too. My solution in #1357 is to take just the LIBDIR from pg_config (which psycopg2's setup already used for compile-time linking) and use it for runtime linking.

@gtzampanakis gtzampanakis changed the title Install script does not pass pg_config's LDFLAGS to the linker Install script does not use pg_config's LIBDIR for runtime linking Sep 22, 2021
gtzampanakis pushed a commit to gtzampanakis/psycopg2 that referenced this issue Sep 22, 2021
@zyv
Copy link

zyv commented Sep 25, 2022

My original suggestion to pass pg_config's LDFLAGS to the linker was not correct because those flags are the ones that were used to build postgres so it doesn't follow that they should be used to build psycopg2 too.

So, the real problem is that some "smarter" linking systems like libtool would use list of library directories to generate LDFLAGS for the linker in such a way, that if the library is already on the default linker path, then this path will be omitted, and if it is not, then it will be passed along with the -rpath flag to the linker, like -Wl,-rpath,'/usr/pgsql-12/lib', so that this path will be remembered in the shared module and used later at the load time.

For better or worse, distutils does not do those libtool dances, so what happens here is that the linking is successful, but loading at runtime fails, because in your case Postgres libdir is not on the loader path for whatever reason (broken installation, broken package, genius plan gone south, ...).

A more "correct" solution as compared to #1357, where you are adding libdir to rpath unconditionally on Linux, would be to actually replicate some of the libtool logic and write platform-specific code to check whether the libdir where libpq resides (or even all library_dirs — how about other dependencies of pscycopg?) is on the default loader path, and if not, then to add those paths that aren't to self.rpath. This would be platform-specific, e.g. on Linux something like the following would work to obtain the loader paths:

$ ldconfig -v 2>/dev/null | grep -v ^$'\t' | cut -d ":" -f 1
/usr/lib64//bind9-export
/usr/lib64/mysql
/lib
/lib64
/lib/sse2
/lib64/sse2
/lib64/tls

Your system is configured in a way that Postgres is not on the loader path, and if you don't like it, then you should fix your system by adding it, or else, if you have reasons why you configured it this way, then you have to live with the consequences of this choice by either supplying LDFLAGS=-Wl,-rpath,'/usr/pgsql-12/lib' at the compile time or adding libdir to LD_LIBRARY_PATH when you run the software depending on libpq.

Anyway, I would not complicate the world to support obscure edge cases like this with a lot of effort, or by adding libdir to rpath for everyone, which is bound to introduce a whole set of different problems and security issues... especially because if such systems are really configured this way for a reason, the best idea maybe would be to not interfere with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants