[tahoe-dev] that problem with comparing types across DLLs

[Zooko O'Whielacronx]

(Note that several individuals, plus tahoe-dev at allmydata.org and
cplusplus-sig at python.org are Cc:'ed.  This is appropriate because this
is fundamentally a "cross project" issue and we need to talk to each
other until we understand what the other developers are planning.
Unfortunately, you're likely to have problems following-up to the
mailing lists if you aren't subscribed to them.)

On May 27, 2009, at 1:01 AM, Matthias Klose wrote:
>
> Well, maybe nobody did ask, because it's policy for distributions to build
> against shared libs? If there needs something to be fixed in the library
> package, you only have to take care of the library packages, not all packages
> which are linked against the static library (of course you would have to find
> out which these packages are). Plus you usually cannot link a python extension
> against a static lib because the static lib is built without -fPIC.

Thanks, Matthias, and thanks to Ruben for the earlier answers about Fedora.

It looks like option #1 from this list is right out:

http://mail.python.org/pipermail/cplusplus-sig/2009-May/014531.html

(Option #1 was to persuade distributions to support Crypto++ as a
static library.)

The option that would be technically the most general would be to
figure out how to annotate and compile Crypto++ as a shared lib that
has visible only the symbols that it needs to have visible, and then
use the RTLD_GLOBAL flag to dlopen() to make sure multiple .so's which
each depend on libcryptopp.so will get those symbols unified at
load-time so that cross-shared-library RTTI will work.  That is what
Niall Douglas recommends and has done successfully in the past.

However, I've already spent a couple of days trying to learn how to do
this and I haven't succeeded yet.  I'm not even sure that it is
possible.  Wei Dai, the author of Crypto++, has also tried in the past
to build such a shared lib of Crypto++ for Unix in the past, and he
too gave up.  That also means that he does not currently support using
Crypto++ as a shared library, which makes me uncomfortable, but I
guess that's the Linux distributions' problem more than mine.

I think that leaves option #2 from that list: refactor pycryptopp so
that it contains only a single shared library, and use RTLD_GLOBAL to
unify the symbols that are visible in the _pycryptopp.so with the
symbols in the the libcryptopp.so.  I think this is doable and that it
will solve all known problems for pycryptopp and Tahoe.  I think this
does mean that if in the future there is a different Python module
than pycryptopp which is dynamically linked to libcryptopp.so, that
anyone who imports both pycryptopp and that other Python module into
their Python interpreter will get a seg fault.  However, I suppose
this too is someone else's problem than mine.

So unless anyone else has a better suggestion, I'm going to proceed
with strategy #2.  I'll probably also continue to support the current
linking strategy of linking pycryptopp statically against
libcryptopp.a, as a build-time option for pycryptopp.  Therefore
people who have chosen the static-link-to-Crypto++ build-time option
(which excludes users of the Fedora, Debian, or Ubuntu packages of
pycryptopp) will be able to also import another Python module which
links to Crypto++.

Thanks, folks!

Regards,

Zooko