[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2193: pyOpenSSL 0.14 pulls in a bunch of new dependencies

[Tahoe-LAFS]

#2193: pyOpenSSL 0.14 pulls in a bunch of new dependencies
-------------------------+-------------------------------------------------
     Reporter:  daira    |      Owner:
         Type:  defect   |     Status:  new
     Priority:  normal   |  Milestone:  undecided
    Component:           |    Version:  1.10.0
  packaging              |   Keywords:  packaging setuptools pyopenssl
   Resolution:           |  cryptography six cffi pycparser
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by glyph):

 Replying to [comment:10 daira]:
 > The actual mechanics of how to make Tahoe tolerate the dependency are
 not so much the issue here.

 I strongly disagree.  The mechanics of how to make Tahoe tolerate the
 dependency are the ''only'' issue here.  If you want to talk about how
 pyOpenSSL should revert to being a giant pile of ad-hoc C code rather than
 a tiny wrapper around some statically-verified and [https://travis-
 ci.org/pyca/cryptography well-tested] wrappers for the OpenSSL API, the
 right place for that discussion would be
 [https://github.com/pyca/pyopenssl/issues the pyOpenSSL tracker], not
 here.  (However, I can tell you that you're not likely to meet with much
 success with that proposal.)

 pyOpenSSL implements a thing, and the ''hope'' here is that the way in
 which it implements this is abstracted and its clients, like Tahoe, should
 not need to care how it does it.  Having dependencies is one way it might
 choose to implement this.  It may acquire new dependencies in the future
 (as might Twisted, for that matter, for reasons having nothing to do with
 cryptography).  It's deeply unfortunate that this addition broke things
 for Tahoe, and hopefully that won't happen in the future, but "your
 dependencies are part of your API contract and you can't ever add new
 ones" is not an acceptable constraint for Twisted; nor, I suspect, for any
 other thing that Tahoe depends on.  We should work out how to do this non-
 destructively in the future.  (Note that I believe the Cryptography 0.3
 release resolves the installation issue at least for Windows users because
 it fixed the thing that would cause binary wheels to need to be rebuilt on
 the installer's system if a new release of cffi came out.)

 > I'm annoyed and frustrated by pyOpenSSL adding a dependency on yet
 another fucking cryptography library. It's going in entirely the wrong
 direction.

 However, since you seem frustrated let me try to address that.  It might
 be helpful to read [https://cryptography.io/en/latest/#why-a-new-crypto-
 library-for-python the Cryptography team's rationale for creating
 something new] rather than trying to maintain one of the existing options,
 but that's pretty brief.

 First of all, Cryptography is not ''really'' "yet another ... cryptography
 library".  My understanding is that it implements exactly ''no''
 cryptographic primitives itself.  Instead, its mission is to be a
 comprehensive wrapper around and common interface to multiple back-ends
 which allow for using your platform's existing cryptography engine,
 allowing you to access things like hardware support and platform security
 updates without updating all of your application's code.  For example,
 it's not just a wrapper around OpenSSL; it also wraps
 [https://github.com/pyca/cryptography/tree/master/cryptography/hazmat/bindings/commoncrypto
 the CommonCrypto API from OS X's security framework].

 (Please be aware that this is my interpretation and I'm not a Cryptography
 core developer.)

 In light of this substantially improved approach, pyOpenSSL is really just
 a light wrapper around one of these layers.  It would be very premature to
 say pyOpenSSL is "deprecated" – lots of existing software uses it, it'll
 likely be around for a very long time, and Cryptography doesn't implement
 nearly enough high-level functionality to replace it – but as Cryptography
 implements higher level functionality I think there's no real reason why
 one would rely on the pyOpenSSL version of that functionality rather than
 Cryptography's.  If you use Cryptography, it will be doing your crypto
 functionality in a way that is multi-backend and doesn't, for example,
 require you to compile OpenSSL on Windows; instead you could just use
 existing Windows APIs for these algorithms.

 For Twisted, all this stuff is a godsend.  Instead of having some stdlib
 stuff and some pyOpenSSL stuff and some PyCrypto stuff (with known
 exploits) we can (eventually) just use one library to provide all of our
 security features.

 Replying to [comment:11 daira]:

 > Note that the only overlap between what cryptography.io provides, and
 what Tahoe-LAFS needs (and can't get from the Python standard lib) is AES
 and RSA. So we can't use it to remove any of our other crypto
 dependencies.

 It's pretty early days for Cryptography.  This is why I said "at some
 point in the future".  They only added RSA like a couple of weeks ago, and
 the only high-level (i.e. non-"hazmat") thing implemented so far is
 Fernet.  So there's a huge swath of functionality they're going to be
 adding already.  If some of the Tahoe developers were to talk to the
 Cryptography folks, it might be easy to convince them to add all of your
 needs to the library in some form.

 Since Tahoe uses more novel cryptographic primitives than many
 applications, it's possible that the Cryptography developers won't be
 interested in adding everything you need, but I really think it would be
 worthwhile to try; given the effort that those developers are investing in
 fixing their build and deployment story, it might make issues like this a
 lot ''easier'' to deal with in the future.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2193#comment:12>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage