[tahoe-dev] [ANN] Foolscap-0.2.6 released

[Brian Warner]

I've just finished releasing Foolscap-0.2.6, the next-generation remote
object protocol, intended as the successor to Perspective Broker.

This release contains more logging improvements, including the long-awaited
"incident-based logging" system (which dumps "black box" records of recent
events when something bad happens), and several bugfixes. The full release
notes are attached below.

Tarballs can be obtained from foolscap.lothar.com and from PyPI. All releases
are signed by my GPG public key: 0x1514A7BD .

 http://pypi.python.org/packages/source/f/foolscap/foolscap-0.2.6.tar.gz#md5=2cfb1e033cff29e13df1beb8befe7c08
 http://pypi.python.org/packages/source/f/foolscap/foolscap-0.2.6.zip#md5=1b2f6038d3016b4fa5c71e6d69f1bac1


For documentation, bugs, and patches, please visit
http://foolscap.lothar.com/trac .

share and enjoy,
 -Brian


* Release 0.2.6 (06 May 2008)

** Compatibility

All releases between 0.1.3 and 0.2.6 (inclusive) are fully wire-compatible.

The saved logfiles produced (by e.g. 'flogtool tail --save-to' and the
log-gatherer) in 0.2.6 and beyond are not readable by tools (e.g. 'flogtool
dump' and 'flogtool filter') from 0.2.5 and earlier.

FURLs which contain "extensions" (described below) will not be tolerated by
foolscap 0.2.5 or earlier. If, at some point in the future, we add such
extensions to published FURLs, then such an application will require
foolscap-0.2.6 or later to interpret those FURLs.

** Logging Changes

*** "Incident" Logging

This release finally implements the "strangeness-triggered logging" espoused
in doc/logging.xhtml . By giving the foolscap logging code a directory to
work with, the logging system will automatically save a compressed pickled
logfile containing the messages that lead up to sufficiently-severe log
event. The docs explain how to control what "sufficiently-severe" means.
These events are retrievable through the logport, although no tools have been
written yet to actually extract them. They are also retrievable by using
'flogtool dump' directly on the incident files.

*** 'flogtool as a subcommand

The implementation of the 'flogtool' executable has been rearranged to make
it possible to add a 'flogtool' subcommand to some other executable.

*** 'flogtool filter' now has --above LEVEL and --from TUBID options
*** 'flogtool dump' has --rx-time option, also shows failure tracebacks
*** gatherer: don't add erroneous UTC-implying "Z" suffix to filename timestamps
*** 'flogtool tail': don't add spurious "0" to timestamps

** constraints no longer reject ('reference',) sequences

The foolscap/banana serialization protocol responds to sending two separate
copies of the same object in the same callRemote message by emitting one
serialized object sequence and one 'reference' sequence: this is the standard
way by which cycles are broken in the serialized data. Unfortunately, the
wire-level constraint checkers in 0.2.5 and earlier would reject reference
sequences with a Violation exception: if they were expecting a tuple, they
would reject anything else, even a reference sequence that pointed at a
tuple.

Worse yet, python's normal constant-object folding code can produce shared
references where you might not expect. In the following example, the two
tuples are identical objects (and result in a 'reference' sequence on the
wire), despite the programmer having typed them separately:

 rref.callRemote("haveTwoTuples", (0,1), (0,1) )

Foolscap-0.2.6 now allows reference sequence in all wire-level constraint
checks, to avoid this false-negative Violation. The post-deserialization
check will still enforce the constraint properly. It just does it late enough
to be able to tell what the reference points to.

** Twisted/pyopenssl compatibility

*** compatible with Twisted-8.0.x

Some unit test failures under Twisted-8.0.x (the most recent release) were
fixed: tests now pass against Twisted-8.0.x, and a buildbot is used to make
sure compatibility is maintained in the future.

*** incompatible with pyOpenSSL-0.7

An incompatibility has been discovered with the most recent version of
PyOpenSSL. pyopenssl 0.6 works correctly, but pyopenssl 0.7 causes modern
versions of Twisted (both 2.5.x and 8.0.x) to follow a code path that breaks
the Foolscap unit tests. This may or may not cause a problem in actual
application use (the symptom is that the non-winning parallel connections are
not disconnected properly, and several negotiation timers are left running).
Until a fix is developed for either Twisted or PyOpenSSL, the recommended
workaround is to downgrade to PyOpenSSL-0.6 . Twisted bug #3218 and Foolscap
bug #62 exist to track this problem.

** setup.py is more setuptools-friendly

The foolscap version string is located with a regular expression rather than
an import, allowing setuptools to install Foolscap as a build-dependency of
some other project without having Twisted available first. If setuptools is
available, we also declare a dependency upon Twisted (at least 2.4.0), to
give more information to the setuptools dependency-auto-installer.

** Unauthenticated FURLs can now contain multiple connection hints

Previously they were limited to a single one

** FURLs can now contain extensions, providing forwards-compatibility

The parsing of FURLs has been refined to tolerate (and ignore) certain kinds
of extensions. The "tubid" section will be able to have additional
identifiers (perhaps stronger hashes for the public key, or an
encryption-ready EC-DSA public key). In addition, the "connection hints"
section will be able to contain alternate protocol specifiers (for TCP over
IPv6, or a less connection-oriented UDP transport).

By ignoring such extensions, foolscap-0.2.6 will tolerate FURLs produced
(with extensions) by some future version of foolscap. This marks the
beginning of a "transition period": when such extensions are introduced,
0.2.6 will be the oldest version still capable of interoperating with the
extension-using new version.