[tahoe-dev] Idea for a Publish/Subscribe Message System, on Tahoe-LAFS

[Stuart W. Card]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

all --

Sorry to chime in late and initially without reference citations, but I
wanted you all to be aware of some work we have been doing recently at
Syracuse University and my company Critical Technologies Inc., using
Tahoe-LAFS as a principal building block for a publish-subscribe-query
distributed blackboard that will be the backbone of a framework for
distributed sensing/computation/control. A unifying theme is use of
capability based security throughout the stack, from the underlying
microkernel-based hypervisor all the way up through the mobile agent
based applications.

We are working towards compliance with OMG Data Distribution Service
(DDS) API specs, but initially have done only proof-of-concept work
without regard for such specs. We are looking at plugging in Tahoe as an
archival message store for OpenDDS, OpenSplice or RTI-DDS.

I am limited in what I can say in public prior to obtaining specific
permission from our funding sources, but as Tahoe is a community
supported product that is central to our architecture, I believe I can
discuss here those aspects directly related to Tahoe development.

I will go out on a limb and say that I think the whole notion of mutable
files is a mistake. We currently use them only for directories, and
there only until such time as we develop a replacement based on our
architecture. Instead of mutable files, we prefer revision control: once
a message (file) is published, it is immutable for all time; but a new
version can be published, as either a complete snapshot or a diff
(respect to rsync), with suitable meta-data markups and pointers in
directories (also to be immutable but version controlled).

To provide notification of new publications (including revisions of old
ones), we use the NACK Oriented Reliable Multicast (NORM) transport
protocol, which is the _only_ IETF standards track reliable multicast
protocol that offers both statistical reliability (erasure code FEC) and
acknowledged delivery (ARQ), combining them in a Type II Hybrid ARQ/FEC
(which can approach the Shannon limit despite dynamically varying
channel error rates). Urgent short messages that do not require
persistent archival can also be sent via NORM without the latencies and
costs inherent to Tahoe. Long term, we also want to enable use of NORM
as a multicast transport for Tahoe, in place of multiple unicast TCP
connections.

We have some notions about using URIs as capabilities, not only for
Tahoe, but also for the microvisor and other subsystems. We want to
define a syntax and semantics for extended capability URIs that can
embed policies (including PKI based restrictions on who can exercise the
capability, thus integrating Role Based Access Control with
capabilities), and short "handle" URIs (that merely reference long URIs
with their embedded information). We intend to develop capability
middleware (including proxies subsuming the capability security notion
of "membrane") that will enable programmers and admins who are not [yet]
familiar with capability security to nonetheless utilize it with a
minimum of unintended capability leaks.

If any of you find any of the above interesting and want to discuss it,
please e-mail me directly, as (unfortunately) I can't go into much more
detail on the list (I must obtain specific approval of the text of each
"public release of information" that I make).

- -- 
Stuart W. Card, PhD: VP & Chief Scientist, Critical Technologies Inc.
* Creativity * Diversity * Expertise * Flexibility * Integrity *
Suite 400 Technology Center, 4th Floor 1001 Broad St, Utica NY 13501
315-793-0248 x141 FAX -9710 <Stu.Card at critical.com> www.critical.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9nWmAACgkQS7PQ0a2weL7ZGwCgnPaITAgigg3PRPUhkNrul3pB
Ur8An1ALUNdzXv76XlO18wkVnUpcjWN+
=/iWs
-----END PGP SIGNATURE-----