id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	launchpad_bug
1565	URL formats for HTTP-based storage server	warner		"Ticket #510 is about speaking to storage servers with mostly-plain HTTP. One
piece of this is deciding what the URLs should look like. Downloading a share
from the storage server should be a simple HTTP ""GET"", using a {{{Range:}}}
header to fetch less than the whole share. But we also need ways to discover
which shares are available for download, and eventually ways to upload data
to the server too.

Here's the starting point that I implemented in my prototype (which still
uses Foolscap and get_buckets() to discover shares):

* {{{GET /storage/imm/SI/%(storage_index)s/share/%(shnum)d}}}: retrieves data
  from the given share. Normal downloads use e.g. `Range:
  bytes=87418-131108,422601-422664,423593-423656` to fetch a bunch of
  spans.
* {{{GET /storage}}}: this currently returns a human-readable page describing
  the state of the storage server.

The next steps:

* {{{GET /storage/imm/SI/%(storage_index)s/shares}}}: return a JSON list of
  share numbers
* {{{GET /storage/imm/SI/%(storage_index)s/all_shares}}}: return a JSON
  dictionary mapping share number to a read data vector. The same spans are
  returned for all shares. This collapses the Do-You-Have-Block query with
  the initial data fetch, allowing one-round-trip downloads.

I put ""imm"" into the URL because the current storage server treats immutable
and mutable shares very differently (they have different container formats).
It's not trivial to take an SI and switch on the type of share that it points
to. It might be cleaner to fix the server to handle this well, and then
remove the ""imm"" from the URL. OTOH, it might be better to leave them
distinct.

We need similar URLs for reading from mutable shares; they can probably be
the same but with ""mut"" instead of ""imm"".

We'll need POST URLs for uploading files and modifying mutable shares, as
well as adding/renewing leases and other storage server methods. The request
bodies will be more complicated since they'll need authorization signatures
or something. But the basic URL target could be:

* {{{POST /storage/imm/SI/%(storage_index)s/shares/%(shnum)d}}}: start
  uploading the given share. Return 302 FOUND if the share already exists.
  The upload can be spread across multiple requests, with a ""finished"" flag
  on the last request. This might involve returning an ""upload token"" which
  subsequent requests must reference.
* {{{POST /storage/mut/SI/%(storage_index)s/shares/%(shnum)d}}}: modify the
  given mutable share. The body will probably be a signed serialized JSON
  modification request, basically a write-vector, along with a test-vector or
  other collision-avoidance scheme.

All of this presumes that Accounting is not being enforced on read access. At
least one of the designs I've drawn up offers {{{read=False}}} control, as a
stick for the storage operator to use against a client who doesn't pay their
bills (but still less drastic than {{{store=False}}}, which deletes all their
data). To enforce {{{read=False}}}, the GETs would need to be authorized,
which either involves adding an extra signature header, or implementing them
with a POST instead (and putting the signature in the request body).
"	task	closed	major	eventually	code-storage	1.9.0b1	fixed	newurls accounting	nejucomo