Ticket #833: docs-diff.txt

--- old-tahoe/docs/frontends/webapi.txt 2010-01-22 06:05:46.188000000 +0000
+++ new-tahoe/docs/frontends/webapi.txt 2010-01-22 06:05:46.257000000 +0000
@@ -70,20 +70,20 @@
 these tasks. In general, everything that can be done with a PUT or DELETE can
 also be done with a POST.
 
-Tahoe's web API is designed for two different consumers. The first is a
-program that needs to manipulate the virtual file system. Such programs are
+Tahoe's web API is designed for two different kinds of consumer. The first is
+a program that needs to manipulate the virtual file system. Such programs are
 expected to use the RESTful interface described above. The second is a human
 using a standard web browser to work with the filesystem. This user is given
 a series of HTML pages with links to download files, and forms that use POST
 actions to upload, rename, and delete files.
 
 When an error occurs, the HTTP response code will be set to an appropriate
-400-series code (like 404 for an unknown childname, or 400 Gone when a file
-is unrecoverable due to insufficient shares), and the HTTP response body will
-usually contain a few lines of explanation as to the cause of the error and
-possible responses. Unusual exceptions may result in a 500 Internal Server
-Error as a catch-all, with a default response body will contain a
-Nevow-generated HTML-ized representation of the Python exception stack trace
+400-series code (like 404 Not Found for an unknown childname, or 400 Bad Request
+when the parameters to a webapi operation are invalid), and the HTTP response
+body will usually contain a few lines of explanation as to the cause of the
+error and possible responses. Unusual exceptions may result in a
+500 Internal Server Error as a catch-all, with a default response body containing
+a Nevow-generated HTML-ized representation of the Python exception stack trace
 that caused the problem. CLI programs which want to copy the response body to
 stderr should provide an "Accept: text/plain" header to their requests to get
 a plain text stack trace instead. If the Accept header contains */*, or
@@ -108,9 +108,9 @@
 read- and write- caps, which start with "URI:SSK", and give access to mutable
 files.
 
-(later versions of Tahoe will make these strings shorter, and will remove the
+(Later versions of Tahoe will make these strings shorter, and will remove the
 unfortunate colons, which must be escaped when these caps are embedded in
-URLs).
+URLs.)
 
 To refer to any Tahoe object through the web API, you simply need to combine
 a prefix (which indicates the HTTP server to use) with the cap (which
@@ -195,9 +195,9 @@
 representable as such.
 
 All Tahoe operations that refer to existing files or directories must include
-a suitable read- or write- cap in the URL: the wapi server won't add one
+a suitable read- or write- cap in the URL: the webapi server won't add one
 for you. If you don't know the cap, you can't access the file. This allows
-the security properties of Tahoe caps to be extended across the wapi
+the security properties of Tahoe caps to be extended across the webapi
 interface.
 
 == Slow Operations, Progress, and Cancelling ==
@@ -271,7 +271,7 @@
    since the operation completed) will remain valid for ten minutes.
 
 Many "slow" operations can begin to use unacceptable amounts of memory when
-operation on large directory structures. The memory usage increases when the
+operating on large directory structures. The memory usage increases when the
 ophandle is polled, as the results must be copied into a JSON string, sent
 over the wire, then parsed by a client. So, as an alternative, many "slow"
 operations have streaming equivalents. These equivalents do not use operation
@@ -314,7 +314,7 @@
  To use the /uri/$FILECAP form, $FILECAP be a write-cap for a mutable file.
 
  In the /uri/$DIRCAP/[SUBDIRS../]FILENAME form, if the target file is a
- writable mutable file, that files contents will be overwritten in-place. If
+ writable mutable file, that file's contents will be overwritten in-place. If
  it is a read-cap for a mutable file, an error will occur. If it is an
  immutable file, the old file will be discarded, and a new one will be put in
  its place.
@@ -333,7 +333,7 @@
 PUT /uri
 
  This uploads a file, and produces a file-cap for the contents, but does not
- attach the file into the virtual drive. No directories will be modified by
+ attach the file into the filesystem. No directories will be modified by
  this operation. The file-cap is returned as the body of the HTTP response.
 
  If "mutable=true" is in the query arguments, the operation will create a
@@ -347,7 +347,7 @@
 
  Create a new empty directory and return its write-cap as the HTTP response
  body. This does not make the newly created directory visible from the
- virtual drive. The "PUT" operation is provided for backwards compatibility:
+ filesystem. The "PUT" operation is provided for backwards compatibility:
  new code should use POST.
 
 POST /uri?t=mkdir-with-children
@@ -388,8 +388,29 @@
             "linkcrtime": 1202777696.7564139,
             "linkmotime": 1202777696.7564139,
           } } } ]
- }
+  }
 
+ For forward-compatibility, a mutable directory can also contain caps in
+ a format that is unknown to the webapi server. When such caps are retrieved
+ from a mutable directory in a "ro_uri" field, they will be prefixed with
+ the string "ro.", indicating that they must not be decoded without
+ checking that they are read-only. The "ro." prefix must not be stripped
+ off without performing this check. (Future versions of the webapi server
+ will perform it where necessary.)
+
+ If both the "rw_uri" and "ro_uri" fields are present in a given PROPDICT,
+ and the webapi server recognizes the rw_uri as a write cap, then it will
+ reset the ro_uri to the corresponding read cap and discard the original
+ contents of ro_uri (in order to ensure that the two caps correspond to the
+ same object and that the ro_uri is in fact read-only). However this may not
+ happen for caps in a format unknown to the webapi server. Therefore, when
+ writing a directory the webapi client should ensure that the contents
+ of "rw_uri" and "ro_uri" for a given PROPDICT are a consistent
+ (write cap, read cap) pair if possible. If the webapi client only has
+ one cap and does not know whether it is a write cap or read cap, then
+ it is acceptable to set "rw_uri" to that cap and omit "ro_uri". The
+ client must not put a write cap into a "ro_uri" field.
+ 
  Note that the webapi-using client application must not provide the
  "Content-Type: multipart/form-data" header that usually accompanies HTML
  form submissions, since the body is not formatted this way. Doing so will
@@ -404,59 +425,95 @@
 
  Like t=mkdir-with-children above, but the new directory will be
  deep-immutable. This means that the directory itself is immutable, and that
- it can only contain deep-immutable objects, like immutable files, literal
- files, and deep-immutable directories. A non-empty request body is
- mandatory, since after the directory is created, it will not be possible to
- add more children to it.
+ it can only contain objects that are treated as being deep-immutable, like
+ immutable files, literal files, and deep-immutable directories.
+
+ For forward-compatibility, a deep-immutable directory can also contain caps
+ in a format that is unknown to the webapi server. When such caps are retrieved
+ from a deep-immutable directory in a "ro_uri" field, they will be prefixed
+ with the string "imm.", indicating that they must not be decoded without
+ checking that they are immutable. The "imm." prefix must not be stripped
+ off without performing this check. (Future versions of the webapi server
+ will perform it where necessary.)
+ 
+ The cap for each child may be given either in the "rw_uri" or "ro_uri"
+ field of the PROPDICT (not both). If a cap is given in the "rw_uri" field,
+ then the webapi server will check that it is an immutable readcap of a
+ *known* format, and give an error if it is not. If a cap is given in the
+ "ro_uri" field, then the webapi server will still check whether known
+ caps are immutable, but for unknown caps it will simply assume that the
+ cap can be stored, as described above. Note that an attacker would be
+ able to store any cap in an immutable directory, so this check when
+ creating the directory is only to help non-malicious clients to avoid
+ accidentally giving away more authority than intended.
+
+ A non-empty request body is mandatory, since after the directory is created,
+ it will not be possible to add more children to it.
 
 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir
 PUT /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir
 
  Create new directories as necessary to make sure that the named target
  ($DIRCAP/SUBDIRS../SUBDIR) is a directory. This will create additional
- intermediate directories as necessary. If the named target directory already
- exists, this will make no changes to it.
+ intermediate mutable directories as necessary. If the named target directory
+ already exists, this will make no changes to it.
 
  If the final directory is created, it will be empty.
 
- This will return an error if a blocking file is present at any of the parent
- names, preventing the server from creating the necessary parent directory.
+ This operation will return an error if a blocking file is present at any of
+ the parent names, preventing the server from creating the necessary parent
+ directory, or if it would require changing an immutable directory.
 
  The write-cap of the new directory will be returned as the HTTP response
  body.
 
 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-with-children
 
- Like above, but if the final directory is created, it will be populated with
- initial children from the POST request body, as described above in the
- /uri?t=mkdir-with-children operation.
+ Like /uri?t=mkdir-with-children, but the final directory is created as a
+ child of an existing mutable directory. This will create additional
+ intermediate mutable directories as necessary. If the final directory is
+ created, it will be populated with initial children from the POST request
+ body, as described above.
 
 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-immutable
 
- Like above, but the final directory will be deep-immutable, with the
- children specified as a JSON dictionary in the POST request body.
+ Like /uri?t=mkdir-immutable, but the final directory is created as a child
+ of an existing mutable directory. The final directory will be deep-immutable,
+ and will be populated with the children specified as a JSON dictionary in
+ the POST request body.
+
+ In Tahoe 1.6 this operation creates intermediate mutable directories if
+ necessary, but that behaviour should not be relied on; see ticket #920.
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME
 
- Create a new empty directory and attach it to the given existing directory.
- This will create additional intermediate directories as necessary.
+ Create a new empty mutable directory and attach it to the given existing
+ directory. This will create additional intermediate directories as necessary.
 
- The URL of this form points to the parent of the bottom-most new directory,
- whereas the previous form has a URL that points directly to the bottom-most
- new directory.
+ This operation will return an error if a blocking file is present at any of
+ the parent names, preventing the server from creating the necessary parent
+ directory, or if it would require changing any immutable directory.
+
+ The URL of this operation points to the parent of the bottommost new directory,
+ whereas the /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir operation above has a URL
+ that points directly to the bottommost new directory.
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME
 
- As above, but the new directory will be populated with initial children via
- the POST request body, as described in /uri?t=mkdir-with-children above.
+ Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME, but the new directory will
+ be populated with initial children via the POST request body.
  Note that the name= argument must be passed as a queryarg, because the POST
  request body is used for the initial children JSON.
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-immutable&name=NAME
 
- As above, but the new directory will be deep-immutable, with the children
- specified as a JSON dictionary in the POST request body. Again, the name=
- argument must be passed as a queryarg.
+ Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME, but the
+ final directory will be deep-immutable. The children are specified as a
+ JSON dictionary in the POST request body. Again, the name= argument must be
+ passed as a queryarg.
+
+ In Tahoe 1.6 this operation creates intermediate mutable directories if
+ necessary, but that behaviour should not be relied on; see ticket #920.
 
 === Get Information About A File Or Directory (as JSON) ===
 
@@ -546,7 +603,7 @@
 
   Then the rw_uri field will be present in the information about a directory
   if and only if you have read-write access to that directory. The verify_uri
-  field will be presend if and only if the object has a verify-cap
+  field will be present if and only if the object has a verify-cap
   (non-distributed LIT files do not have verify-caps).
 
 ==== About the metadata ====
@@ -622,11 +679,11 @@
   link points.
 
   4. Also, quite apart from Tahoe, you might be confused about the meaning of
-  the 'ctime' in unix local filesystems, which people sometimes think means
-  file creation time, but which actually means, in unix local filesystems, the
+  the 'ctime' in UNIX local filesystems, which people sometimes think means
+  file creation time, but which actually means, in UNIX local filesystems, the
   most recent time that the file contents or the file metadata (such as owner,
   permission bits, extended attributes, etc.) has changed. Note that although
-  'ctime' does not mean file creation time in Unix, it does mean link creation
+  'ctime' does not mean file creation time in UNIX, it does mean link creation
   time in Tahoe, unless the "tahoe backup" command has been used on that link,
   in which case it means something about the local filesystem file which
   corresponds to the Tahoe file which is pointed at by the link. It means
@@ -634,7 +691,6 @@
   Windows) or file-contents-or-metadata-update-time of the local file (if
   "tahoe backup" was run on a different operating system).
 
-
 === Attaching an existing File or Directory by its read- or write- cap ===
 
 PUT /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=uri
@@ -658,10 +714,15 @@
   if there is already an object at the given location, rather than
   overwriting the existing object. To allow the operation to overwrite a
   file, but return an error when trying to overwrite a directory, use
-  "replace=only-files" (this behavior is closer to the traditional unix "mv"
+  "replace=only-files" (this behavior is closer to the traditional UNIX "mv"
   command). Note that "true", "t", and "1" are all synonyms for "True", and
   "false", "f", and "0" are synonyms for "False", and the parameter is
   case-insensitive.
+  
+  Note that this operation does not take its child cap in the form of
+  separate "rw_uri" and "ro_uri" fields. Therefore, it cannot accept a
+  child cap in a format unknown to the webapi server, because the server
+  is not able to attenuate an unknown write cap to a read cap.
 
 === Adding multiple files or directories to a parent directory at once ===
 
@@ -678,7 +739,9 @@
   "childinfo" is a dictionary that contains "rw_uri", "ro_uri", and
   "metadata" keys. You can take the output of "GET /uri/$DIRCAP1?t=json" and
   use it as the input to "POST /uri/$DIRCAP2?t=set_children" to make DIR2
-  look very much like DIR1.
+  look very much like DIR1 (except for any existing children of DIR2 that
+  were not overwritten, and any existing "tahoe" metadata keys as described
+  below).
 
   When the set_children request contains a child name that already exists in
   the target directory, this command defaults to overwriting that child with
@@ -695,7 +758,6 @@
   the section above entitled "Get Information About A File Or Directory (as
   JSON)", in the "About the metadata" subsection.
 
-
 === Deleting a File or Directory ===
 
 DELETE /uri/$DIRCAP/[SUBDIRS../]CHILDNAME
@@ -715,7 +777,7 @@
   The object will only become completely unreachable once 1: there are no
   reachable directories that reference it, and 2: nobody is holding a read-
   or write- cap to the object. (This behavior is very similar to the way
-  hardlinks and anonymous files work in traditional unix filesystems).
+  hardlinks and anonymous files work in traditional UNIX filesystems).
 
   This operation will not modify more than a single directory. Intermediate
   directories which were implicitly created by PUT or POST methods will *not*
@@ -844,7 +906,7 @@
 POST /uri?t=upload
 
  This uploads a file, and produces a file-cap for the contents, but does not
- attach the file into the virtual drive. No directories will be modified by
+ attach the file into the filesystem. No directories will be modified by
  this operation.
 
  The file must be provided as the "file" field of an HTML encoded form body,
@@ -874,9 +936,9 @@
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=upload
 
- This uploads a file, and attaches it as a new child of the given directory.
- The file must be provided as the "file" field of an HTML encoded form body,
- produced in response to an HTML form like this:
+ This uploads a file, and attaches it as a new child of the given directory,
+ which must be mutable. The file must be provided as the "file" field of an
+ HTML-encoded form body, produced in response to an HTML form like this:
   <form action="." method="POST" enctype="multipart/form-data">
    <input type="hidden" name="t" value="upload" />
    <input type="file" name="file" />
@@ -919,9 +981,10 @@
 POST /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=upload
 
  This also uploads a file and attaches it as a new child of the given
- directory. It is a slight variant of the previous operation, as the URL
- refers to the target file rather than the parent directory. It is otherwise
- identical: this accepts mutable= and when_done= arguments too.
+ directory, which must be mutable. It is a slight variant of the previous
+ operation, as the URL refers to the target file rather than the parent
+ directory. It is otherwise identical: this accepts mutable= and when_done=
+ arguments too.
 
 POST /uri/$FILECAP?t=upload
 
@@ -949,20 +1012,21 @@
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=delete&name=CHILDNAME
 
- This instructs the node to delete a child object (file or subdirectory) from
- the given directory. Note that the entire subtree is removed. This is
- somewhat like "rm -rf" (from the point of view of the parent), but other
- references into the subtree will see that the child subdirectories are not
- modified by this operation. Only the link from the given directory to its
- child is severed.
+ This instructs the node to remove a child object (file or subdirectory) from
+ the given directory, which must be mutable. Note that the entire subtree is
+ unlinked from the parent. Unlike deleting a subdirectory in a UNIX local
+ filesystem, the subtree need not be empty; if it isn't, then other references
+ into the subtree will see that the child subdirectories are not modified by
+ this operation. Only the link from the given directory to its child is severed.
 
 === Renaming A Child ===
 
 POST /uri/$DIRCAP/[SUBDIRS../]?t=rename&from_name=OLD&to_name=NEW
 
- This instructs the node to rename a child of the given directory. This is
- exactly the same as removing the child, then adding the same child-cap under
- the new name. This operation cannot move the child to a different directory.
+ This instructs the node to rename a child of the given directory, which must
+ be mutable. This has a similar effect to removing the child, then adding the
+ same child-cap under the new name, except that it preserves metadata. This
+ operation cannot move the child to a different directory.
 
  This operation will replace any existing child of the new name, making it
  behave like the UNIX "mv -f" command.
@@ -1584,7 +1648,7 @@
 
 == Static Files in /public_html ==
 
-The wapi server will take any request for a URL that starts with /static
+The webapi server will take any request for a URL that starts with /static
 and serve it from a configurable directory which defaults to
 $BASEDIR/public_html . This is configured by setting the "[node]web.static"
 value in $BASEDIR/tahoe.cfg . If this is left at the default value of
@@ -1592,7 +1656,7 @@
 served with the contents of the file $BASEDIR/public_html/subdir/foo.html .
 
 This can be useful to serve a javascript application which provides a
-prettier front-end to the rest of the Tahoe wapi.
+prettier front-end to the rest of the Tahoe webapi.
 
 
 == safety and security issues -- names vs. URIs ==
@@ -1635,6 +1699,17 @@
 child's name and the child's URI are included in the results of listing the
 parent directory, so it isn't any harder to use the URI for this purpose.
 
+The read and write caps in a given directory node are separate URIs, and
+can't be assumed to point to the same object even if they were retrieved in
+the same operation (although the webapi server attempts to ensure this
+in most cases). If you need to rely on that property, you should explicitly
+verify it. More generally, you should not make assumptions about the
+internal consistency of the contents of mutable directories. As a result
+of the signatures on mutable object versions, it is guaranteed that a given
+version was written in a single update, but -- as in the case of a file --
+the contents may have been chosen by a malicious writer in a way that is
+designed to confuse applications that rely on their consistency.
+
 In general, use names if you want "whatever object (whether file or
 directory) is found by following this name (or sequence of names) when my
 request reaches the server". Use URIs if you want "this particular object".
@@ -1670,7 +1745,7 @@
 
 Tahoe nodes implement internal serialization to make sure that a single Tahoe
 node cannot conflict with itself. For example, it is safe to issue two
-directory modification requests to a single tahoe node's wapi server at the
+directory modification requests to a single tahoe node's webapi server at the
 same time, because the Tahoe node will internally delay one of them until
 after the other has finished being applied. (This feature was introduced in
 Tahoe-1.1; back with Tahoe-1.0 the web client was responsible for serializing

--- old-tahoe/relnotes.txt      2010-01-22 06:05:46.198000000 +0000
+++ new-tahoe/relnotes.txt      2010-01-22 06:05:46.261000000 +0000
@@ -1,7 +1,7 @@
-ANNOUNCING Tahoe, the Lofty-Atmospheric Filesystem, v1.5
+ANNOUNCING Tahoe, the Lofty-Atmospheric Filesystem, v1.6
 
 The Tahoe-LAFS team is pleased to announce the immediate
-availability of version 1.5 of Tahoe, the Lofty Atmospheric
+availability of version 1.6 of Tahoe, the Lofty Atmospheric
 File System.
 
 Tahoe-LAFS is the first cloud storage technology which offers
@@ -29,15 +29,20 @@
 
 COMPATIBILITY
 
-Version 1.5 is fully compatible with the version 1 series of
-Tahoe-LAFS. Files written by v1.5 clients can be read by
-clients of all versions back to v1.0. v1.5 clients can read
-files produced by clients of all versions since v1.0.  v1.5
-servers can serve clients of all versions back to v1.0 and v1.5
+Version 1.6 is fully compatible with the version 1 series of
+Tahoe-LAFS. Files written by v1.6 clients can be read by
+clients of all versions back to v1.0. v1.6 clients can read
+files produced by clients of all versions since v1.0.  v1.6
+servers can serve clients of all versions back to v1.0 and v1.6
 clients can use servers of all versions back to v1.0.
 
-This is the sixth release in the version 1 series. The version
-1 series of Tahoe-LAFS will be actively supported and
+In addition, version 1.6 improves forward-compatibility with
+planned future cap formats, allowing updates to a directory
+containing both current and future caps, without loss of
+information.
+
+This is the seventh major release in the version 1 series. The
+version 1 series of Tahoe-LAFS will be actively supported and
 maintained for the forseeable future, and future versions of
 Tahoe-LAFS will retain the ability to read and write files
 compatible with Tahoe-LAFS v1.