source: trunk/NEWS.rst

Last change on this file was eff5df1, checked in by meejah <meejah@…>, at 2020-03-12T00:41:32Z

update title

  • Property mode set to 100644
File size: 144.1 KB
Line 
1.. -*- coding: utf-8-with-signature -*-
2
3==================================
4User-Visible Changes in Tahoe-LAFS
5==================================
6
7.. towncrier start line
8Release 1.14.0 (2020-03-11)
9'''''''''''''''''''''''''''
10
11Features
12--------
13
14- Magic-Folders are now supported on macOS. (`#1432 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1432>`_)
15- Add a "tox -e draftnews" which runs towncrier in draft mode (`#2942 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2942>`_)
16- Fedora 29 is now tested as part of the project's continuous integration system. (`#2955 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2955>`_)
17- The Magic-Folder frontend now emits structured, causal logs.  This makes it easier for developers to make sense of its behavior and for users to submit useful debugging information alongside problem reports. (`#2972 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2972>`_)
18- The `tahoe` CLI now accepts arguments for configuring structured logging messages which Tahoe-LAFS is being converted to emit.  This change does not introduce any new defaults for on-filesystem logging. (`#2975 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2975>`_)
19- The web API now publishes streaming Eliot logs via a token-protected WebSocket at /private/logs/v1. (`#3006 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3006>`_)
20- End-to-end in-memory tests for websocket features (`#3041 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3041>`_)
21- allmydata.interfaces.IFoolscapStoragePlugin has been introduced, an extension point for customizing the storage protocol. (`#3049 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3049>`_)
22- Static storage server "announcements" in ``private/servers.yaml`` are now individually logged and ignored if they cannot be interpreted. (`#3051 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3051>`_)
23- Storage servers can now be configured to load plugins for allmydata.interfaces.IFoolscapStoragePlugin and offer them to clients. (`#3053 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3053>`_)
24- Storage clients can now be configured to load plugins for allmydata.interfaces.IFoolscapStoragePlugin and use them to negotiate with servers. (`#3054 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3054>`_)
25- The [storage] configuration section now accepts a boolean *anonymous* item to enable or disable anonymous storage access.  The default behavior remains unchanged. (`#3184 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3184>`_)
26- Enable the helper when creating a node with `tahoe create-node --helper` (`#3235 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3235>`_)
27
28
29Bug Fixes
30---------
31
32- refactor initialization code to be more async-friendly (`#2870 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2870>`_)
33- Configuration-checking code wasn't being called due to indenting (`#2935 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2935>`_)
34- refactor configuration handling out of Node into _Config (`#2936 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2936>`_)
35- "tox -e codechecks" no longer dirties the working tree. (`#2941 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2941>`_)
36- Updated the Tor release key, used by the integration tests. (`#2944 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2944>`_)
37- `tahoe backup` no longer fails with an unhandled exception when it encounters a special file (device, fifo) in the backup source. (`#2950 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2950>`_)
38- Magic-Folders now creates spurious conflict files in fewer cases.  In particular, if files are added to the folder while a client is offline, that client will not create conflict files for all those new files when it starts up. (`#2965 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2965>`_)
39- The confusing and misplaced sub-command group headings in `tahoe --help` output have been removed. (`#2976 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2976>`_)
40- The Magic-Folder frontend is now more responsive to subtree changes on Windows. (`#2997 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2997>`_)
41- remove ancient bundled jquery and d3, and the "dowload timeline" feature they support (`#3228 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3228>`_)
42
43
44Dependency/Installation Changes
45-------------------------------
46
47- Tahoe-LAFS no longer makes start-up time assertions about the versions of its dependencies.  It is the responsibility of the administrator of the installation to ensure the correct version of dependencies are supplied. (`#2749 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2749>`_)
48- Tahoe-LAFS now depends on Twisted 16.6 or newer. (`#2957 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2957>`_)
49
50
51Removed Features
52----------------
53
54- "tahoe rm", an old alias for "tahoe unlink", has been removed. (`#1827 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1827>`_)
55- The direct dependencies on pyutil and zbase32 have been removed. (`#2098 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2098>`_)
56- Untested and unmaintained code for running Tahoe-LAFS as a Windows service has been removed. (`#2239 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2239>`_)
57- The redundant "pypywin32" dependency has been removed. (`#2392 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2392>`_)
58- Fedora 27 is no longer tested as part of the project's continuous integration system. (`#2955 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2955>`_)
59- "tahoe start", "tahoe daemonize", "tahoe restart", and "tahoe stop" are now deprecated in favor of using "tahoe run", possibly with a third-party process manager. (`#3273 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3273>`_)
60
61
62Other Changes
63-------------
64
65- Tahoe-LAFS now tests for PyPy compatibility on CI. (`#2479 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2479>`_)
66- Tahoe-LAFS now requires Twisted 18.4.0 or newer. (`#2771 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2771>`_)
67- Tahoe-LAFS now uses towncrier to maintain the NEWS file. (`#2908 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2908>`_)
68- The release process document has been updated. (`#2920 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2920>`_)
69- allmydata.test.test_system.SystemTest is now more reliable with respect to bound address collisions. (`#2933 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2933>`_)
70- The Tox configuration has been fixed to work around a problem on Windows CI. (`#2956 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2956>`_)
71- The PyInstaller CI job now works around a pip/pyinstaller incompatibility. (`#2958 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2958>`_)
72- Some CI jobs for integration tests have been moved from TravisCI to CircleCI. (`#2959 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2959>`_)
73- Several warnings from a new release of pyflakes have been fixed. (`#2960 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2960>`_)
74- Some Slackware 14.2 continuous integration problems have been resolved. (`#2961 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2961>`_)
75- Some macOS continuous integration failures have been fixed. (`#2962 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2962>`_)
76- The NoNetworkGrid implementation has been somewhat improved. (`#2966 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2966>`_)
77- A bug in the test suite for the create-alias command has been fixed. (`#2967 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2967>`_)
78- The integration test suite has been updated to use pytest-twisted instead of deprecated pytest APIs. (`#2968 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2968>`_)
79- The magic-folder integration test suite now performs more aggressive cleanup of the processes it launches. (`#2969 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2969>`_)
80- The integration tests now correctly document the `--keep-tempdir` option. (`#2970 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2970>`_)
81- A misuse of super() in the integration tests has been fixed. (`#2971 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2971>`_)
82- Several utilities to facilitate the use of the Eliot causal logging library have been introduced. (`#2973 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2973>`_)
83- The Windows CI configuration has been tweaked. (`#2974 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2974>`_)
84- The Magic-Folder frontend has had additional logging improvements. (`#2977 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2977>`_)
85-  (`#2981 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2981>`_, `#2982 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2982>`_)
86- Added a simple sytax checker so that once a file has reached python3 compatibility, it will not regress. (`#3001 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3001>`_)
87- Converted all uses of the print statement to the print function in the ./misc/ directory. (`#3002 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3002>`_)
88- The contributor guidelines are now linked from the GitHub pull request creation page. (`#3003 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3003>`_)
89- Updated the testing code to use the print function instead of the print statement. (`#3008 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3008>`_)
90- Replaced print statement with print fuction for all tahoe_* scripts. (`#3009 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3009>`_)
91- Replaced all remaining instances of the print statement with the print function. (`#3010 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3010>`_)
92- Replace StringIO imports with six.moves. (`#3011 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3011>`_)
93- Updated all Python files to use PEP-3110 exception syntax for Python3 compatibility. (`#3013 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3013>`_)
94- Update raise syntax for Python3 compatibility. (`#3014 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3014>`_)
95- Updated instances of octal literals to use the format 0o123 for Python3 compatibility. (`#3015 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3015>`_)
96- allmydata.test.no_network, allmydata.test.test_system, and allmydata.test.web.test_introducer are now more reliable with respect to bound address collisions. (`#3016 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3016>`_)
97- Removed tuple unpacking from function and lambda definitions for Python3 compatibility. (`#3019 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3019>`_)
98- Updated Python2 long numeric literals for Python3 compatibility. (`#3020 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3020>`_)
99- CircleCI jobs are now faster as a result of pre-building configured Docker images for the CI jobs. (`#3024 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3024>`_)
100- Removed used of backticks for "repr" for Python3 compatibility. (`#3027 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3027>`_)
101- Updated string literal syntax for Python3 compatibility. (`#3028 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3028>`_)
102- Updated CI to enforce Python3 syntax for entire repo. (`#3030 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3030>`_)
103- Replaced pycryptopp with cryptography. (`#3031 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3031>`_)
104- All old-style classes ported to new-style. (`#3042 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3042>`_)
105- Whitelisted "/bin/mv" as command for codechecks performed by tox. This fixes a current warning and prevents future errors (for tox 4). (`#3043 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3043>`_)
106- Progress towards Python 3 compatibility is now visible at <https://tahoe-lafs.github.io/tahoe-depgraph/>. (`#3152 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3152>`_)
107- Collect coverage information from integration tests (`#3234 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3234>`_)
108- NixOS is now a supported Tahoe-LAFS platform. (`#3266 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3266>`_)
109
110
111Misc/Other
112----------
113
114- `#1893 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1893>`_, `#2266 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2266>`_, `#2283 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2283>`_, `#2766 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2766>`_, `#2980 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2980>`_, `#2985 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2985>`_, `#2986 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2986>`_, `#2987 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2987>`_, `#2988 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2988>`_, `#2989 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2989>`_, `#2990 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2990>`_, `#2991 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2991>`_, `#2992 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2992>`_, `#2995 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2995>`_, `#3000 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3000>`_, `#3004 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3004>`_, `#3005 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3005>`_, `#3007 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3007>`_, `#3012 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3012>`_, `#3017 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3017>`_, `#3021 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3021>`_, `#3023 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3023>`_, `#3025 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3025>`_, `#3026 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3026>`_, `#3029 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3029>`_, `#3036 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3036>`_, `#3038 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3038>`_, `#3048 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3048>`_, `#3086 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3086>`_, `#3097 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3097>`_, `#3111 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3111>`_, `#3118 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3118>`_, `#3119 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3119>`_, `#3227 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3227>`_, `#3229 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3229>`_, `#3232 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3232>`_, `#3233 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3233>`_, `#3237 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3237>`_, `#3238 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3238>`_, `#3239 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3239>`_, `#3240 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3240>`_, `#3242 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3242>`_, `#3243 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3243>`_, `#3245 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3245>`_, `#3246 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3246>`_, `#3248 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3248>`_, `#3250 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3250>`_, `#3252 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3252>`_, `#3255 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3255>`_, `#3256 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3256>`_, `#3259 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3259>`_, `#3261 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3261>`_, `#3262 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3262>`_, `#3263 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3263>`_, `#3264 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3264>`_, `#3265 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3265>`_, `#3267 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3267>`_, `#3268 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3268>`_, `#3271 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3271>`_, `#3272 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3272>`_, `#3274 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3274>`_, `#3275 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3275>`_, `#3276 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3276>`_, `#3279 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3279>`_, `#3281 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3281>`_, `#3282 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3282>`_, `#3285 <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3285>`_
115
116
117Release 1.13.0 (05-August-2018)
118'''''''''''''''''''''''''''''''
119
120New Features
121------------
122
123The ``tahoe list-aliases`` command gained the ``--readonly-uri``
124option in `PR400`_, which lists read-only capabilities (the default
125shows read/write capabilities if available). This command also gained
126a ``--json`` option in `PR452`_, providing machine-readable output.
127
128A new command ``tahoe status`` is added, showing some statistics and
129currently active operations (similar to the ``/status`` page in the
130Web UI). See also `PR502`_.
131
132Immutable uploads now use the "servers of happiness" algorithm for
133uploading shares. This means better placement of shares on available
134servers. See `PR416`_.
135
136To join a new client to a grid, the command ``tahoe invite`` was
137added. This uses `magic wormhole`_ to connect two computers and
138exchange the required information to start the client. The "client
139side" of this command is the also new option ``tahoe
140create-client --join=``. Together, this provides a way to provision a
141new client without having to securely transmit the fURL and other
142details. `PR418`_
143
144``tahoe backup`` now reports progress. `PR474`_
145
146The ``tub.port=`` option can now accept ``listen:i2p`` or
147``listen:tor`` options to use popular anonymity networks with storage
148servers. See `PR437`_
149
150The place where storage servers put shares (the "storage path") is now
151configurable (`PR472`_).
152
153A PyInstaller-based build is now available (`PR421`_). A "Docker
154compose" setup for development purposes is now available (`PR445`_).
155
156There is now a recommended workflow for Zcash-based donations to support
157storage server operators (`PR506`_).
158
159Bug Fixes in Core
160-----------------
161
162Some bugs with pidfile handling were fixed (`PR440`_ and `PR450`_)
163meaning invalid pidfiles are now deleted. Error-messages related to
164``tahoe.cfg`` now include the full path to the file. `PR501`_ fixes
165"address already in use" test failures. `PR502`_ fixes ticket #2926
166("tahoe status" failures). `PR487`_ fixes ticket #1455 (setting
167``X-Frame-Options: DENY``)
168
169
170Web UI Changes
171--------------
172
173We set the "Referrer-Policy: no-referrer" header on all requests. The
174Welcome page now understands the JSON option (`PR430`_) and OPTIONS
175requests are handled (`PR447`_).
176
177
178Magic Folder Changes
179--------------------
180
181Multiple magic-folders in a single Tahoe client are now
182supported. Bugs with ``.backup`` files have been fixed, meaning
183spurious ``.backup`` files will be produced less often (`PR448`_,
184`PR475`_). Handling of default umask on new magic-folder files is
185fixed in `PR458`_. The user mtime value is now correctly preserved
186(`PR457`_).
187
188A bug in ``tahoe magic-folder status`` causing active operations to
189sometimes not show up is fixed (`PR461`_). If a directory is missing,
190it is created (`PR492`_).
191
192
193Raw Pull Requests
194-----------------
195
196In total, 50 Pull Requests were merged for this release, including
197contributions of code or review from 15 different GitHub users. Thanks
198everyone! A complete list of these PRs and contributions:
199
200`PR380`_: `daira`_
201`PR400`_: `meejah`_ (with `warner`_)
202`PR403`_: `meejah`_
203`PR405`_: `meejah`_ (with `warner`_)
204`PR406`_: `meejah`_ (with `warner`_)
205`PR407`_: `david415`_ (with `meejah`_, `warner`_)
206`PR409`_: `str4d`_ (with `warner`_)
207`PR410`_: `tpltnt`_ (with `warner`_)
208`PR411`_: `tpltnt`_ (with `warner`_, `meejah`_)
209`PR412`_: `tpltnt`_ (with `warner`_)
210`PR414`_: `tpltnt`_ (with `meejah`_, `warner`_)
211`PR416`_: `david415`_, `meejah`_, `markberger`_, `warner`_
212`PR417`_: `meejah`_ (with `pataquets`_, `warner`_)
213`PR418`_: `meejah`_ (with `crwood`_, `exarkun`_, `warner`_)
214`PR419`_: `tpltnt`_ (with `warner`_)
215`PR420`_: `ValdikSS`_ (with `warner`_)
216`PR421`_: `crwood`_ (with `meejah`_, `warner`_)
217`PR423`_: `warner`_
218`PR428`_: `warner`_
219`PR429`_: `exarkun`_ (with `warner`_)
220`PR430`_: `david415`_, `exarkun`_ (with `warner`_)
221`PR432`_: `exarkun`_ (with `meejah`_)
222`PR433`_: `exarkun`_ (with `warner`_)
223`PR434`_: `exarkun`_ (with `warner`_)
224`PR437`_: `warner`_
225`PR438`_: `warner`_ (with `meejah`_)
226`PR440`_: `exarkun`_, `lpirl`_ (with `meejah`_)
227`PR444`_: `AnBuKu`_ (with `warner`_)
228`PR445`_: `bookchin`_ (with `warner`_)
229`PR447`_: `meejah`_ (with `tpltnt`_, `meejah`_)
230`PR448`_: `meejah`_ (with `warner`_)
231`PR450`_: `exarkun`_, `meejah`_, `lpirl`_
232`PR452`_: `meejah`_ (with `tpltnt`_)
233`PR453`_: `meejah`_
234`PR454`_: `meejah`_ (with `tpltnt`_, `meejah`_, `warner`_)
235`PR455`_: `tpltnt`_ (with `meejah`_)
236`PR456`_: `meejah`_ (with `meejah`_)
237`PR457`_: `meejah`_ (with `crwood`_, `tpltnt`_)
238`PR458`_: `meejah`_ (with `tpltnt`_)
239`PR460`_: `tpltnt`_ (with `exarkun`_, `meejah`_)
240`PR462`_: `meejah`_ (with `crwood`_)
241`PR464`_: `meejah`_
242`PR470`_: `meejah`_ (with `exarkun`_, `tpltnt`_, `warner`_)
243`PR472`_: `exarkun`_, `meskio`_
244`PR474`_: `exarkun`_
245`PR475`_: `meejah`_ (with `exarkun`_)
246`PR482`_: `crwood`_ (with `warner`_)
247`PR485`_: `warner`_
248`PR486`_: `exarkun`_ (with `warner`_)
249`PR487`_: `exarkun`_ (with `tpltnt`_)
250`PR489`_: `exarkun`_
251`PR490`_: `exarkun`_
252`PR491`_: `exarkun`_ (with `meejah`_)
253`PR492`_: `exarkun`_ (with `meejah`_, `tpltnt`_)
254`PR493`_: `exarkun`_ (with `meejah`_)
255`PR494`_: `exarkun`_ (with `meejah`_)
256`PR497`_: `meejah`_ (with `multikatt`_, `exarkun`_)
257`PR499`_: `exarkun`_ (with `meejah`_)
258`PR501`_: `exarkun`_ (with `meejah`_)
259`PR502`_: `exarkun`_ (with `meejah`_)
260`PR506`_: `exarkun`_ (with `crwood`_, `nejucomo`_)
261
262
263Developer and Internal Changes
264------------------------------
265
266People hacking on Tahoe-LAFS code will be interested in some internal
267improvements which shouldn't have any user-visible effects:
268
269* internal: skip some unicode tests on non-unicode platforms #2912
270* internal: tox: pre-install Incremental to workaround setuptools bug #2913
271* internal: fix PyInstaller builds `PR482`_
272* internal: use @implementer instead of implements `PR406`_
273* internal: improve happiness integration test #2895 `PR432`_
274* web internal: refactor response-format (?t=) logic #2893 `PR429`_
275* internal: fix pyflakes issues #2898 `PR434`_
276* internal: setup.py use find_packages #2897 `PR433`_
277* internal: ValueOrderedDict fixes #2891
278* internal: remove unnused NumDict #2891 `PR438`_
279* internal: setup.py use python_requires= so tox3 works #2876
280* internal: rewrite tahoe stop/start/daemonize refs #1148 #275 #1121 #1377 #2149 #719 `PR417`_
281* internal: add docs links to RFCs/etc `PR456`_
282* internal: magic-folder test improvement `PR453`_
283* internal: pytest changes `PR462`_
284* internal: upload appveyor generated wheels as artifacts #2903
285* internal: fix tox-vs-setuptools-upgrade #2910
286* deps: require txi2p>=0.3.2 to work around TLS who-is-client issue #2861 `PR409`_
287* deps: now need libyaml-dev from system before build `PR420`_
288* deps: twisted>=16.4.0 for "python -m twisted.trial" `PR454`_
289* deps: pin pypiwin32 to 219 until upstream bug resolved `PR464`_
290* deps: setuptools >=28.8.0 for something `PR470`_
291* deps: use stdlib "json" instead of external "simplejson" #2766 `PR405`_
292* complain more loudly in setup.py under py3 `PR414`_
293* rename "filesystem" to "file store" #2345 `PR380`_
294* replace deprecated twisted.web.client with treq #2857 `PR428`_
295* improve/stablize some test coverage #2891
296* TODO: can we remove this now? pypiwin32 is now at 223
297* use secure mkstemp() `PR460`_
298* test "tahoe list-aliases --readonly-uri" #2863 `PR403`_
299* #455: remove outdated comment
300* `PR407`_ fix stopService calls
301* `PR410`_ explicit python2.7 virtualenv
302* `PR419`_ fix list of supported OSes
303* `PR423`_ switch travis to a supported Ubuntu
304* deps: no longer declare a PyCrypto dependency (actual use vanished long ago) `PR514`_
305
306
307
308.. _PR380: https://github.com/tahoe-lafs/tahoe-lafs/pull/380
309.. _PR400: https://github.com/tahoe-lafs/tahoe-lafs/pull/400
310.. _PR403: https://github.com/tahoe-lafs/tahoe-lafs/pull/403
311.. _PR405: https://github.com/tahoe-lafs/tahoe-lafs/pull/405
312.. _PR406: https://github.com/tahoe-lafs/tahoe-lafs/pull/406
313.. _PR407: https://github.com/tahoe-lafs/tahoe-lafs/pull/407
314.. _PR409: https://github.com/tahoe-lafs/tahoe-lafs/pull/409
315.. _PR410: https://github.com/tahoe-lafs/tahoe-lafs/pull/410
316.. _PR412: https://github.com/tahoe-lafs/tahoe-lafs/pull/412
317.. _PR414: https://github.com/tahoe-lafs/tahoe-lafs/pull/414
318.. _PR416: https://github.com/tahoe-lafs/tahoe-lafs/pull/416
319.. _PR417: https://github.com/tahoe-lafs/tahoe-lafs/pull/417
320.. _PR418: https://github.com/tahoe-lafs/tahoe-lafs/pull/418
321.. _PR419: https://github.com/tahoe-lafs/tahoe-lafs/pull/419
322.. _PR420: https://github.com/tahoe-lafs/tahoe-lafs/pull/420
323.. _PR421: https://github.com/tahoe-lafs/tahoe-lafs/pull/421
324.. _PR423: https://github.com/tahoe-lafs/tahoe-lafs/pull/423
325.. _PR428: https://github.com/tahoe-lafs/tahoe-lafs/pull/428
326.. _PR429: https://github.com/tahoe-lafs/tahoe-lafs/pull/429
327.. _PR430: https://github.com/tahoe-lafs/tahoe-lafs/pull/430
328.. _PR432: https://github.com/tahoe-lafs/tahoe-lafs/pull/432
329.. _PR433: https://github.com/tahoe-lafs/tahoe-lafs/pull/433
330.. _PR434: https://github.com/tahoe-lafs/tahoe-lafs/pull/434
331.. _PR437: https://github.com/tahoe-lafs/tahoe-lafs/pull/437
332.. _PR438: https://github.com/tahoe-lafs/tahoe-lafs/pull/438
333.. _PR440: https://github.com/tahoe-lafs/tahoe-lafs/pull/440
334.. _PR444: https://github.com/tahoe-lafs/tahoe-lafs/pull/444
335.. _PR445: https://github.com/tahoe-lafs/tahoe-lafs/pull/445
336.. _PR447: https://github.com/tahoe-lafs/tahoe-lafs/pull/447
337.. _PR448: https://github.com/tahoe-lafs/tahoe-lafs/pull/448
338.. _PR450: https://github.com/tahoe-lafs/tahoe-lafs/pull/450
339.. _PR452: https://github.com/tahoe-lafs/tahoe-lafs/pull/452
340.. _PR453: https://github.com/tahoe-lafs/tahoe-lafs/pull/453
341.. _PR454: https://github.com/tahoe-lafs/tahoe-lafs/pull/454
342.. _PR456: https://github.com/tahoe-lafs/tahoe-lafs/pull/456
343.. _PR457: https://github.com/tahoe-lafs/tahoe-lafs/pull/457
344.. _PR458: https://github.com/tahoe-lafs/tahoe-lafs/pull/458
345.. _PR460: https://github.com/tahoe-lafs/tahoe-lafs/pull/460
346.. _PR462: https://github.com/tahoe-lafs/tahoe-lafs/pull/462
347.. _PR464: https://github.com/tahoe-lafs/tahoe-lafs/pull/464
348.. _PR470: https://github.com/tahoe-lafs/tahoe-lafs/pull/470
349.. _PR472: https://github.com/tahoe-lafs/tahoe-lafs/pull/472
350.. _PR474: https://github.com/tahoe-lafs/tahoe-lafs/pull/474
351.. _PR482: https://github.com/tahoe-lafs/tahoe-lafs/pull/482
352.. _PR502: https://github.com/tahoe-lafs/tahoe-lafs/pull/502
353.. _PR506: https://github.com/tahoe-lafs/tahoe-lafs/pull/506
354.. _PR514: https://github.com/tahoe-lafs/tahoe-lafs/pull/514
355.. _AnBuKu: https://github.com/AnBuKu
356.. _ValdikSS: https://github.com/ValdikSS
357.. _bookchin: https://github.com/bookchin
358.. _crwood: https://github.com/crwood
359.. _nejucomo: https://github.com/nejucomo
360.. _daira: https://github.com/daira
361.. _david415: https://github.com/david415
362.. _exarkun: https://github.com/exarkun
363.. _lpirl: https://github.com/lpirl
364.. _markberger: https://github.com/markberger
365.. _meejah: https://github.com/meejah
366.. _meskio: https://github.com/meskio
367.. _multikatt: https://github.com/multikatt
368.. _pataquets: https://github.com/pataquets
369.. _str4d: https://github.com/str4d
370.. _tpltnt: https://github.com/tpltnt
371.. _warner: https://github.com/warner
372
373
374
375
376Release 1.12.1 (18-Jan-2017)
377''''''''''''''''''''''''''''
378
379This fixes a few small problems discovered just after 1.12.0 was released.
380
381* ``introducers.yaml`` was entirely broken (due to a unicode-vs-ascii
382  problem), and the documentation recommended an invalid syntax. Both have
383  been fixed. (#2862)
384* Creating a node with ``--hide-ip`` shouldn't set ``tcp = tor`` if txtorcon
385  is unavailable. I2P-only systems should get ``tcp = disabled``. (#2860)
386* As a result, we now require foolscap-0.12.6 .
387* setup.py now creates identical wheels on win32 and unix. Previously wheels
388  created on windows got an unconditional dependency upon ``pypiwin32``,
389  making them uninstallable on unix. Now that dependency is marked as
390  ``sys_platform=win32`` only. (#2763)
391
392Some other small changes include:
393
394* The deep-stats t=json response now includes an "api-version" field,
395  currently set to 1. (#567)
396* WUI Directory listings use ``rel=noreferrer`` to avoid leaking the dircap
397  to the JS contents of the target file. (#151, #378)
398* Remove the dependency on ``shutilwhich`` (#2856)
399
400
401Release 1.12.0 (17-Dec-2016)
402''''''''''''''''''''''''''''
403
404New Features
405------------
406
407This release features improved Tor/I2P integration. It is now easy to::
408
409* use Tor to hide your IP address during external network activity
410* connect to Tor/I2P-based storage servers
411* run an Introducer or a storage node as a Tor "onion service"
412
413See docs/anonymity-configuration.rst for instructions and new node-creation
414arguments (--hide-ip, --listen=tor), which include ways to use SOCKS servers
415for outbound connections. Tor/I2P/Socks support requires extra python
416libraries to be installed (e.g. 'pip install tahoe-lafs[tor]'), as well as
417matching (non-python) daemons available on the host system. (tickets #517,
418#2490, #2838)
419
420Nodes can use multiple introducers by adding entries to a new
421``private/introducers.yaml`` file, or stop using introduction entirely by
422omitting the ``introducer.furl`` key from tahoe.cfg (introducerless clients
423will need static servers configured to connect anywhere). Server
424announcements are sent to all connected Introducers, and clients merge all
425announcements they see, which can improve grid reliability. (#68)
426
427In addition, nodes now cache the announcements they receive in a YAML file,
428and use their cached information at startup until the Introducer connection
429is re-established. This makes nodes more tolerant of Introducers that are
430temporarily offline. Nodes admins can copy text from the cache into a new
431``private/servers.yaml`` file to add "static servers", which augment/override
432what the Introducer offers. This can modify aspects of the server, or use
433servers that were never announced in the first place. (#2788)
434
435Nodes now use a separate Foolscap "Tub" for each server connection, so
436``servers.yaml`` can override the connection rules (Tor vs direct-TCP) for
437each one independently. This offers a slight privacy improvement, but slows
438down connections slightly (perhaps 75ms per server), and breaks an obscure
439NAT-bypass trick which enabled storage servers to run behind NAT boxes (but
440only when all the *clients* of the storage server had public IP addresses,
441and were also configured as servers). (#2759, #517)
442
443"Magic Folders" is an experimental two-way directory synchronization tool,
444contributed by Least Authority Enterprises, which replaces the previous
445experimental (one-way) "drop-upload" feature. This allows multiple users to
446keep a single directory in-sync, using Tahoe as the backing store. See
447docs/frontends/magic-folder.rst for details and configuration instructions.
448
449Compatibility Issues
450--------------------
451
452The old version-1 Introducer protocol has been removed. Tahoe has used the
453version-2 protocol since 1.10 (released in 2013), but all nodes (clients,
454servers, and the Introducer itself) provided backwards-compatibility
455translations when encountering older peers. These translations were finally
456removed, so Tahoe nodes at 1.12 or later will not be able to interact with
457nodes at 1.9 or older. (#2784)
458
459The versions of Tahoe (1.11.0) and Foolscap (0.6.5) that shipped in
460Debian/Jesse (the most recent stable release, as of December 2016) are
461regrettably not forwards-compatible with this new version. Nodes running
462Jesse will not be able to connect to servers or introducers created with this
463release because they cannot parse the new ``tcp:HOST:PORT`` hint syntax (this
464syntax has been around for a while, but this is the first Tahoe release to
465automatically generate such hints). If you need to work around this, then
466after creating your new node, edit the tahoe.cfg of your new
467server/introducer: in ``[node] tub.location``, make each connection hint look
468like ``HOST:PORT`` instead of ``tcp:HOST:PORT``. If your grid only has nodes
469with Foolscap-0.7.0 or later, you will not need this workaround. (#2831)
470
471Nodes now use an Ed25519 public key as a serverid, instead of a Foolscap "tub
472id", so status displays will report a different serverid after upgrade. For
473the most part this should be self-consistent, however if you have an old
474(1.11) client talking to a new (1.12) Helper, then the client's upload
475results (on the "Recent Uploads And Downloads" web page) will show unusual
476server ids. (#1363)
477
478Dependency/Installation changes
479-------------------------------
480
481Tahoe now requires Twisted >= 16.1.0, so ensure that unit tests do not fail
482because of uncancelled timers left running by HostnameEndpoint. It also
483requires the Tor/I2P supporting code from Foolscap >= 0.12.5 . (#2781)
484
485Configuration Changes
486---------------------
487
488Some small changes were made to the way Tahoe-LAFS is configured, via
489``tahoe.cfg`` and other files. In general, node behavior should now be more
490predictable, and errors should be surfaced earlier.
491
492* ``tub.port`` is now an Endpoint server specification string (which is
493  pretty much just like a strports string, but can be extended by plugins).
494  It now rejects "tcp:0" and "0". The tahoe.cfg value overrides anything
495  stored on disk (in client.port). This should have no effect on most old
496  nodes (which did not set tub.port in tahoe.cfg, and which wrote an
497  allocated port number to client.port the first time they launched). Folks
498  who want to listen on a specific port number typically set tub.port to
499  "tcp:12345" or "12345", not "0". (ticket #2491)
500* This should enable IPv6 on servers, either via AAAA records or colon-hex
501  addresses. (#2827)
502* The "portnumfile" (e.g. NODEDIR/client.port) is written as soon as the port
503  is allocated, before the tub is created, and only if "tub.port" was empty.
504  The old code wrote to it unconditionally, and after Tub startup. So if the
505  user allows NODEDIR/client.port to be written, then later modifies
506  tahoe.cfg to set "tub.port" to a different value, this difference will
507  persist (and the node will honor tahoe.cfg "tub.port" exclusively).
508* We now encourage static allocation of tub.port, and pre-configuration of
509  the node's externally-reachable IP address or hostname (by setting
510  tub.location). Automatic IP-address detection is deprecated. Automatic port
511  allocation is discouraged. Both are managed by the new arguments to "tahoe
512  create-node".
513* "tahoe start" now creates the Tub, and all primary software components,
514  before the child process daemonizes. Many configuration errors which would
515  previously have been reported in a logfile (after node startup), will now
516  be signalled immediately, via stderr. In these cases, the "tahoe start"
517  process will exit with a non-zero return code. (#2491)
518* Unrecognized tahoe.cfg options are rejected at startup, not ignored (#2809)
519* ``tub.port`` can take multple (comma-separated) endpoints, to listen on
520  multiple ports at the same time, useful for dual IPv4+IPv6 servers. (#867)
521* An empty ``tub.port`` means don't listen at all, which is appropriate for
522  client-only nodes (#2816)
523* A new setting, ``reveal-ip-address = false``, acts as a safety belt,
524  causing an error to be thrown if any other setting might reveal the node's
525  IP address (i.e. it requires Tor or I2P to be used, rather than direct TCP
526  connections). This is set automatically by ``tahoe create-client
527  --hide-ip``. (#1010)
528
529Server-like nodes (Storage Servers and Introducers), created with ``tahoe
530create-node`` and ``tahoe create-introducer``, now accept new arguments to
531control how they listen for connections, and how they advertise themselves to
532other nodes. You can use ``--listen=tcp`` and ``--hostname=`` to choose a
533port automatically, or ``--listen=tor`` / ``--listen=i2p`` to use Tor/I2P
534hidden services instead. You can also use ``--port=`` and ``--location=`` to
535explicitly control the listener and the advertised location. (#2773, #2490)
536
537The "stats-gatherer", used by enterprise deployments to collect runtime
538statistics from a fleet of Tahoe storage servers, must now be assigned a
539hostname, or location+port pair, at creation time. It will no longer attempt
540to guess its location (with /sbin/ifconfig). The "tahoe
541create-stats-gatherer" command requires either "--hostname=", or both
542"--location=" and "--port". (#2773)
543
544To keep your old stats-gatherers working, with their original FURL, you must
545determine a suitable --location and --port, and write their values into
546NODEDIR/location and NODEDIR/port, respectively. Or you could simply rebuild
547it by re-running "tahoe create-stats-gatherer" with the new arguments.
548
549The stats gatherer now updates a JSON file named "stats.json", instead of a
550Pickle named "stats.pickle". The munin plugins in
551misc/operations_helpers/munin/ have been updated to match, and must be
552re-installed and re-configured if you use munin.
553
554Removed Features
555----------------
556
557The "key-generator" node type has been removed. This was a standalone process
558that maintained a queue of RSA keys, and clients could offload their
559key-generation work by adding "key_generator.furl=" in their tahoe.cfg files,
560to create mutable files and directories faster. This seemed important back in
5612006, but these days computers are faster and RSA key generation only takes
562about 90ms. This removes the "tahoe create-key-generator" command. Any
563"key_generator.furl" settings in tahoe.cfg will log a warning and are
564otherwise ignored. Attempts to "tahoe start" a previously-generated
565key-generator node will result in an error. (#2783)
566
567Tahoe's HTTP Web-API (aka "the WAPI") had an endpoint named "/file/". This
568has been deprecated, and applications should use "/named/" instead. (#1903)
569
570The little-used "manhole" debugging feature has been removed. This allowed
571you to SSH or Telnet "into" a Tahoe node, providing an interactive
572Read-Eval-Print-Loop (REPL) that executed inside the context of the running
573process. (#2367)
574
575The "tahoe debug trial" and "tahoe debug repl" CLI commands were removed, as
576"tox" is now the preferred way to run tests. (#2735)
577
578One of the "recent uploads and downloads" status pages was using a
579Google-hosted API to draw a timing chart of the "mapupdate" operation. This
580has been removed, both for privacy (to avoid revealing the serverids to
581Google) and because the API was deprecated several years ago. (#1942)
582
583The "_appname.py" feature was removed. Early in Tahoe's history (at
584AllMyData), this file allowed the "tahoe" executable to be given a different
585name depending upon which Darcs patches were included in the particular
586source tree (one for production, another for development, etc). We haven't
587needed this for a long time, so it was removed. (#2754)
588
589Other Changes
590-------------
591
592Documentation is now hosted at http://tahoe-lafs.readthedocs.io/ (not .org).
593
594Tahoe's testing-only dependencies can now be installed by asking for the
595[test] extra, so if you want to set up a virtualenv for testing, use "pip
596install -e .[test]" instead just of "pip install -e ." . This includes "tox",
597"coverage", "pyflakes", "mock", and all the Tor/I2P extras. Most developer
598tooling (code-checks, documentation builds, deprecation warnings, etc) have
599been moved from a Makefile into tox environments. (#2776)
600
601The "Welcome" (web) page now shows more detail about the introducer and
602storage-server connections, including which connection handler is being used
603(tcp/tor/i2p) and why specific connection hints failed to connect. (#2818,
604#2819)
605
606The little-used "control port" now uses a separate (ephemeral) Tub. This
607means the FURL changes each time the node is restarted, and it only listens
608on the loopback (127.0.0.1) interface, on a random port. As the control port
609is only used by some automated tests (check_memory, check_speed), this
610shouldn't affect anyone. (#2794)
611
612The slightly-more-used "log port" now also uses a separate (ephemeral) Tub,
613with the same consequences. The lack of a stable (and externally-reachable)
614logport.furl means it is no longer possible to use ``flogtool tail FURL``
615against a distant Tahoe server, however ``flogtool tail
616.../nodedir/private/logport.furl`` still works just fine (and is the more
617common use case anyways). We might bring back the ability to configure the
618port and location of the logport in the future, if there is sufficient
619demand, but for now it seems better to avoid the complexity.
620
621The default tahoe.cfg setting of ``web.static = public_html``, when
622``NODEDIR/public_html/`` does not exist, no longer causes web browsers to
623display a traceback which reveals somewhat-private information like the value
624of NODEDIR, and the Python/OS versions in use. Instead it just shows a plain
625404 error. (#1720)
626
627
628Release 1.11.0 (30-Mar-2016)
629''''''''''''''''''''''''''''
630
631New Build Process
632-----------------
633
634``pip install`` (in a virtualenv) is now the recommended way to install
635Tahoe-LAFS. The old "bin/tahoe" script (created inside the source tree,
636rather than in a virtualenv) has been removed, as has the ancient
637"zetuptoolz" fork of setuptools.
638
639Tahoe was started in 2006, and predates pip and virtualenv. From the
640very beginning it used a home-made build process that attempted to make
641``setup.py build`` behave somewhat like a modern ``pip
642install --editable .``. It included a local copy of ``setuptools`` (to
643avoid requiring it to be pre-installed), which was then forked as
644``zetuptoolz`` to fix bugs during the bad old days of setuptools
645non-maintenance. The pseudo-virtualenv used a script named
646``bin/tahoe``, created during ``setup.py build``, to set up the $PATH
647and $PYTHONPATH as necessary.
648
649Starting with this release, all the custom build process has been
650removed, and Tahoe should be installable with standard modern tools. You
651will need ``virtualenv`` installed (which provides ``pip`` and
652setuptools). Many Python installers include ``virtualenv`` already, and
653Debian-like systems can use ``apt-get install python-virtualenv``. If
654the command is not available on your system, follow the installation
655instructions at https://virtualenv.pypa.io/en/latest/ .
656
657Then, to install the latest version, create a virtualenv and use
658``pip``::
659
660    virtualenv venv
661    . venv/bin/activate
662    (venv) pip install tahoe-lafs
663    (venv) tahoe --version
664
665To run Tahoe from a source checkout (so you can hack on Tahoe), use
666``pip install --editable .`` from the git tree::
667
668    git clone https://github.com/tahoe-lafs/tahoe-lafs.git
669    cd tahoe-lafs
670    virtualenv venv
671    . venv/bin/activate
672    (venv) pip install --editable .
673    (venv) tahoe --version
674
675The ``pip install`` will download and install all necessary Python
676dependencies. Some dependencies require a C compiler and system
677libraries to build: on Debian/Ubuntu-like systems, use ``apt-get install
678build-essential python-dev libffi-dev libssl-dev``. On Windows and OS-X
679platforms, we provide pre-compiled binary wheels at
680``https://tahoe-lafs.org/deps/``, removing the need for a compiler.
681
682(#1582, #2445, also helped to close: #142, #709, #717, #799, #1220,
683#1260, #1270, #1403, #1450, #1451, #1504, #1896, #2044, #2221, #2021,
684#2028, #2066, #2077, #2247, #2255, #2286, #2306, #2473, #2475, #2530,
685#657, #2446, #2439, #2317, #1753, #1009, #1168, #1238, #1258, #1334,
686#1346, #1464, #2356, #2570)
687
688New PyPI Distribution Name
689--------------------------
690
691Tahoe-LAFS is now known on PyPI as ``tahoe-lafs``. It was formerly known
692as ``allmydata-tahoe``. This affects ``pip install`` commands. (#2011)
693
694Because of this change, if you use a git checkout, you may need to run
695``make distclean`` (to delete the machine-generated
696``src/allmydata/_appname.py`` file). You may also need to remove
697``allmydata-tahoe`` from any virtualenvs you've created, before
698installing ``tahoe-lafs`` into them. If all else fails, make a new git
699checkout, and use a new virtualenv.
700
701Note that the importable *package* name is still ``allmydata``, but this
702only affects developers, not end-users. This name scheduled to be
703changed in a future release. (#1950)
704
705
706Compatibility and Dependency Updates
707------------------------------------
708
709Tahoe now requires Python 2.7 on all platforms. (#2445)
710
711Tahoe now requires Foolscap 0.10.1, which fixes incompatibilities with
712recent Twisted releases. (#2510, #2722, #2567)
713
714Tahoe requires Twisted 15.1.0 or later, so it can request the
715``Twisted[tls]`` "extra" (this asks Twisted to ask for everything it
716needs to provide proper TLS support). (#2760)
717
718Tests should now work with both Nevow 0.11 and 0.12 . (#2663)
719
720Binary wheels for Windows and OS-X (for all dependencies) have been
721built and are hosted at https://tahoe-lafs.org/deps . Use ``pip
722install --find-links=URL tahoe-lafs`` to take advantage of them. (#2001)
723
724We've removed the SUMO and tahoe-deps tarballs. Please see
725docs/desert-island.rst for instructions to build tahoe from offline
726systems. (#1009, #2530, #2446, #2439)
727
728Configuration Changes
729---------------------
730
731A new "peers.preferred" item was added to the ``[client]`` section. This
732identifies servers that will be promoted to the front of the
733peer-selection list when uploading or downloading files. Servers are
734identified by their Node ID (visible on the welcome page). This may be
735useful to ensure that one full set of shares are placed on nearby
736servers, making later downloads fast (and avoid using scarce remote
737bandwidth). The remaining shares can go to distant backup servers. (git
738commit 96eaca6)
739
740Aliases can now be unicode. (git commit 46719a8b)
741
742The introducer's "set_encoding_parameters" feature was removed. Once
743upon a time, the Introducer could recommend encoding parameters
744(shares.needed and shares.total) to all clients, the idea being that the
745Introducer had a slightly better idea about the expected size of the
746storage server pool than clients might. Client-side support for this was
747removed long ago, but the Introducer itself kept delivering
748recommendations until this release. (git commit 56a9f5ad)
749
750Other Fixes
751-----------
752
753The OS-X .pkg installer has been improved slightly, to clean up after
754previous installations better. (#2493)
755
756All WUI (Web UI) timestamps should now be a consistent format, using the
757gateway machine's local time zone. (#1077)
758
759The web "welcome page" has been improved: it shows how long a server has
760been connected (in minutes/hours/days, instead of the date+time when the
761connection was most recently established). The "announced" column has
762been replaced with "Last RX" column that shows when we last heard
763anything from the server. The mostly-useless "storage" column has been
764removed. (#1973)
765
766In the ``tahoe ls`` command, the ``-u`` shortcut for ``--uri`` has been
767removed, leaving the shortcut free for the global ``--node-url`` option.
768(#1949, #2137)
769
770Some internal logging was disabled, to avoid a temporary bug that caused
771excessive (perhaps infinite) log messages to be created. (#2567)
772
773Other non-user-visible tickets were fixed. (#2499, #2511, #2556, #2663,
774#2723, #2543)
775
776
777Release 1.10.2 (2015-07-30)
778'''''''''''''''''''''''''''
779
780Packaging Changes
781-----------------
782
783This release no longer requires the ``mock`` library (which was previously
784used in the unit test suite). Shortly after the Tahoe-LAFS 1.10.1 release, a
785new version of ``mock`` was released (1.1.0) that proved to be incompatible
786with Tahoe's fork of setuptools, preventing Tahoe-1.10.1 from building at
787all. `#2465`_
788
789The ``tahoe --version`` output is now less likely to include scary diagnostic
790warnings that look deceptively like stack traces. `#2436`_
791
792The pyasn1 requirement was increased to >= 0.1.8.
793
794.. _`#2465`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2465
795.. _`#2436`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2436
796
797Other Fixes
798-----------
799
800A WebAPI ``GET`` would sometimes hang when using the HTTP Range header to
801read just part of the file. `#2459`_
802
803Using ``tahoe cp`` to copy two different files of the same name into the same
804target directory now raises an error, rather than silently overwriting one of
805them. `#2447`_
806
807All tickets closed in this release: 2328 2436 2446 2447 2459 2460 2461 2462
8082465 2470.
809
810.. _`#2459`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2459
811.. _`#2447`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2447
812
813
814Release 1.10.1 (2015-06-15)
815'''''''''''''''''''''''''''
816
817User Interface / Configuration Changes
818--------------------------------------
819
820The "``tahoe cp``" CLI command's ``--recursive`` option is now more predictable,
821but behaves slightly differently than before. See below for details. Tickets
822`#712`_, `#2329`_.
823
824The SFTP server can now use public-key authentication (instead of only
825password-based auth). Public keys are configured through an "account file",
826just like passwords. See docs/frontends/FTP-and-SFTP for examples of the
827format. `#1411`_
828
829The Tahoe node can now be configured to disable automatic IP-address
830detection. Using "AUTO" in tahoe.cfg [node]tub.location= (which is now the
831default) triggers autodetection. Omit "AUTO" to disable autodetection. "AUTO"
832can be combined with static addresses to e.g. use both a stable
833UPnP-configured tunneled address and a DHCP-assigned dynamic (local subnet
834only) address. See `configuration.rst`_ for details. `#754`_
835
836The web-based user interface ("WUI") Directory and Welcome pages have been
837redesigned, with improved CSS for narrow windows and more-accessible icons
838(using distinctive shapes instead of just colors). `#1931`_ `#1961`_ `#1966`_
839`#1972`_ `#1901`_
840
841.. _`#712`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/712
842.. _`#754`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/754
843.. _`#1411`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1411
844.. _`#1901`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1901
845.. _`#1931`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1931
846.. _`#1961`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1961
847.. _`#1966`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1966
848.. _`#1972`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1972
849.. _`#2329`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2329
850.. _`configuration.rst`: docs/configuration.rst
851
852"tahoe cp" changes
853------------------
854
855The many ``cp``-like tools in the Unix world (POSIX ``/bin/cp``, the ``scp``
856provided by SSH, ``rsync``) all behave slightly differently in unusual
857circumstances, especially when copying whole directories into a target that
858may or may not already exist. The most common difference is whether the user
859is referring to the source directory as a whole, or to its contents. For
860example, should "``cp -r foodir bardir``" create a new directory named
861"``bardir/foodir``"? Or should it behave more like "``cp -r foodir/* bardir``"?
862Some tools use the presence of a trailing slash to indicate which behavior
863you want. Others ignore trailing slashes.
864
865"``tahoe cp``" is no exception to having exceptional cases. This release fixes
866some bad behavior and attempts to establish a consistent rationale for its
867behavior. The new rule is:
868
869- If the thing being copied is a directory, and it has a name (e.g. it's not
870  a raw Tahoe-LAFS directorycap), then you are referring to the directory
871  itself.
872- If the thing being copied is an unnamed directory (e.g. raw dircap or
873  alias), then you are referring to the contents.
874- Trailing slashes do not affect the behavior of the copy (although putting
875  a trailing slash on a file-like target is an error).
876- The "``-r``" (``--recursive``) flag does not affect the behavior of the
877  copy (although omitting ``-r`` when the source is a directory is an error).
878- If the target refers to something that does not yet exist:
879  - and if the source is a single file, then create a new file;
880  - otherwise, create a directory.
881
882There are two main cases where the behavior of Tahoe-LAFS v1.10.1 differs
883from that of the previous v1.10.0 release:
884
885- "``cp DIRCAP/file.txt ./local/missing``" , where "``./local``" is a
886  directory but "``./local/missing``" does not exist. The implication is
887  that you want Tahoe to create a new file named "``./local/missing``" and
888  fill it with the contents of the Tahoe-side ``DIRCAP/file.txt``. In
889  v1.10.0, a plain "``cp``" would do just this, but "``cp -r``" would do
890  "``mkdir ./local/missing``" and then create a file named
891  "``./local/missing/file.txt``". In v1.10.1, both "``cp``" and "``cp -r``"
892  create a file named "``./local/missing``".
893- "``cp -r PARENTCAP/dir ./local/missing``", where ``PARENTCAP/dir/``
894  contains "``file.txt``", and again "``./local``" is a directory but
895  "``./local/missing``" does not exist. In both v1.10.0 and v1.10.1, this
896  first does "``mkdir ./local/missing``". In v1.10.0, it would then copy
897  the contents of the source directory into the new directory, resulting
898  in "``./local/missing/file.txt``". In v1.10.1, following the new rule
899  of "a named directory source refers to the directory itself", the tool
900  creates "``./local/missing/dir/file.txt``".
901
902Compatibility and Dependency Updates
903------------------------------------
904
905Windows now requires Python 2.7. Unix/OS-X platforms can still use either
906Python 2.6 or 2.7, however this is probably the last release that will
907support 2.6 (it is no longer receiving security updates, and most OS
908distributions have switched to 2.7). Tahoe-LAFS now has the following
909dependencies:
910
911- Twisted >= 13.0.0
912- Nevow >= 0.11.1
913- foolscap >= 0.8.0
914- service-identity
915- characteristic >= 14.0.0
916- pyasn1 >= 0.1.4
917- pyasn1-modules >= 0.0.5
918
919On Windows, if pywin32 is not installed then the dependencies on Twisted
920and Nevow become:
921
922- Twisted >= 11.1.0, <= 12.1.0
923- Nevow >= 0.9.33, <= 0.10
924
925On all platforms, if pyOpenSSL >= 0.14 is installed, then it will be used,
926but if not then only pyOpenSSL >= 0.13, <= 0.13.1 will be built when directly
927invoking `setup.py build` or `setup.py install`.
928
929We strongly advise OS packagers to take the option of making a tahoe-lafs
930package depend on pyOpenSSL >= 0.14. In order for that to work, the following
931additional Python dependencies are needed:
932
933- cryptography
934- cffi >= 0.8
935- six >= 1.4.1
936- enum34
937- pycparser
938
939as well as libffi (for Debian/Ubuntu, the name of the needed OS package is
940`libffi6`).
941
942Tahoe-LAFS is now compatible with Setuptools version 8 and Pip version 6 or
943later, which should fix execution on Ubuntu 15.04 (it now tolerates PEP440
944semantics in dependency specifications). `#2354`_ `#2242`_
945
946Tahoe-LAFS now depends upon foolscap-0.8.0, which creates better private keys
947and certificates than previous versions. To benefit from the improvements
948(2048-bit RSA keys and SHA256-based certificates), you must re-generate your
949Tahoe nodes (which changes their TubIDs and FURLs). `#2400`_
950
951.. _`#2242`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2242
952.. _`#2354`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2354
953.. _`#2400`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2400
954
955Packaging
956---------
957
958A preliminary OS-X package, named "``tahoe-lafs-VERSION-osx.pkg``", is now
959being generated. It is a standard double-clickable installer, which creates
960``/Applications/tahoe.app`` that embeds a complete runtime tree. However
961launching the ``.app`` only brings up a notice on how to run tahoe from the
962command line. A future release may turn this into a fully-fledged application
963launcher. `#182`_ `#2393`_ `#2323`_
964
965Preliminary Docker support was added. Tahoe container images may be available
966on DockerHub. `PR#165`_ `#2419`_ `#2421`_
967
968Old and obsolete Debian packaging tools have been removed. `#2282`_
969
970.. _`#182`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/182
971.. _`#2282`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2282
972.. _`#2323`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2323
973.. _`#2393`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2393
974.. _`#2419`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2419
975.. _`#2421`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2421
976.. _`PR#165`: https://github.com/tahoe-lafs/tahoe-lafs/pull/165
977
978Minor Changes
979-------------
980
981- Welcome page: add per-server "(space) Available" column. `#648`_
982- check/deep-check learned to accept multiple location arguments. `#740`_
983- Checker reports: remove needs-rebalancing, add count-happiness. `#1784`_ `#2105`_
984- CLI ``--help``: cite (but don't list) global options on each command. `#2233`_
985- Fix ftp "``ls``" to work with Twisted 15.0.0. `#2394`_
986
987.. _`#648`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/648
988.. _`#740`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/740
989.. _`#1784`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1784
990.. _`#2105`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2105
991.. _`#2233`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2233
992.. _`#2394`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2394
993
994Roughly 75 tickets were closed in this release: 623 648 712 740 754 898 1146
9951159 1336 1381 1411 1634 1674 1698 1707 1717 1737 1784 1800 1807 1842 1847
9961901 1918 1953 1960 1961 1966 1969 1972 1974 1988 1992 2005 2008 2023 2027
9972028 2034 2048 2067 2086 2105 2121 2128 2165 2193 2208 2209 2233 2235 2242
9982245 2248 2249 2249 2280 2281 2282 2290 2305 2312 2323 2340 2354 2380 2393
9992394 2398 2400 2415 2416 2417 2433. Another dozen were referenced but not
1000closed: 182 666 982 1064 1258 1531 1536 1742 1834 1931 1935 2286. Roughly 40
1001GitHub pull-requests were closed: 32 48 50 56 57 61 62 62 63 64 69 73 81 82
100284 85 87 91 94 95 96 103 107 109 112 114 120 122 125 126 133 135 136 137 142
1003146 149 152 165.
1004
1005For more information about any ticket, visit e.g.
1006https://tahoe-lafs.org/trac/tahoe-lafs/ticket/754
1007
1008
1009Release 1.10.0 (2013-05-01)
1010'''''''''''''''''''''''''''
1011
1012New Features
1013------------
1014
1015- The Welcome page has been redesigned. This is a preview of the design style
1016  that is likely to be used in other parts of the WUI in future Tahoe-LAFS
1017  versions. (`#1713`_, `#1457`_, `#1735`_)
1018- A new extensible Introducer protocol has been added, as the basis for
1019  future improvements such as accounting. Compatibility with older nodes is
1020  not affected. When server, introducer, and client are all upgraded, the
1021  welcome page will show node IDs that start with "v0-" instead of the old
1022  tubid. See `<docs/nodekeys.rst>`__ for details. (`#466`_)
1023- The web-API has a new ``relink`` operation that supports directly moving
1024  files between directories. (`#1579`_)
1025
1026Security Improvements
1027---------------------
1028
1029- The ``introducer.furl`` for new Introducers is now unguessable. In previous
1030  releases, this FURL used a predictable swissnum, allowing a network
1031  eavesdropper who observes any node connecting to the Introducer to access
1032  the Introducer themselves, and thus use servers or offer storage service to
1033  clients (i.e. "join the grid"). In the new code, the only way to join a
1034  grid is to be told the ``introducer.furl`` by someone who already knew it.
1035  Note that pre-existing introducers are not changed. To force an introducer
1036  to generate a new FURL, delete the existing ``introducer.furl`` file and
1037  restart it. After doing this, the ``[client]introducer.furl`` setting of
1038  every client and server that should connect to that introducer must be
1039  updated. Note that other users of a shared machine may be able to read
1040  ``introducer.furl`` from your ``tahoe.cfg`` file unless you configure the
1041  file permissions to prevent them. (`#1802`_)
1042- Both ``introducer.furl`` and ``helper.furl`` are now censored from the
1043  Welcome page, to prevent users of your gateway from learning enough to
1044  create gateway nodes of their own.  For existing guessable introducer
1045  FURLs, the ``introducer`` swissnum is still displayed to show that a
1046  guessable FURL is in use. (`#860`_)
1047
1048Command-line Syntax Changes
1049---------------------------
1050
1051- Global options to ``tahoe``, such as ``-d``/``--node-directory``, must now
1052  come before rather than after the command name (for example,
1053  ``tahoe -d BASEDIR cp -r foo: bar:`` ). (`#166`_)
1054
1055Notable Bugfixes
1056----------------
1057
1058- In earlier versions, if a connection problem caused a download failure for
1059  an immutable file, subsequent attempts to download the same file could also
1060  fail. This is now fixed. (`#1679`_)
1061- Filenames in WUI directory pages are now displayed correctly when they
1062  contain characters that require HTML escaping. (`#1143`_)
1063- Non-ASCII node nicknames no longer cause WUI errors. (`#1298`_)
1064- Checking a LIT file using ``tahoe check`` no longer results in an
1065  exception. (`#1758`_)
1066- The SFTP frontend now works with recent versions of Twisted, rather than
1067  giving errors or warnings about use of ``IFinishableConsumer``. (`#1926`_,
1068  `#1564`_, `#1525`_)
1069- ``tahoe cp --verbose`` now counts the files being processed correctly.
1070  (`#1805`_, `#1783`_)
1071- Exceptions no longer trigger an unhelpful crash reporter on Ubuntu 12.04
1072  ("Precise") or later. (`#1746`_)
1073- The error message displayed when a CLI tool cannot connect to a gateway has
1074  been improved. (`#974`_)
1075- Other minor fixes: `#1781`_, `#1812`_, `#1915`_, `#1484`_, `#1525`_
1076
1077Compatibility and Dependencies
1078------------------------------
1079
1080- Python >= 2.6, except Python 3 (`#1658`_)
1081- Twisted >= 11.0.0 (`#1771`_)
1082- mock >= 0.8 (for unit tests)
1083- pycryptopp >= 0.6.0 (for Ed25519 signatures)
1084- zope.interface >= 3.6.0 (except 3.6.3 or 3.6.4)
1085
1086Other Changes
1087-------------
1088
1089- The ``flogtool`` utility, used to read detailed event logs, can now be
1090  accessed as ``tahoe debug flogtool`` even when Foolscap is not installed
1091  system-wide. (`#1693`_)
1092- The provisioning/reliability pages were removed from the main client's web
1093  interface, and moved into a standalone web-based tool. Use the ``run.py``
1094  script in ``misc/operations_helpers/provisioning/`` to access them.
1095- Web clients can now cache (ETag) immutable directory pages. (`#443`_)
1096- `<docs/convergence_secret.rst>`__ was added to document the adminstration
1097  of convergence secrets. (`#1761`_)
1098
1099Precautions when Upgrading
1100--------------------------
1101
1102- When upgrading a grid from a recent revision of trunk, follow the
1103  precautions from this `message to the tahoe-dev mailing list`_, to ensure
1104  that announcements to the Introducer are recognized after the upgrade.
1105  This is not necessary when upgrading from a previous release like 1.9.2.
1106
1107.. _`#166`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/166
1108.. _`#443`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/443
1109.. _`#466`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/466
1110.. _`#860`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/860
1111.. _`#974`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/974
1112.. _`#1143`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1143
1113.. _`#1298`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1298
1114.. _`#1457`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1457
1115.. _`#1484`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1484
1116.. _`#1525`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1525
1117.. _`#1564`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1564
1118.. _`#1579`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1579
1119.. _`#1658`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1658
1120.. _`#1679`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1679
1121.. _`#1693`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1693
1122.. _`#1713`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1713
1123.. _`#1735`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1735
1124.. _`#1746`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1746
1125.. _`#1758`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1758
1126.. _`#1761`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1761
1127.. _`#1771`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1771
1128.. _`#1781`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1781
1129.. _`#1783`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1783
1130.. _`#1802`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1802
1131.. _`#1805`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1805
1132.. _`#1812`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1812
1133.. _`#1915`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1915
1134.. _`#1926`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1926
1135.. _`message to the tahoe-dev mailing list`:
1136             https://tahoe-lafs.org/pipermail/tahoe-dev/2013-March/008096.html
1137
1138
1139Release 1.9.2 (2012-07-03)
1140''''''''''''''''''''''''''
1141
1142Notable Bugfixes
1143----------------
1144
1145- Several regressions in support for reading (`#1636`_), writing/modifying
1146  (`#1670`_, `#1749`_), verifying (`#1628`_) and repairing (`#1655`_, `#1669`_,
1147  `#1676`_, `#1689`_) mutable files have been fixed.
1148- FTP can now list directories containing mutable files, although it
1149  still does not support reading or writing mutable files. (`#680`_)
1150- The FTP frontend would previously show Jan 1 1970 for all timestamps;
1151  now it shows the correct modification time of the directory entry.
1152  (`#1688`_)
1153- If a node is configured to report incidents to a log gatherer, but the
1154  gatherer is offline when some incidents occur, it would previously not
1155  "catch up" with those incidents as intended. (`#1725`_)
1156- OpenBSD 5 is now supported. (`#1584`_)
1157- The ``count-good-share-hosts`` field of file check results is now
1158  computed correctly. (`#1115`_)
1159
1160Configuration/Behavior Changes
1161------------------------------
1162
1163- The capability of the upload directory for the drop-upload frontend
1164  is now specified in the file ``private/drop_upload_dircap`` under
1165  the gateway's node directory, rather than in its ``tahoe.cfg``.
1166  (`#1593`_)
1167
1168Packaging Changes
1169-----------------
1170
1171- Tahoe-LAFS can be built correctly from a git repository as well as
1172  from darcs.
1173
1174Compatibility and Dependencies
1175------------------------------
1176
1177- foolscap >= 0.6.3 is required, in order to make Tahoe-LAFS compatible
1178  with Twisted >= 11.1.0. (`#1788`_)
1179- Versions 2.0.1 and 2.4 of PyCrypto are excluded. (`#1631`_, `#1574`_)
1180
1181.. _`#680`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/680
1182.. _`#1115`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1115
1183.. _`#1574`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1574
1184.. _`#1584`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1584
1185.. _`#1593`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1593
1186.. _`#1628`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1628
1187.. _`#1631`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1631
1188.. _`#1636`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1636
1189.. _`#1655`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1655
1190.. _`#1669`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1669
1191.. _`#1670`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1670
1192.. _`#1676`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1676
1193.. _`#1688`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1688
1194.. _`#1689`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1689
1195.. _`#1725`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1725
1196.. _`#1749`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1749
1197.. _`#1788`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1788
1198
1199
1200Release 1.9.1 (2012-01-12)
1201''''''''''''''''''''''''''
1202
1203Security-related Bugfix
1204-----------------------
1205
1206- Fix flaw that would allow servers to cause undetected corruption when
1207  retrieving the contents of mutable files (both SDMF and MDMF). (`#1654`_)
1208
1209.. _`#1654`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1654
1210
1211
1212Release 1.9.0 (2011-10-30)
1213''''''''''''''''''''''''''
1214
1215New Features
1216------------
1217
1218- The most significant new feature in this release is MDMF: "Medium-size
1219  Distributed Mutable Files". Unlike standard SDMF files, these provide
1220  efficient partial-access (reading and modifying small portions of the file
1221  instead of the whole thing). MDMF is opt-in (it is not yet the default
1222  format for mutable files), both to ensure compatibility with previous
1223  versions, and because the algorithm does not yet meet memory-usage goals.
1224  Enable it with ``--format=MDMF`` in the CLI (``tahoe put`` and ``tahoe
1225  mkdir``), or the "format" radioboxes in the web interface. See
1226  `<docs/specifications/mutable.rst>`__ for more details (`#393`_, `#1507`_)
1227- A "blacklist" feature allows blocking access to specific files through
1228  a particular gateway. See the "Access Blacklist" section of
1229  `<docs/configuration.rst>`__ for more details. (`#1425`_)
1230- A "drop-upload" feature has been added, which allows you to upload
1231  files to a Tahoe-LAFS directory just by writing them to a local
1232  directory. This feature is experimental and should not be relied on
1233  to store the only copy of valuable data. It is currently available
1234  only on Linux. See `<docs/frontends/drop-upload.rst>`__ for documentation.
1235  (`#1429`_)
1236- The timeline of immutable downloads can be viewed using a zoomable and
1237  pannable JavaScript-based visualization. This is accessed using the
1238  'timeline' link on the File Download Status page for the download, which
1239  can be reached from the Recent Uploads and Downloads page.
1240
1241Configuration/Behavior Changes
1242------------------------------
1243
1244- Prior to Tahoe-LAFS v1.3, the configuration of some node options could
1245  be specified using individual config files rather than via ``tahoe.cfg``.
1246  These files now cause an error if present. (`#1385`_)
1247- Storage servers now calculate their remaining space based on the filesystem
1248  containing the ``storage/shares/`` directory. Previously they looked at the
1249  filesystem containing the ``storage/`` directory. This allows
1250  ``storage/shares/``, rather than ``storage/``, to be a mount point or a
1251  symlink pointing to another filesystem. (`#1384`_)
1252- ``tahoe cp xyz MUTABLE`` will modify the existing mutable file instead of
1253  creating a new one. (`#1304`_)
1254- The button for unlinking a file from its directory on a WUI directory
1255  listing is now labelled "unlink" rather than "del". (`#1104`_)
1256
1257Notable Bugfixes
1258----------------
1259
1260- The security bugfix for the vulnerability allowing deletion of shares,
1261  detailed in the news for v1.8.3 below, is also included in this
1262  release. (`#1528`_)
1263- Some cases of immutable upload, for example using the ``tahoe put`` and
1264  ``tahoe cp`` commands or SFTP, did not appear in the history of Recent
1265  Uploads and Downloads. (`#1079`_)
1266- The memory footprint of the verifier has been reduced by serializing
1267  block fetches. (`#1395`_)
1268- Large immutable downloads are now a little faster than in v1.8.3 (about
1269  5% on a fast network). (`#1268`_)
1270
1271Packaging Changes
1272-----------------
1273
1274- The files related to Debian packaging have been removed from the Tahoe
1275  source tree, since they are now maintained as part of the official
1276  Debian packages. (`#1454`_)
1277- The unmaintained FUSE plugins were removed from the source tree. See
1278  ``docs/frontends/FTP-and-SFTP.rst`` for how to mount a Tahoe filesystem on
1279  Unix via sshfs. (`#1409`_)
1280- The Tahoe licenses now give explicit permission to combine Tahoe-LAFS
1281  with code distributed under the following additional open-source licenses
1282  (any version of each):
1283
1284  * Academic Free License
1285  * Apple Public Source License
1286  * BitTorrent Open Source License
1287  * Lucent Public License
1288  * Jabber Open Source License
1289  * Common Development and Distribution License
1290  * Microsoft Public License
1291  * Microsoft Reciprocal License
1292  * Sun Industry Standards Source License
1293  * Open Software License
1294
1295Compatibility and Dependencies
1296------------------------------
1297
1298- To resolve an incompatibility between Nevow and zope.interface (versions
1299  3.6.3 and 3.6.4), Tahoe-LAFS now requires an earlier or later
1300  version of zope.interface. (`#1435`_)
1301- The Twisted dependency has been raised to version 10.1 to ensure we no
1302  longer require pywin32 on Windows, the new drop-upload feature has the
1303  required support from Twisted on Linux, and that it is never necessary to
1304  patch Twisted in order to use the FTP frontend. (`#1274`_, `#1429`_,
1305  `#1438`_)
1306- An explicit dependency on pyOpenSSL has been added, replacing the indirect
1307  dependency via the "secure_connections" option of foolscap. (`#1383`_)
1308
1309Minor Changes
1310-------------
1311
1312- A ``man`` page has been added (`#1420`_). All other docs are in ReST
1313  format.
1314- The ``tahoe_files`` munin plugin reported an incorrect count of the number
1315  of share files. (`#1391`_)
1316- Minor documentation updates: #627, #1104, #1225, #1297, #1342, #1404
1317- Other minor changes: #636, #1355, #1363, #1366, #1388, #1392, #1412, #1344,
1318  #1347, #1359, #1389, #1441, #1442, #1446, #1474, #1503
1319
1320.. _`#393`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/393
1321.. _`#1079`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1079
1322.. _`#1104`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1104
1323.. _`#1268`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1268
1324.. _`#1274`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1274
1325.. _`#1304`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1304
1326.. _`#1383`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1383
1327.. _`#1384`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1384
1328.. _`#1385`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1385
1329.. _`#1391`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1391
1330.. _`#1395`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1395
1331.. _`#1409`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1409
1332.. _`#1420`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1420
1333.. _`#1425`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1425
1334.. _`#1429`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1429
1335.. _`#1435`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1435
1336.. _`#1438`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1438
1337.. _`#1454`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1454
1338.. _`#1507`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1507
1339
1340
1341Release 1.8.3 (2011-09-13)
1342''''''''''''''''''''''''''
1343
1344Security-related Bugfix
1345-----------------------
1346
1347- Fix flaw that would allow a person who knows a storage index of a file to
1348  delete shares of that file. (`#1528`_)
1349- Remove corner cases in mutable file bounds management which could expose
1350  extra lease info or old share data (from prior versions of the mutable
1351  file) if someone with write authority to that mutable file exercised these
1352  corner cases in a way that no actual Tahoe-LAFS client does. (Probably not
1353  exploitable.) (`#1528`_)
1354
1355.. _`#1528`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1528
1356
1357
1358Release 1.8.2 (2011-01-30)
1359''''''''''''''''''''''''''
1360
1361Compatibility and Dependencies
1362------------------------------
1363
1364- Tahoe is now compatible with Twisted-10.2 (released last month), as
1365  well as with earlier versions. The previous Tahoe-1.8.1 release
1366  failed to run against Twisted-10.2, raising an AttributeError on
1367  StreamServerEndpointService (`#1286`_)
1368- Tahoe now depends upon the "mock" testing library, and the foolscap
1369  dependency was raised to 0.6.1 . It no longer requires pywin32
1370  (which was used only on windows). Future developers should note that
1371  reactor.spawnProcess and derivatives may no longer be used inside
1372  Tahoe code.
1373
1374Other Changes
1375-------------
1376
1377- the default reserved_space value for new storage nodes is 1 GB
1378  (`#1208`_)
1379- documentation is now in reStructuredText (.rst) format
1380- "tahoe cp" should now handle non-ASCII filenames
1381- the unmaintained Mac/Windows GUI applications have been removed
1382  (`#1282`_)
1383- tahoe processes should appear in top and ps as "tahoe", not
1384  "python", on some unix platforms. (`#174`_)
1385- "tahoe debug trial" can be used to run the test suite (`#1296`_)
1386- the SFTP frontend now reports unknown sizes as "0" instead of "?",
1387  to improve compatibility with clients like FileZilla (`#1337`_)
1388- "tahoe --version" should now report correct values in situations
1389  where 1.8.1 might have been wrong (`#1287`_)
1390
1391.. _`#1208`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1208
1392.. _`#1282`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1282
1393.. _`#1286`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1286
1394.. _`#1287`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1287
1395.. _`#1296`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1296
1396.. _`#1337`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1337
1397
1398
1399Release 1.8.1 (2010-10-28)
1400''''''''''''''''''''''''''
1401
1402Bugfixes and Improvements
1403-------------------------
1404
1405- Allow the repairer to improve the health of a file by uploading some
1406  shares, even if it cannot achieve the configured happiness
1407  threshold. This fixes a regression introduced between v1.7.1 and
1408  v1.8.0. (`#1212`_)
1409- Fix a memory leak in the ResponseCache which is used during mutable
1410  file/directory operations. (`#1045`_)
1411- Fix a regression and add a performance improvement in the
1412  downloader.  This issue caused repair to fail in some special
1413  cases. (`#1223`_)
1414- Fix a bug that caused 'tahoe cp' to fail for a grid-to-grid copy
1415  involving a non-ASCII filename. (`#1224`_)
1416- Fix a rarely-encountered bug involving printing large strings to the
1417  console on Windows. (`#1232`_)
1418- Perform ~ expansion in the --exclude-from filename argument to
1419  'tahoe backup'. (`#1241`_)
1420- The CLI's 'tahoe mv' and 'tahoe ln' commands previously would try to
1421  use an HTTP proxy if the HTTP_PROXY environment variable was set.
1422  These now always connect directly to the WAPI, thus avoiding giving
1423  caps to the HTTP proxy (and also avoiding failures in the case that
1424  the proxy is failing or requires authentication). (`#1253`_)
1425- The CLI now correctly reports failure in the case that 'tahoe mv'
1426  fails to unlink the file from its old location. (`#1255`_)
1427- 'tahoe start' now gives a more positive indication that the node has
1428  started. (`#71`_)
1429- The arguments seen by 'ps' or other tools for node processes are now
1430  more useful (in particular, they include the path of the 'tahoe'
1431  script, rather than an obscure tool named 'twistd'). (`#174`_)
1432
1433Removed Features
1434----------------
1435
1436- The tahoe start/stop/restart and node creation commands no longer
1437  accept the -m or --multiple option, for consistency between
1438  platforms.  (`#1262`_)
1439
1440Packaging
1441---------
1442
1443- We now host binary packages so that users on certain operating
1444  systems can install without having a compiler.
1445  <https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html>
1446- Use a newer version of a dependency if needed, even if an older
1447  version is installed. This would previously cause a VersionConflict
1448  error. (`#1190`_)
1449- Use a precompiled binary of a dependency if one with a sufficiently
1450  high version number is available, instead of attempting to compile
1451  the dependency from source, even if the source version has a higher
1452  version number. (`#1233`_)
1453
1454Documentation
1455-------------
1456
1457- All current documentation in .txt format has been converted to .rst
1458  format. (`#1225`_)
1459- Added docs/backdoors.rst declaring that we won't add backdoors to
1460  Tahoe-LAFS, or add anything to facilitate government access to data.
1461  (`#1216`_)
1462
1463.. _`#71`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/71
1464.. _`#174`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/174
1465.. _`#1212`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1212
1466.. _`#1045`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1045
1467.. _`#1190`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1190
1468.. _`#1216`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1216
1469.. _`#1223`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1223
1470.. _`#1224`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1224
1471.. _`#1225`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1225
1472.. _`#1232`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1232
1473.. _`#1233`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1233
1474.. _`#1241`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1241
1475.. _`#1253`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1253
1476.. _`#1255`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1255
1477.. _`#1262`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1262
1478
1479
1480Release 1.8.0 (2010-09-23)
1481''''''''''''''''''''''''''
1482
1483New Features
1484------------
1485
1486- A completely new downloader which improves performance and
1487  robustness of immutable-file downloads. It uses the fastest K
1488  servers to download the data in K-way parallel. It automatically
1489  fails over to alternate servers if servers fail in mid-download. It
1490  allows seeking to arbitrary locations in the file (the previous
1491  downloader which would only read the entire file sequentially from
1492  beginning to end). It minimizes unnecessary round trips and
1493  unnecessary bytes transferred to improve performance. It sends
1494  requests to fewer servers to reduce the load on servers (the
1495  previous one would send a small request to every server for every
1496  download) (`#287`_, `#288`_, `#448`_, `#798`_, `#800`_, `#990`_,
1497  `#1170`_, `#1191`_)
1498- Non-ASCII command-line arguments and non-ASCII outputs now work on
1499  Windows. In addition, the command-line tool now works on 64-bit
1500  Windows. (`#1074`_)
1501
1502Bugfixes and Improvements
1503-------------------------
1504
1505- Document and clean up the command-line options for specifying the
1506  node's base directory. (`#188`_, `#706`_, `#715`_, `#772`_,
1507  `#1108`_)
1508- The default node directory for Windows is ".tahoe" in the user's
1509  home directory, the same as on other platforms. (`#890`_)
1510- Fix a case in which full cap URIs could be logged. (`#685`_,
1511  `#1155`_)
1512- Fix bug in WUI in Python 2.5 when the system clock is set back to
1513  1969. Now you can use Tahoe-LAFS with Python 2.5 and set your system
1514  clock to 1969 and still use the WUI. (`#1055`_)
1515- Many improvements in code organization, tests, logging,
1516  documentation, and packaging. (`#983`_, `#1074`_, `#1108`_,
1517  `#1127`_, `#1129`_, `#1131`_, `#1166`_, `#1175`_)
1518
1519Dependency Updates
1520------------------
1521
1522- on x86 and x86-64 platforms, pycryptopp >= 0.5.20
1523- pycrypto 2.2 is excluded due to a bug
1524
1525.. _`#188`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/188
1526.. _`#288`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/288
1527.. _`#448`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/448
1528.. _`#685`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/685
1529.. _`#706`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/706
1530.. _`#715`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/715
1531.. _`#772`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/772
1532.. _`#798`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/798
1533.. _`#800`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/800
1534.. _`#890`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/890
1535.. _`#983`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/983
1536.. _`#990`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/990
1537.. _`#1055`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1055
1538.. _`#1074`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1074
1539.. _`#1108`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1108
1540.. _`#1155`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1155
1541.. _`#1170`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1170
1542.. _`#1191`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1191
1543.. _`#1127`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1127
1544.. _`#1129`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1129
1545.. _`#1131`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1131
1546.. _`#1166`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1166
1547.. _`#1175`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1175
1548
1549Release 1.7.1 (2010-07-18)
1550''''''''''''''''''''''''''
1551
1552Bugfixes and Improvements
1553-------------------------
1554
1555- Fix bug in which uploader could fail with AssertionFailure or report
1556  that it had achieved servers-of-happiness when it hadn't. (`#1118`_)
1557- Fix bug in which servers could get into a state where they would
1558  refuse to accept shares of a certain file (`#1117`_)
1559- Add init scripts for managing the gateway server on Debian/Ubuntu
1560  (`#961`_)
1561- Fix bug where server version number was always 0 on the welcome page
1562  (`#1067`_)
1563- Add new command-line command "tahoe unlink" as a synonym for "tahoe
1564  rm" (`#776`_)
1565- The FTP frontend now encrypts its temporary files, protecting their
1566  contents from an attacker who is able to read the disk. (`#1083`_)
1567- Fix IP address detection on FreeBSD 7, 8, and 9 (`#1098`_)
1568- Fix minor layout issue in the Web User Interface with Internet
1569  Explorer (`#1097`_)
1570- Fix rarely-encountered incompatibility between Twisted logging
1571  utility and the new unicode support added in v1.7.0 (`#1099`_)
1572- Forward-compatibility improvements for non-ASCII caps (`#1051`_)
1573
1574Code improvements
1575-----------------
1576
1577- Simplify and tidy-up directories, unicode support, test code
1578  (`#923`_, `#967`_, `#1072`_)
1579
1580.. _`#776`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/776
1581.. _`#923`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/923
1582.. _`#961`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/961
1583.. _`#967`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/967
1584.. _`#1051`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1051
1585.. _`#1067`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1067
1586.. _`#1072`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1072
1587.. _`#1083`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1083
1588.. _`#1097`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1097
1589.. _`#1098`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1098
1590.. _`#1099`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1099
1591.. _`#1117`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1117
1592.. _`#1118`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1118
1593
1594
1595Release 1.7.0 (2010-06-18)
1596''''''''''''''''''''''''''
1597
1598New Features
1599------------
1600
1601- SFTP support (`#1037`_)
1602  Your Tahoe-LAFS gateway now acts like a full-fledged SFTP server. It
1603  has been tested with sshfs to provide a virtual filesystem in Linux.
1604  Many users have asked for this feature.  We hope that it serves them
1605  well! See the `FTP-and-SFTP.rst`_ document to get
1606  started.
1607- support for non-ASCII character encodings (`#534`_)
1608  Tahoe-LAFS now correctly handles filenames containing non-ASCII
1609  characters on all supported platforms:
1610
1611 - when reading files in from the local filesystem (such as when you
1612   run "tahoe backup" to back up your local files to a Tahoe-LAFS
1613   grid);
1614 - when writing files out to the local filesystem (such as when you
1615   run "tahoe cp -r" to recursively copy files out of a Tahoe-LAFS
1616   grid);
1617 - when displaying filenames to the terminal (such as when you run
1618   "tahoe ls"), subject to limitations of the terminal and locale;
1619 - when parsing command-line arguments, except on Windows.
1620
1621- Servers of Happiness (`#778`_)
1622  Tahoe-LAFS now measures during immutable file upload to see how well
1623  distributed it is across multiple servers. It aborts the upload if
1624  the pieces of the file are not sufficiently well-distributed.
1625  This behavior is controlled by a configuration parameter called
1626  "servers of happiness". With the default settings for its erasure
1627  coding, Tahoe-LAFS generates 10 shares for each file, such that any
1628  3 of those shares are sufficient to recover the file. The default
1629  value of "servers of happiness" is 7, which means that Tahoe-LAFS
1630  will guarantee that there are at least 7 servers holding some of the
1631  shares, such that any 3 of those servers can completely recover your
1632  file.  The new upload code also distributes the shares better than the
1633  previous version in some cases and takes better advantage of
1634  pre-existing shares (when a file has already been previously
1635  uploaded). See the `architecture.rst`_ document [3] for details.
1636
1637Bugfixes and Improvements
1638-------------------------
1639
1640- Premature abort of upload if some shares were already present and
1641  some servers fail. (`#608`_)
1642- python ./setup.py install -- can't create or remove files in install
1643  directory. (`#803`_)
1644- Network failure => internal TypeError. (`#902`_)
1645- Install of Tahoe on CentOS 5.4. (`#933`_)
1646- CLI option --node-url now supports https url. (`#1028`_)
1647- HTML/CSS template files were not correctly installed under
1648  Windows. (`#1033`_)
1649- MetadataSetter does not enforce restriction on setting "tahoe"
1650  subkeys.  (`#1034`_)
1651- ImportError: No module named
1652  setuptools_darcs.setuptools_darcs. (`#1054`_)
1653- Renamed Title in xhtml files. (`#1062`_)
1654- Increase Python version dependency to 2.4.4, to avoid a critical
1655  CPython security bug. (`#1066`_)
1656- Typo correction for the munin plugin tahoe_storagespace. (`#968`_)
1657- Fix warnings found by pylint. (`#973`_)
1658- Changing format of some documentation files. (`#1027`_)
1659- the misc/ directory was tied up. (`#1068`_)
1660- The 'ctime' and 'mtime' metadata fields are no longer written except
1661  by "tahoe backup". (`#924`_)
1662- Unicode filenames in Tahoe-LAFS directories are normalized so that
1663  names that differ only in how accents are encoded are treated as the
1664  same. (`#1076`_)
1665- Various small improvements to documentation. (`#937`_, `#911`_,
1666  `#1024`_, `#1082`_)
1667
1668Removals
1669--------
1670
1671- The 'tahoe debug consolidate' subcommand (for converting old
1672  allmydata Windows client backups to a newer format) has been
1673  removed.
1674
1675Dependency Updates
1676------------------
1677
1678- the Python version dependency is raised to 2.4.4 in some cases
1679  (2.4.3 for Redhat-based Linux distributions, 2.4.2 for UCS-2 builds)
1680  (`#1066`_)
1681- pycrypto >= 2.0.1
1682- pyasn1 >= 0.0.8a
1683- mock (only required by unit tests)
1684
1685.. _`#534`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/534
1686.. _`#608`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/608
1687.. _`#778`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/778
1688.. _`#803`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/803
1689.. _`#902`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/902
1690.. _`#911`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/911
1691.. _`#924`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/924
1692.. _`#937`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/937
1693.. _`#933`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/933
1694.. _`#968`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/968
1695.. _`#973`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/973
1696.. _`#1024`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1024
1697.. _`#1027`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1027
1698.. _`#1028`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1028
1699.. _`#1033`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1033
1700.. _`#1034`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1034
1701.. _`#1037`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1037
1702.. _`#1054`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1054
1703.. _`#1062`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1062
1704.. _`#1066`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1066
1705.. _`#1068`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1068
1706.. _`#1076`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1076
1707.. _`#1082`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1082
1708.. _architecture.rst: docs/architecture.rst
1709.. _FTP-and-SFTP.rst: docs/frontends/FTP-and-SFTP.rst
1710
1711Release 1.6.1 (2010-02-27)
1712''''''''''''''''''''''''''
1713
1714Bugfixes
1715--------
1716
1717- Correct handling of Small Immutable Directories
1718
1719  Immutable directories can now be deep-checked and listed in the web
1720  UI in all cases. (In v1.6.0, some operations, such as deep-check, on
1721  a directory graph that included very small immutable directories,
1722  would result in an exception causing the whole operation to abort.)
1723  (`#948`_)
1724
1725Usability Improvements
1726----------------------
1727
1728- Improved user interface messages and error reporting. (`#681`_,
1729  `#837`_, `#939`_)
1730- The timeouts for operation handles have been greatly increased, so
1731  that you can view the results of an operation up to 4 days after it
1732  has completed. After viewing them for the first time, the results
1733  are retained for a further day. (`#577`_)
1734
1735Release 1.6.0 (2010-02-01)
1736''''''''''''''''''''''''''
1737
1738New Features
1739------------
1740
1741- Immutable Directories
1742
1743  Tahoe-LAFS can now create and handle immutable
1744  directories. (`#607`_, `#833`_, `#931`_) These are read just like
1745  normal directories, but are "deep-immutable", meaning that all their
1746  children (and everything reachable from those children) must be
1747  immutable objects (i.e. immutable or literal files, and other
1748  immutable directories).
1749
1750  These directories must be created in a single webapi call that
1751  provides all of the children at once. (Since they cannot be changed
1752  after creation, the usual create/add/add sequence cannot be used.)
1753  They have URIs that start with "URI:DIR2-CHK:" or "URI:DIR2-LIT:",
1754  and are described on the human-facing web interface (aka the "WUI")
1755  with a "DIR-IMM" abbreviation (as opposed to "DIR" for the usual
1756  read-write directories and "DIR-RO" for read-only directories).
1757
1758  Tahoe-LAFS releases before 1.6.0 cannot read the contents of an
1759  immutable directory. 1.5.0 will tolerate their presence in a
1760  directory listing (and display it as "unknown"). 1.4.1 and earlier
1761  cannot tolerate them: a DIR-IMM child in any directory will prevent
1762  the listing of that directory.
1763
1764  Immutable directories are repairable, just like normal immutable
1765  files.
1766
1767  The webapi "POST t=mkdir-immutable" call is used to create immutable
1768  directories. See `webapi.rst`_ for details.
1769
1770- "tahoe backup" now creates immutable directories, backupdb has
1771  dircache
1772
1773  The "tahoe backup" command has been enhanced to create immutable
1774  directories (in previous releases, it created read-only mutable
1775  directories) (`#828`_). This is significantly faster, since it does
1776  not need to create an RSA keypair for each new directory. Also
1777  "DIR-IMM" immutable directories are repairable, unlike "DIR-RO"
1778  read-only mutable directories at present. (A future Tahoe-LAFS
1779  release should also be able to repair DIR-RO.)
1780
1781  In addition, the backupdb (used by "tahoe backup" to remember what
1782  it has already copied) has been enhanced to store information about
1783  existing immutable directories. This allows it to re-use directories
1784  that have moved but still contain identical contents, or that have
1785  been deleted and later replaced. (The 1.5.0 "tahoe backup" command
1786  could only re-use directories that were in the same place as they
1787  were in the immediately previous backup.)  With this change, the
1788  backup process no longer needs to read the previous snapshot out of
1789  the Tahoe-LAFS grid, reducing the network load
1790  considerably. (`#606`_)
1791
1792  A "null backup" (in which nothing has changed since the previous
1793  backup) will require only two Tahoe-side operations: one to add an
1794  Archives/$TIMESTAMP entry, and a second to update the Latest/
1795  link. On the local disk side, it will readdir() all your local
1796  directories and stat() all your local files.
1797
1798  If you've been using "tahoe backup" for a while, you will notice
1799  that your first use of it after upgrading to 1.6.0 may take a long
1800  time: it must create proper immutable versions of all the old
1801  read-only mutable directories. This process won't take as long as
1802  the initial backup (where all the file contents had to be uploaded
1803  too): it will require time proportional to the number and size of
1804  your directories. After this initial pass, all subsequent passes
1805  should take a tiny fraction of the time.
1806
1807  As noted above, Tahoe-LAFS versions earlier than 1.5.0 cannot list a
1808  directory containing an immutable subdirectory. Tahoe-LAFS versions
1809  earlier than 1.6.0 cannot read the contents of an immutable
1810  directory.
1811
1812  The "tahoe backup" command has been improved to skip over unreadable
1813  objects (like device files, named pipes, and files with permissions
1814  that prevent the command from reading their contents), instead of
1815  throwing an exception and terminating the backup process. It also
1816  skips over symlinks, because these cannot be represented faithfully
1817  in the Tahoe-side filesystem. A warning message will be emitted each
1818  time something is skipped. (`#729`_, `#850`_, `#641`_)
1819
1820- "create-node" command added, "create-client" now implies
1821  --no-storage
1822
1823  The basic idea behind Tahoe-LAFS's client+server and client-only
1824  processes is that you are creating a general-purpose Tahoe-LAFS
1825  "node" process, which has several components that can be
1826  activated. Storage service is one of these optional components, as
1827  is the Helper, FTP server, and SFTP server. Web gateway
1828  functionality is nominally on this list, but it is always active; a
1829  future release will make it optional. There are three special
1830  purpose servers that can't currently be run as a component in a
1831  node: introducer, key-generator, and stats-gatherer.
1832
1833  So now "tahoe create-node" will create a Tahoe-LAFS node process,
1834  and after creation you can edit its tahoe.cfg to enable or disable
1835  the desired services. It is a more general-purpose replacement for
1836  "tahoe create-client".  The default configuration has storage
1837  service enabled. For convenience, the "--no-storage" argument makes
1838  a tahoe.cfg file that disables storage service. (`#760`_)
1839
1840  "tahoe create-client" has been changed to create a Tahoe-LAFS node
1841  without a storage service. It is equivalent to "tahoe create-node
1842  --no-storage". This helps to reduce the confusion surrounding the
1843  use of a command with "client" in its name to create a storage
1844  *server*. Use "tahoe create-client" to create a purely client-side
1845  node. If you want to offer storage to the grid, use "tahoe
1846  create-node" instead.
1847
1848  In the future, other services will be added to the node, and they
1849  will be controlled through options in tahoe.cfg . The most important
1850  of these services may get additional --enable-XYZ or --disable-XYZ
1851  arguments to "tahoe create-node".
1852
1853- Performance Improvements
1854
1855  Download of immutable files begins as soon as the downloader has
1856  located the K necessary shares (`#928`_, `#287`_). In both the
1857  previous and current releases, a downloader will first issue queries
1858  to all storage servers on the grid to locate shares before it begins
1859  downloading the shares. In previous releases of Tahoe-LAFS, download
1860  would not begin until all storage servers on the grid had replied to
1861  the query, at which point K shares would be chosen for download from
1862  among the shares that were located. In this release, download begins
1863  as soon as any K shares are located. This means that downloads start
1864  sooner, which is particularly important if there is a server on the
1865  grid that is extremely slow or even hung in such a way that it will
1866  never respond. In previous releases such a server would have a
1867  negative impact on all downloads from that grid. In this release,
1868  such a server will have no impact on downloads, as long as K shares
1869  can be found on other, quicker, servers.  This also means that
1870  downloads now use the "best-alacrity" servers that they talk to, as
1871  measured by how quickly the servers reply to the initial query. This
1872  might cause downloads to go faster, especially on grids with
1873  heterogeneous servers or geographical dispersion.
1874
1875Minor Changes
1876-------------
1877
1878- The webapi acquired a new "t=mkdir-with-children" command, to create
1879  and populate a directory in a single call. This is significantly
1880  faster than using separate "t=mkdir" and "t=set-children" operations
1881  (it uses one gateway-to-grid roundtrip, instead of three or
1882  four). (`#533`_)
1883
1884- The t=set-children (note the hyphen) operation is now documented in
1885  webapi.rst, and is the new preferred spelling of the
1886  old t=set_children (with an underscore). The underscore version
1887  remains for backwards compatibility. (`#381`_, `#927`_)
1888
1889- The tracebacks produced by errors in CLI tools should now be in
1890  plain text, instead of HTML (which is unreadable outside of a
1891  browser). (`#646`_)
1892
1893- The [storage]reserved_space configuration knob (which causes the
1894  storage server to refuse shares when available disk space drops
1895  below a threshold) should work on Windows now, not just
1896  UNIX. (`#637`_)
1897
1898- "tahoe cp" should now exit with status "1" if it cannot figure out a
1899  suitable target filename, such as when you copy from a bare
1900  filecap. (`#761`_)
1901
1902- "tahoe get" no longer creates a zero-length file upon
1903  error. (`#121`_)
1904
1905- "tahoe ls" can now list single files. (`#457`_)
1906
1907- "tahoe deep-check --repair" should tolerate repair failures now,
1908  instead of halting traversal. (`#874`_, `#786`_)
1909
1910- "tahoe create-alias" no longer corrupts the aliases file if it had
1911  previously been edited to have no trailing newline. (`#741`_)
1912
1913- Many small packaging improvements were made to facilitate the
1914  "tahoe-lafs" package being included in Ubuntu. Several mac/win32
1915  binary libraries were removed, some figleaf code-coverage files were
1916  removed, a bundled copy of darcsver-1.2.1 was removed, and
1917  additional licensing text was added.
1918
1919- Several DeprecationWarnings for python2.6 were silenced. (`#859`_)
1920
1921- The checker --add-lease option would sometimes fail for shares
1922  stored on old (Tahoe v1.2.0) servers. (`#875`_)
1923
1924- The documentation for installing on Windows (docs/quickstart.rst)
1925  has been improved. (`#773`_)
1926
1927For other changes not mentioned here, see
1928<https://tahoe-lafs.org/trac/tahoe-lafs/query?milestone=1.6.0&keywords=!~news-done>.
1929To include the tickets mentioned above, go to
1930<https://tahoe-lafs.org/trac/tahoe-lafs/query?milestone=1.6.0>.
1931
1932.. _`#121`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/121
1933.. _`#287`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/287
1934.. _`#381`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/381
1935.. _`#457`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/457
1936.. _`#533`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/533
1937.. _`#577`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/577
1938.. _`#606`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/606
1939.. _`#607`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/607
1940.. _`#637`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/637
1941.. _`#641`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/641
1942.. _`#646`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/646
1943.. _`#681`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/681
1944.. _`#729`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/729
1945.. _`#741`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/741
1946.. _`#760`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/760
1947.. _`#761`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/761
1948.. _`#773`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/773
1949.. _`#786`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/786
1950.. _`#828`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/828
1951.. _`#833`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/833
1952.. _`#859`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/859
1953.. _`#874`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/874
1954.. _`#875`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/875
1955.. _`#931`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/931
1956.. _`#837`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/837
1957.. _`#850`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/850
1958.. _`#927`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/927
1959.. _`#928`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/928
1960.. _`#939`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/939
1961.. _`#948`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/948
1962.. _webapi.rst: docs/frontends/webapi.rst
1963
1964Release 1.5.0 (2009-08-01)
1965''''''''''''''''''''''''''
1966
1967Improvements
1968------------
1969
1970- Uploads of immutable files now use pipelined writes, improving
1971  upload speed slightly (10%) over high-latency connections. (`#392`_)
1972
1973- Processing large directories has been sped up, by removing a O(N^2)
1974  algorithm from the dirnode decoding path and retaining unmodified
1975  encrypted entries.  (`#750`_, `#752`_)
1976
1977- The human-facing web interface (aka the "WUI") received a
1978  significant CSS makeover by Kevin Reid, making it much prettier and
1979  easier to read. The WUI "check" and "deep-check" forms now include a
1980  "Renew Lease" checkbox, mirroring the CLI --add-lease option, so
1981  leases can be added or renewed from the web interface.
1982
1983- The CLI "tahoe mv" command now refuses to overwrite
1984  directories. (`#705`_)
1985
1986- The CLI "tahoe webopen" command, when run without arguments, will
1987  now bring up the "Welcome Page" (node status and mkdir/upload
1988  forms).
1989
1990- The 3.5MB limit on mutable files was removed, so it should be
1991  possible to upload arbitrarily-sized mutable files. Note, however,
1992  that the data format and algorithm remains the same, so using
1993  mutable files still requires bandwidth, computation, and RAM in
1994  proportion to the size of the mutable file.  (`#694`_)
1995
1996- This version of Tahoe-LAFS will tolerate directory entries that
1997  contain filecap formats which it does not recognize: files and
1998  directories from the future.  This should improve the user
1999  experience (for 1.5.0 users) when we add new cap formats in the
2000  future. Previous versions would fail badly, preventing the user from
2001  seeing or editing anything else in those directories. These
2002  unrecognized objects can be renamed and deleted, but obviously not
2003  read or written. Also they cannot generally be copied. (`#683`_)
2004
2005Bugfixes
2006--------
2007
2008- deep-check-and-repair now tolerates read-only directories, such as
2009  the ones produced by the "tahoe backup" CLI command. Read-only
2010  directories and mutable files are checked, but not
2011  repaired. Previous versions threw an exception when attempting the
2012  repair and failed to process the remaining contents. We cannot yet
2013  repair these read-only objects, but at least this version allows the
2014  rest of the check+repair to proceed. (`#625`_)
2015
2016- A bug in 1.4.1 which caused a server to be listed multiple times
2017  (and frequently broke all connections to that server) was
2018  fixed. (`#653`_)
2019
2020- The plaintext-hashing code was removed from the Helper interface,
2021  removing the Helper's ability to mount a
2022  partial-information-guessing attack. (`#722`_)
2023
2024Platform/packaging changes
2025--------------------------
2026
2027- Tahoe-LAFS now runs on NetBSD, OpenBSD, ArchLinux, and NixOS, and on
2028  an embedded system based on an ARM CPU running at 266 MHz.
2029
2030- Unit test timeouts have been raised to allow the tests to complete
2031  on extremely slow platforms like embedded ARM-based NAS boxes, which
2032  may take several hours to run the test suite. An ARM-specific
2033  data-corrupting bug in an older version of Crypto++ (5.5.2) was
2034  identified: ARM-users are encouraged to use recent
2035  Crypto++/pycryptopp which avoids this problem.
2036
2037- Tahoe-LAFS now requires a SQLite library, either the sqlite3 that
2038  comes built-in with python2.5/2.6, or the add-on pysqlite2 if you're
2039  using python2.4. In the previous release, this was only needed for
2040  the "tahoe backup" command: now it is mandatory.
2041
2042- Several minor documentation updates were made.
2043
2044- To help get Tahoe-LAFS into Linux distributions like Fedora and
2045  Debian, packaging improvements are being made in both Tahoe-LAFS and
2046  related libraries like pycryptopp and zfec.
2047
2048- The Crypto++ library included in the pycryptopp package has been
2049  upgraded to version 5.6.0 of Crypto++, which includes a more
2050  efficient implementation of SHA-256 in assembly for x86 or amd64
2051  architectures.
2052
2053dependency updates
2054------------------
2055
2056- foolscap-0.4.1
2057- no python-2.4.0 or 2.4.1 (2.4.2 is good) (they contained a bug in base64.b32decode)
2058- avoid python-2.6 on windows with mingw: compiler issues
2059- python2.4 requires pysqlite2 (2.5,2.6 does not)
2060- no python-3.x
2061- pycryptopp-0.5.15
2062
2063.. _#392: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/392
2064.. _#625: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/625
2065.. _#653: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/653
2066.. _#683: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/683
2067.. _#694: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/694
2068.. _#705: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/705
2069.. _#722: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/722
2070.. _#750: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/750
2071.. _#752: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/752
2072
2073Release 1.4.1 (2009-04-13)
2074''''''''''''''''''''''''''
2075
2076Garbage Collection
2077------------------
2078
2079- The big feature for this release is the implementation of garbage
2080  collection, allowing Tahoe storage servers to delete shares for old
2081  deleted files. When enabled, this uses a "mark and sweep" process:
2082  clients are responsible for updating the leases on their shares
2083  (generally by running "tahoe deep-check --add-lease"), and servers
2084  are allowed to delete any share which does not have an up-to-date
2085  lease. The process is described in detail in
2086  `garbage-collection.rst`_.
2087
2088  The server must be configured to enable garbage-collection, by
2089  adding directives to the [storage] section that define an age limit
2090  for shares. The default configuration will not delete any shares.
2091
2092  Both servers and clients should be upgraded to this release to make
2093  the garbage-collection as pleasant as possible. 1.2.0 servers have
2094  code to perform the update-lease operation but it suffers from a
2095  fatal bug, while 1.3.0 servers have update-lease but will return an
2096  exception for unknown storage indices, causing clients to emit an
2097  Incident for each exception, slowing the add-lease process down to a
2098  crawl. 1.1.0 servers did not have the add-lease operation at all.
2099
2100Security/Usability Problems Fixed
2101---------------------------------
2102
2103- A super-linear algorithm in the Merkle Tree code was fixed, which
2104  previously caused e.g. download of a 10GB file to take several hours
2105  before the first byte of plaintext could be produced. The new
2106  "alacrity" is about 2 minutes. A future release should reduce this
2107  to a few seconds by fixing ticket `#442`_.
2108
2109- The previous version permitted a small timing attack (due to our use
2110  of strcmp) against the write-enabler and lease-renewal/cancel
2111  secrets. An attacker who could measure response-time variations of
2112  approximatly 3ns against a very noisy background time of about 15ms
2113  might be able to guess these secrets. We do not believe this attack
2114  was actually feasible. This release closes the attack by first
2115  hashing the two strings to be compared with a random secret.
2116
2117webapi changes
2118--------------
2119
2120- In most cases, HTML tracebacks will only be sent if an "Accept:
2121  text/html" header was provided with the HTTP request. This will
2122  generally cause browsers to get an HTMLized traceback but send
2123  regular text/plain tracebacks to non-browsers (like the CLI
2124  clients). More errors have been mapped to useful HTTP error codes.
2125
2126- The streaming webapi operations (deep-check and manifest) now have a
2127  way to indicate errors (an output line that starts with "ERROR"
2128  instead of being legal JSON). See `webapi.rst`_ for
2129  details.
2130
2131- The storage server now has its own status page (at /storage), linked
2132  from the Welcome page. This page shows progress and results of the
2133  two new share-crawlers: one which merely counts shares (to give an
2134  estimate of how many files/directories are being stored in the
2135  grid), the other examines leases and reports how much space would be
2136  freed if GC were enabled. The page also shows how much disk space is
2137  present, used, reserved, and available for the Tahoe server, and
2138  whether the server is currently running in "read-write" mode or
2139  "read-only" mode.
2140
2141- When a directory node cannot be read (perhaps because of insufficent
2142  shares), a minimal webapi page is created so that the "more-info"
2143  links (including a Check/Repair operation) will still be accessible.
2144
2145- A new "reliability" page was added, with the beginnings of work on a
2146  statistical loss model. You can tell this page how many servers you
2147  are using and their independent failure probabilities, and it will
2148  tell you the likelihood that an arbitrary file will survive each
2149  repair period. The "numpy" package must be installed to access this
2150  page. A partial paper, written by Shawn Willden, has been added to
2151  docs/proposed/lossmodel.lyx .
2152
2153CLI changes
2154-----------
2155
2156- "tahoe check" and "tahoe deep-check" now accept an "--add-lease"
2157  argument, to update a lease on all shares. This is the "mark" side
2158  of garbage collection.
2159
2160- In many cases, CLI error messages have been improved: the ugly
2161  HTMLized traceback has been replaced by a normal python traceback.
2162
2163- "tahoe deep-check" and "tahoe manifest" now have better error
2164  reporting.  "tahoe cp" is now non-verbose by default.
2165
2166- "tahoe backup" now accepts several "--exclude" arguments, to ignore
2167  certain files (like editor temporary files and version-control
2168  metadata) during backup.
2169
2170- On windows, the CLI now accepts local paths like "c:\dir\file.txt",
2171  which previously was interpreted as a Tahoe path using a "c:" alias.
2172
2173- The "tahoe restart" command now uses "--force" by default (meaning
2174  it will start a node even if it didn't look like there was one
2175  already running).
2176
2177- The "tahoe debug consolidate" command was added. This takes a series
2178  of independent timestamped snapshot directories (such as those
2179  created by the allmydata.com windows backup program, or a series of
2180  "tahoe cp -r" commands) and creates new snapshots that used shared
2181  read-only directories whenever possible (like the output of "tahoe
2182  backup"). In the most common case (when the snapshots are fairly
2183  similar), the result will use significantly fewer directories than
2184  the original, allowing "deep-check" and similar tools to run much
2185  faster. In some cases, the speedup can be an order of magnitude or
2186  more.  This tool is still somewhat experimental, and only needs to
2187  be run on large backups produced by something other than "tahoe
2188  backup", so it was placed under the "debug" category.
2189
2190- "tahoe cp -r --caps-only tahoe:dir localdir" is a diagnostic tool
2191  which, instead of copying the full contents of files into the local
2192  directory, merely copies their filecaps. This can be used to verify
2193  the results of a "consolidation" operation.
2194
2195other fixes
2196-----------
2197
2198- The codebase no longer rauses RuntimeError as a kind of
2199  assert(). Specific exception classes were created for each previous
2200  instance of RuntimeError.
2201
2202- Many unit tests were changed to use a non-network test harness,
2203  speeding them up considerably.
2204
2205- Deep-traversal operations (manifest and deep-check) now walk
2206  individual directories in alphabetical order. Occasional turn breaks
2207  are inserted to prevent a stack overflow when traversing directories
2208  with hundreds of entries.
2209
2210- The experimental SFTP server had its path-handling logic changed
2211  slightly, to accomodate more SFTP clients, although there are still
2212  issues (`#645`_).
2213
2214.. _#442: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/442
2215.. _#645: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/645
2216.. _garbage-collection.rst: docs/garbage-collection.rst
2217
2218Release 1.3.0 (2009-02-13)
2219''''''''''''''''''''''''''
2220
2221Checker/Verifier/Repairer
2222-------------------------
2223
2224- The primary focus of this release has been writing a checker /
2225  verifier / repairer for files and directories.  "Checking" is the
2226  act of asking storage servers whether they have a share for the
2227  given file or directory: if there are not enough shares available,
2228  the file or directory will be unrecoverable. "Verifying" is the act
2229  of downloading and cryptographically asserting that the server's
2230  share is undamaged: it requires more work (bandwidth and CPU) than
2231  checking, but can catch problems that simple checking
2232  cannot. "Repair" is the act of replacing missing or damaged shares
2233  with new ones.
2234
2235- This release includes a full checker, a partial verifier, and a
2236  partial repairer. The repairer is able to handle missing shares: new
2237  shares are generated and uploaded to make up for the missing
2238  ones. This is currently the best application of the repairer: to
2239  replace shares that were lost because of server departure or
2240  permanent drive failure.
2241
2242- The repairer in this release is somewhat able to handle corrupted
2243  shares. The limitations are:
2244
2245 - Immutable verifier is incomplete: not all shares are used, and not
2246   all fields of those shares are verified. Therefore the immutable
2247   verifier has only a moderate chance of detecting corrupted shares.
2248 - The mutable verifier is mostly complete: all shares are examined,
2249   and most fields of the shares are validated.
2250 - The storage server protocol offers no way for the repairer to
2251   replace or delete immutable shares. If corruption is detected, the
2252   repairer will upload replacement shares to other servers, but the
2253   corrupted shares will be left in place.
2254 - read-only directories and read-only mutable files must be repaired
2255   by someone who holds the write-cap: the read-cap is
2256   insufficient. Moreover, the deep-check-and-repair operation will
2257   halt with an error if it attempts to repair one of these read-only
2258   objects.
2259 - Some forms of corruption can cause both download and repair
2260   operations to fail. A future release will fix this, since download
2261   should be tolerant of any corruption as long as there are at least
2262   'k' valid shares, and repair should be able to fix any file that is
2263   downloadable.
2264
2265- If the downloader, verifier, or repairer detects share corruption,
2266  the servers which provided the bad shares will be notified (via a
2267  file placed in the BASEDIR/storage/corruption-advisories directory)
2268  so their operators can manually delete the corrupted shares and
2269  investigate the problem. In addition, the "incident gatherer"
2270  mechanism will automatically report share corruption to an incident
2271  gatherer service, if one is configured. Note that corrupted shares
2272  indicate hardware failures, serious software bugs, or malice on the
2273  part of the storage server operator, so a corrupted share should be
2274  considered highly unusual.
2275
2276- By periodically checking/repairing all files and directories,
2277  objects in the Tahoe filesystem remain resistant to recoverability
2278  failures due to missing and/or broken servers.
2279
2280- This release includes a wapi mechanism to initiate checks on
2281  individual files and directories (with or without verification, and
2282  with or without automatic repair). A related mechanism is used to
2283  initiate a "deep-check" on a directory: recursively traversing the
2284  directory and its children, checking (and/or verifying/repairing)
2285  everything underneath. Both mechanisms can be run with an
2286  "output=JSON" argument, to obtain machine-readable check/repair
2287  status results. These results include a copy of the filesystem
2288  statistics from the "deep-stats" operation (including total number
2289  of files, size histogram, etc). If repair is possible, a "Repair"
2290  button will appear on the results page.
2291
2292- The client web interface now features some extra buttons to initiate
2293  check and deep-check operations. When these operations finish, they
2294  display a results page that summarizes any problems that were
2295  encountered. All long-running deep-traversal operations, including
2296  deep-check, use a start-and-poll mechanism, to avoid depending upon
2297  a single long-lived HTTP connection. `webapi.rst`_ has
2298  details.
2299
2300Efficient Backup
2301----------------
2302
2303- The "tahoe backup" command is new in this release, which creates
2304  efficient versioned backups of a local directory. Given a local
2305  pathname and a target Tahoe directory, this will create a read-only
2306  snapshot of the local directory in $target/Archives/$timestamp. It
2307  will also create $target/Latest, which is a reference to the latest
2308  such snapshot. Each time you run "tahoe backup" with the same source
2309  and target, a new $timestamp snapshot will be added. These snapshots
2310  will share directories that have not changed since the last backup,
2311  to speed up the process and minimize storage requirements. In
2312  addition, a small database is used to keep track of which local
2313  files have been uploaded already, to avoid uploading them a second
2314  time. This drastically reduces the work needed to do a "null backup"
2315  (when nothing has changed locally), making "tahoe backup' suitable
2316  to run from a daily cronjob.
2317
2318  Note that the "tahoe backup" CLI command must be used in conjunction
2319  with a 1.3.0-or-newer Tahoe client node; there was a bug in the
2320  1.2.0 webapi implementation that would prevent the last step (create
2321  $target/Latest) from working.
2322
2323Large Files
2324-----------
2325
2326- The 12GiB (approximate) immutable-file-size limitation is
2327  lifted. This release knows how to handle so-called "v2 immutable
2328  shares", which permit immutable files of up to about 18 EiB (about
2329  3*10^14). These v2 shares are created if the file to be uploaded is
2330  too large to fit into v1 shares. v1 shares are created if the file
2331  is small enough to fit into them, so that files created with
2332  tahoe-1.3.0 can still be read by earlier versions if they are not
2333  too large. Note that storage servers also had to be changed to
2334  support larger files, and this release is the first release in which
2335  they are able to do that. Clients will detect which servers are
2336  capable of supporting large files on upload and will not attempt to
2337  upload shares of a large file to a server which doesn't support it.
2338
2339FTP/SFTP Server
2340---------------
2341
2342- Tahoe now includes experimental FTP and SFTP servers. When
2343  configured with a suitable method to translate username+password
2344  into a root directory cap, it provides simple access to the virtual
2345  filesystem. Remember that FTP is completely unencrypted: passwords,
2346  filenames, and file contents are all sent over the wire in
2347  cleartext, so FTP should only be used on a local (127.0.0.1)
2348  connection. This feature is still in development: there are no unit
2349  tests yet, and behavior with respect to Unicode filenames is
2350  uncertain. Please see `FTP-and-SFTP.rst`_ for
2351  configuration details. (`#512`_, `#531`_)
2352
2353CLI Changes
2354-----------
2355
2356- This release adds the 'tahoe create-alias' command, which is a
2357  combination of 'tahoe mkdir' and 'tahoe add-alias'. This also allows
2358  you to start using a new tahoe directory without exposing its URI in
2359  the argv list, which is publicly visible (through the process table)
2360  on most unix systems.  Thanks to Kevin Reid for bringing this issue
2361  to our attention.
2362
2363- The single-argument form of "tahoe put" was changed to create an
2364  unlinked file. I.e. "tahoe put bar.txt" will take the contents of a
2365  local "bar.txt" file, upload them to the grid, and print the
2366  resulting read-cap; the file will not be attached to any
2367  directories. This seemed a bit more useful than the previous
2368  behavior (copy stdin, upload to the grid, attach the resulting file
2369  into your default tahoe: alias in a child named 'bar.txt').
2370
2371- "tahoe put" was also fixed to handle mutable files correctly: "tahoe
2372  put bar.txt URI:SSK:..." will read the contents of the local bar.txt
2373  and use them to replace the contents of the given mutable file.
2374
2375- The "tahoe webopen" command was modified to accept aliases. This
2376  means "tahoe webopen tahoe:" will cause your web browser to open to
2377  a "wui" page that gives access to the directory associated with the
2378  default "tahoe:" alias. It should also accept leading slashes, like
2379  "tahoe webopen tahoe:/stuff".
2380
2381- Many esoteric debugging commands were moved down into a "debug"
2382  subcommand:
2383
2384 - tahoe debug dump-cap
2385 - tahoe debug dump-share
2386 - tahoe debug find-shares
2387 - tahoe debug catalog-shares
2388 - tahoe debug corrupt-share
2389
2390   The last command ("tahoe debug corrupt-share") flips a random bit
2391   of the given local sharefile. This is used to test the file
2392   verifying/repairing code, and obviously should not be used on user
2393   data.
2394
2395The cli might not correctly handle arguments which contain non-ascii
2396characters in Tahoe v1.3 (although depending on your platform it
2397might, especially if your platform can be configured to pass such
2398characters on the command-line in utf-8 encoding).  See
2399https://tahoe-lafs.org/trac/tahoe-lafs/ticket/565 for details.
2400
2401Web changes
2402-----------
2403
2404- The "default webapi port", used when creating a new client node (and
2405  in the getting-started documentation), was changed from 8123 to
2406  3456, to reduce confusion when Tahoe accessed through a Firefox
2407  browser on which the "Torbutton" extension has been installed. Port
2408  8123 is occasionally used as a Tor control port, so Torbutton adds
2409  8123 to Firefox's list of "banned ports" to avoid CSRF attacks
2410  against Tor. Once 8123 is banned, it is difficult to diagnose why
2411  you can no longer reach a Tahoe node, so the Tahoe default was
2412  changed. Note that 3456 is reserved by IANA for the "vat" protocol,
2413  but there are argueably more Torbutton+Tahoe users than vat users
2414  these days. Note that this will only affect newly-created client
2415  nodes. Pre-existing client nodes, created by earlier versions of
2416  tahoe, may still be listening on 8123.
2417
2418- All deep-traversal operations (start-manifest, start-deep-size,
2419  start-deep-stats, start-deep-check) now use a start-and-poll
2420  approach, instead of using a single (fragile) long-running
2421  synchronous HTTP connection. All these "start-" operations use POST
2422  instead of GET. The old "GET manifest", "GET deep-size", and "POST
2423  deep-check" operations have been removed.
2424
2425- The new "POST start-manifest" operation, when it finally completes,
2426  results in a table of (path,cap), instead of the list of verifycaps
2427  produced by the old "GET manifest". The table is available in
2428  several formats: use output=html, output=text, or output=json to
2429  choose one. The JSON output also includes stats, and a list of
2430  verifycaps and storage-index strings. The "return_to=" and
2431  "when_done=" arguments have been removed from the t=check and
2432  deep-check operations.
2433
2434- The top-level status page (/status) now has a machine-readable form,
2435  via "/status/?t=json". This includes information about the
2436  currently-active uploads and downloads, which may be useful for
2437  frontends that wish to display progress information. There is no
2438  easy way to correlate the activities displayed here with recent wapi
2439  requests, however.
2440
2441- Any files in BASEDIR/public_html/ (configurable) will be served in
2442  response to requests in the /static/ portion of the URL space. This
2443  will simplify the deployment of javascript-based frontends that can
2444  still access wapi calls by conforming to the (regrettable)
2445  "same-origin policy".
2446
2447- The welcome page now has a "Report Incident" button, which is tied
2448  into the "Incident Gatherer" machinery. If the node is attached to
2449  an incident gatherer (via log_gatherer.furl), then pushing this
2450  button will cause an Incident to be signalled: this means recent log
2451  events are aggregated and sent in a bundle to the gatherer. The user
2452  can push this button after something strange takes place (and they
2453  can provide a short message to go along with it), and the relevant
2454  data will be delivered to a centralized incident-gatherer for later
2455  processing by operations staff.
2456
2457- The "HEAD" method should now work correctly, in addition to the
2458  usual "GET", "PUT", and "POST" methods. "HEAD" is supposed to return
2459  exactly the same headers as "GET" would, but without any of the
2460  actual response body data. For mutable files, this now does a brief
2461  mapupdate (to figure out the size of the file that would be
2462  returned), without actually retrieving the file's contents.
2463
2464- The "GET" operation on files can now support the HTTP "Range:"
2465  header, allowing requests for partial content. This allows certain
2466  media players to correctly stream audio and movies out of a Tahoe
2467  grid. The current implementation uses a disk-based cache in
2468  BASEDIR/private/cache/download , which holds the plaintext of the
2469  files being downloaded. Future implementations might not use this
2470  cache. GET for immutable files now returns an ETag header.
2471
2472- Each file and directory now has a "Show More Info" web page, which
2473  contains much of the information that was crammed into the directory
2474  page before. This includes readonly URIs, storage index strings,
2475  object type, buttons to control checking/verifying/repairing, and
2476  deep-check/deep-stats buttons (for directories). For mutable files,
2477  the "replace contents" upload form has been moved here too. As a
2478  result, the directory page is now much simpler and cleaner, and
2479  several potentially-misleading links (like t=uri) are now gone.
2480
2481- Slashes are discouraged in Tahoe file/directory names, since they
2482  cause problems when accessing the filesystem through the
2483  wapi. However, there are a couple of accidental ways to generate
2484  such names. This release tries to make it easier to correct such
2485  mistakes by escaping slashes in several places, allowing slashes in
2486  the t=info and t=delete commands, and in the source (but not the
2487  target) of a t=rename command.
2488
2489Packaging
2490---------
2491
2492- Tahoe's dependencies have been extended to require the
2493  "[secure_connections]" feature from Foolscap, which will cause
2494  pyOpenSSL to be required and/or installed. If OpenSSL and its
2495  development headers are already installed on your system, this can
2496  occur automatically. Tahoe now uses pollreactor (instead of the
2497  default selectreactor) to work around a bug between pyOpenSSL and
2498  the most recent release of Twisted (8.1.0). This bug only affects
2499  unit tests (hang during shutdown), and should not impact regular
2500  use.
2501
2502- The Tahoe source code tarballs now come in two different forms:
2503  regular and "sumo". The regular tarball contains just Tahoe, nothing
2504  else. When building from the regular tarball, the build process will
2505  download any unmet dependencies from the internet (starting with the
2506  index at PyPI) so it can build and install them. The "sumo" tarball
2507  contains copies of all the libraries that Tahoe requires (foolscap,
2508  twisted, zfec, etc), so using the "sumo" tarball should not require
2509  any internet access during the build process. This can be useful if
2510  you want to build Tahoe while on an airplane, a desert island, or
2511  other bandwidth-limited environments.
2512
2513- Similarly, tahoe-lafs.org now hosts a "tahoe-deps" tarball which
2514  contains the latest versions of all these dependencies. This
2515  tarball, located at
2516  https://tahoe-lafs.org/source/tahoe/deps/tahoe-deps.tar.gz, can be
2517  unpacked in the tahoe source tree (or in its parent directory), and
2518  the build process should satisfy its downloading needs from it
2519  instead of reaching out to PyPI.  This can be useful if you want to
2520  build Tahoe from a darcs checkout while on that airplane or desert
2521  island.
2522
2523- Because of the previous two changes ("sumo" tarballs and the
2524  "tahoe-deps" bundle), most of the files have been removed from
2525  misc/dependencies/ . This brings the regular Tahoe tarball down to
2526  2MB (compressed), and the darcs checkout (without history) to about
2527  7.6MB. A full darcs checkout will still be fairly large (because of
2528  the historical patches which included the dependent libraries), but
2529  a 'lazy' one should now be small.
2530
2531- The default "make" target is now an alias for "setup.py build",
2532  which itself is an alias for "setup.py develop --prefix support",
2533  with some extra work before and after (see setup.cfg). Most of the
2534  complicated platform-dependent code in the Makefile was rewritten in
2535  Python and moved into setup.py, simplifying things considerably.
2536
2537- Likewise, the "make test" target now delegates most of its work to
2538  "setup.py test", which takes care of getting PYTHONPATH configured
2539  to access the tahoe code (and dependencies) that gets put in
2540  support/lib/ by the build_tahoe step. This should allow unit tests
2541  to be run even when trial (which is part of Twisted) wasn't already
2542  installed (in this case, trial gets installed to support/bin because
2543  Twisted is a dependency of Tahoe).
2544
2545- Tahoe is now compatible with the recently-released Python 2.6 ,
2546  although it is recommended to use Tahoe on Python 2.5, on which it
2547  has received more thorough testing and deployment.
2548
2549- Tahoe is now compatible with simplejson-2.0.x . The previous release
2550  assumed that simplejson.loads always returned unicode strings, which
2551  is no longer the case in 2.0.x .
2552
2553Grid Management Tools
2554---------------------
2555
2556- Several tools have been added or updated in the misc/ directory,
2557  mostly munin plugins that can be used to monitor a storage grid.
2558
2559 - The misc/spacetime/ directory contains a "disk watcher" daemon
2560   (startable with 'tahoe start'), which can be configured with a set
2561   of HTTP URLs (pointing at the wapi '/statistics' page of a bunch of
2562   storage servers), and will periodically fetch
2563   disk-used/disk-available information from all the servers. It keeps
2564   this information in an Axiom database (a sqlite-based library
2565   available from divmod.org). The daemon computes time-averaged rates
2566   of disk usage, as well as a prediction of how much time is left
2567   before the grid is completely full.
2568
2569 - The misc/munin/ directory contains a new set of munin plugins
2570   (tahoe_diskleft, tahoe_diskusage, tahoe_doomsday) which talk to the
2571   disk-watcher and provide graphs of its calculations.
2572
2573 - To support the disk-watcher, the Tahoe statistics component
2574   (visible through the wapi at the /statistics/ URL) now includes
2575   disk-used and disk-available information. Both are derived through
2576   an equivalent of the unix 'df' command (i.e. they ask the kernel
2577   for the number of free blocks on the partition that encloses the
2578   BASEDIR/storage directory). In the future, the disk-available
2579   number will be further influenced by the local storage policy: if
2580   that policy says that the server should refuse new shares when less
2581   than 5GB is left on the partition, then "disk-available" will
2582   report zero even though the kernel sees 5GB remaining.
2583
2584 - The 'tahoe_overhead' munin plugin interacts with an
2585   allmydata.com-specific server which reports the total of the
2586   'deep-size' reports for all active user accounts, compares this
2587   with the disk-watcher data, to report on overhead percentages. This
2588   provides information on how much space could be recovered once
2589   Tahoe implements some form of garbage collection.
2590
2591Configuration Changes: single INI-format tahoe.cfg file
2592-------------------------------------------------------
2593
2594- The Tahoe node is now configured with a single INI-format file,
2595  named "tahoe.cfg", in the node's base directory. Most of the
2596  previous multiple-separate-files are still read for backwards
2597  compatibility (the embedded SSH debug server and the
2598  advertised_ip_addresses files are the exceptions), but new
2599  directives will only be added to tahoe.cfg . The "tahoe
2600  create-client" command will create a tahoe.cfg for you, with sample
2601  values commented out. (ticket `#518`_)
2602
2603- tahoe.cfg now has controls for the foolscap "keepalive" and
2604  "disconnect" timeouts (`#521`_).
2605
2606- tahoe.cfg now has controls for the encoding parameters:
2607  "shares.needed" and "shares.total" in the "[client]" section. The
2608  default parameters are still 3-of-10.
2609
2610- The inefficient storage 'sizelimit' control (which established an
2611  upper bound on the amount of space that a storage server is allowed
2612  to consume) has been replaced by a lightweight 'reserved_space'
2613  control (which establishes a lower bound on the amount of remaining
2614  space). The storage server will reject all writes that would cause
2615  the remaining disk space (as measured by a '/bin/df' equivalent) to
2616  drop below this value. The "[storage]reserved_space=" tahoe.cfg
2617  parameter controls this setting. (note that this only affects
2618  immutable shares: it is an outstanding bug that reserved_space does
2619  not prevent the allocation of new mutable shares, nor does it
2620  prevent the growth of existing mutable shares).
2621
2622Other Changes
2623-------------
2624
2625- Clients now declare which versions of the protocols they
2626  support. This is part of a new backwards-compatibility system:
2627  https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Versioning .
2628
2629- The version strings for human inspection (as displayed on the
2630  Welcome web page, and included in logs) now includes a platform
2631  identifer (frequently including a linux distribution name, processor
2632  architecture, etc).
2633
2634- Several bugs have been fixed, including one that would cause an
2635  exception (in the logs) if a wapi download operation was cancelled
2636  (by closing the TCP connection, or pushing the "stop" button in a
2637  web browser).
2638
2639- Tahoe now uses Foolscap "Incidents", writing an "incident report"
2640  file to logs/incidents/ each time something weird occurs. These
2641  reports are available to an "incident gatherer" through the flogtool
2642  command. For more details, please see the Foolscap logging
2643  documentation. An incident-classifying plugin function is provided
2644  in misc/incident-gatherer/classify_tahoe.py .
2645
2646- If clients detect corruption in shares, they now automatically
2647  report it to the server holding that share, if it is new enough to
2648  accept the report.  These reports are written to files in
2649  BASEDIR/storage/corruption-advisories .
2650
2651- The 'nickname' setting is now defined to be a UTF-8 -encoded string,
2652  allowing non-ascii nicknames.
2653
2654- The 'tahoe start' command will now accept a --syslog argument and
2655  pass it through to twistd, making it easier to launch non-Tahoe
2656  nodes (like the cpu-watcher) and have them log to syslogd instead of
2657  a local file. This is useful when running a Tahoe node out of a USB
2658  flash drive.
2659
2660- The Mac GUI in src/allmydata/gui/ has been improved.
2661
2662.. _#512: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/512
2663.. _#518: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/518
2664.. _#521: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/521
2665.. _#531: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/531
2666
2667Release 1.2.0 (2008-07-21)
2668''''''''''''''''''''''''''
2669
2670Security
2671--------
2672
2673- This release makes the immutable-file "ciphertext hash tree"
2674  mandatory.  Previous releases allowed the uploader to decide whether
2675  their file would have an integrity check on the ciphertext or not. A
2676  malicious uploader could use this to create a readcap that would
2677  download as one file or a different one, depending upon which shares
2678  the client fetched first, with no errors raised. There are other
2679  integrity checks on the shares themselves, preventing a storage
2680  server or other party from violating the integrity properties of the
2681  read-cap: this failure was only exploitable by the uploader who
2682  gives you a carefully constructed read-cap. If you download the file
2683  with Tahoe 1.2.0 or later, you will not be vulnerable to this
2684  problem. `#491`_
2685
2686  This change does not introduce a compatibility issue, because all
2687  existing versions of Tahoe will emit the ciphertext hash tree in
2688  their shares.
2689
2690Dependencies
2691------------
2692
2693- Tahoe now requires Foolscap-0.2.9 . It also requires pycryptopp 0.5
2694  or newer, since earlier versions had a bug that interacted with
2695  specific compiler versions that could sometimes result in incorrect
2696  encryption behavior. Both packages are included in the Tahoe source
2697  tarball in misc/dependencies/ , and should be built automatically
2698  when necessary.
2699
2700Web API
2701-------
2702
2703- Web API directory pages should now contain properly-slash-terminated
2704  links to other directories. They have also stopped using absolute
2705  links in forms and pages (which interfered with the use of a
2706  front-end load-balancing proxy).
2707
2708- The behavior of the "Check This File" button changed, in conjunction
2709  with larger internal changes to file checking/verification. The
2710  button triggers an immediate check as before, but the outcome is
2711  shown on its own page, and does not get stored anywhere. As a
2712  result, the web directory page no longer shows historical checker
2713  results.
2714
2715- A new "Deep-Check" button has been added, which allows a user to
2716  initiate a recursive check of the given directory and all files and
2717  directories reachable from it. This can cause quite a bit of work,
2718  and has no intermediate progress information or feedback about the
2719  process. In addition, the results of the deep-check are extremely
2720  limited. A later release will improve this behavior.
2721
2722- The web server's behavior with respect to non-ASCII (unicode)
2723  filenames in the "GET save=true" operation has been improved. To
2724  achieve maximum compatibility with variously buggy web browsers, the
2725  server does not try to figure out the character set of the inbound
2726  filename. It just echoes the same bytes back to the browser in the
2727  Content-Disposition header. This seems to make both IE7 and Firefox
2728  work correctly.
2729
2730Checker/Verifier/Repairer
2731-------------------------
2732
2733- Tahoe is slowly acquiring convenient tools to check up on file
2734  health, examine existing shares for errors, and repair files that
2735  are not fully healthy. This release adds a mutable
2736  checker/verifier/repairer, although testing is very limited, and
2737  there are no web interfaces to trigger repair yet. The "Check"
2738  button next to each file or directory on the wapi page will perform
2739  a file check, and the "deep check" button on each directory will
2740  recursively check all files and directories reachable from there
2741  (which may take a very long time).
2742
2743  Future releases will improve access to this functionality.
2744
2745Operations/Packaging
2746--------------------
2747
2748- A "check-grid" script has been added, along with a Makefile
2749  target. This is intended (with the help of a pre-configured node
2750  directory) to check upon the health of a Tahoe grid, uploading and
2751  downloading a few files. This can be used as a monitoring tool for a
2752  deployed grid, to be run periodically and to signal an error if it
2753  ever fails. It also helps with compatibility testing, to verify that
2754  the latest Tahoe code is still able to handle files created by an
2755  older version.
2756
2757- The munin plugins from misc/munin/ are now copied into any generated
2758  debian packages, and are made executable (and uncompressed) so they
2759  can be symlinked directly from /etc/munin/plugins/ .
2760
2761- Ubuntu "Hardy" was added as a supported debian platform, with a
2762  Makefile target to produce hardy .deb packages. Some notes have been
2763  added to `debian.rst`_ about building Tahoe on a debian/ubuntu
2764  system.
2765
2766- Storage servers now measure operation rates and
2767  latency-per-operation, and provides results through the /statistics
2768  web page as well as the stats gatherer. Munin plugins have been
2769  added to match.
2770
2771Other
2772-----
2773
2774- Tahoe nodes now use Foolscap "incident logging" to record unusual
2775  events to their NODEDIR/logs/incidents/ directory. These incident
2776  files can be examined by Foolscap logging tools, or delivered to an
2777  external log-gatherer for further analysis. Note that Tahoe now
2778  requires Foolscap-0.2.9, since 0.2.8 had a bug that complained about
2779  "OSError: File exists" when trying to create the incidents/
2780  directory for a second time.
2781
2782- If no servers are available when retrieving a mutable file (like a
2783  directory), the node now reports an error instead of hanging
2784  forever. Earlier releases would not only hang (causing the wapi
2785  directory listing to get stuck half-way through), but the internal
2786  dirnode serialization would cause all subsequent attempts to
2787  retrieve or modify the same directory to hang as well. `#463`_
2788
2789- A minor internal exception (reported in logs/twistd.log, in the
2790  "stopProducing" method) was fixed, which complained about
2791  "self._paused_at not defined" whenever a file download was stopped
2792  from the web browser end.
2793
2794.. _#463: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/463
2795.. _#491: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/491
2796.. _debian.rst: docs/debian.rst
2797
2798Release 1.1.0 (2008-06-11)
2799''''''''''''''''''''''''''
2800
2801CLI: new "alias" model
2802----------------------
2803
2804- The new CLI code uses an scp/rsync -like interface, in which
2805  directories in the Tahoe storage grid are referenced by a
2806  colon-suffixed alias. The new commands look like:
2807
2808 - tahoe cp local.txt tahoe:virtual.txt
2809 - tahoe ls work:subdir
2810
2811- More functionality is available through the CLI: creating unlinked
2812  files and directories, recursive copy in or out of the storage grid,
2813  hardlinks, and retrieving the raw read- or write- caps through the
2814  'ls' command. Please read `CLI.rst`_ for complete details.
2815
2816wapi: new pages, new commands
2817-----------------------------
2818
2819- Several new pages were added to the web API:
2820
2821 - /helper_status : to describe what a Helper is doing
2822 - /statistics : reports node uptime, CPU usage, other stats
2823 - /file : for easy file-download URLs, see `#221`_
2824 - /cap == /uri : future compatibility
2825
2826- The localdir=/localfile= and t=download operations were
2827  removed. These required special configuration to enable anyways, but
2828  this feature was a security problem, and was mostly obviated by the
2829  new "cp -r" command.
2830
2831- Several new options to the GET command were added:
2832
2833 -  t=deep-size : add up the size of all immutable files reachable from the directory
2834 -  t=deep-stats : return a JSON-encoded description of number of files, size distribution, total size, etc
2835
2836- POST is now preferred over PUT for most operations which cause
2837  side-effects.
2838
2839- Most wapi calls now accept overwrite=, and default to overwrite=true
2840
2841- "POST /uri/DIRCAP/parent/child?t=mkdir" is now the preferred API to
2842  create multiple directories at once, rather than ...?t=mkdir-p .
2843
2844- PUT to a mutable file ("PUT /uri/MUTABLEFILECAP", "PUT
2845  /uri/DIRCAP/child") will modify the file in-place.
2846
2847- more munin graphs in misc/munin/
2848
2849 - tahoe-introstats
2850 - tahoe-rootdir-space
2851 - tahoe_estimate_files
2852 - mutable files published/retrieved
2853 - tahoe_cpu_watcher
2854 - tahoe_spacetime
2855
2856New Dependencies
2857----------------
2858-  zfec 1.1.0
2859-  foolscap 0.2.8
2860-  pycryptopp 0.5
2861-  setuptools (now required at runtime)
2862
2863New Mutable-File Code
2864---------------------
2865
2866- The mutable-file handling code (mostly used for directories) has
2867  been completely rewritten. The new scheme has a better API (with a
2868  modify() method) and is less likely to lose data when several
2869  uncoordinated writers change a file at the same time.
2870
2871- In addition, a single Tahoe process will coordinate its own
2872  writes. If you make two concurrent directory-modifying wapi calls to
2873  a single tahoe node, it will internally make one of them wait for
2874  the other to complete. This prevents auto-collision (`#391`_).
2875
2876- The new mutable-file code also detects errors during publish
2877  better. Earlier releases might believe that a mutable file was
2878  published when in fact it failed.
2879
2880other features
2881--------------
2882
2883- The node now monitors its own CPU usage, as a percentage, measured
2884  every 60 seconds. 1/5/15 minute moving averages are available on the
2885  /statistics web page and via the stats-gathering interface.
2886
2887- Clients now accelerate reconnection to all servers after being
2888  offline (`#374`_). When a client is offline for a long time, it
2889  scales back reconnection attempts to approximately once per hour, so
2890  it may take a while to make the first attempt, but once any attempt
2891  succeeds, the other server connections will be retried immediately.
2892
2893- A new "offloaded KeyGenerator" facility can be configured, to move
2894  RSA key generation out from, say, a wapi node, into a separate
2895  process. RSA keys can take several seconds to create, and so a wapi
2896  node which is being used for directory creation will be unavailable
2897  for anything else during this time. The Key Generator process will
2898  pre-compute a small pool of keys, to speed things up further. This
2899  also takes better advantage of multi-core CPUs, or SMP hosts.
2900
2901- The node will only use a potentially-slow "du -s" command at startup
2902  (to measure how much space has been used) if the "sizelimit"
2903  parameter has been configured (to limit how much space is
2904  used). Large storage servers should turn off sizelimit until a later
2905  release improves the space-management code, since "du -s" on a
2906  terabyte filesystem can take hours.
2907
2908- The Introducer now allows new announcements to replace old ones, to
2909  avoid buildups of obsolete announcements.
2910
2911- Immutable files are limited to about 12GiB (when using the default
2912  3-of-10 encoding), because larger files would be corrupted by the
2913  four-byte share-size field on the storage servers (`#439`_). A later
2914  release will remove this limit. Earlier releases would allow >12GiB
2915  uploads, but the resulting file would be unretrievable.
2916
2917- The docs/ directory has been rearranged, with old docs put in
2918  docs/historical/ and not-yet-implemented ones in docs/proposed/ .
2919
2920- The Mac OS-X FUSE plugin has a significant bug fix: earlier versions
2921  would corrupt writes that used seek() instead of writing the file in
2922  linear order.  The rsync tool is known to perform writes in this
2923  order. This has been fixed.
2924
2925.. _#221: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/221
2926.. _#374: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/374
2927.. _#391: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/391
2928.. _#439: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/439
2929.. _CLI.rst: docs/CLI.rst
Note: See TracBrowser for help on using the repository browser.