1 | diff -rN -u old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/configuration.rst new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/configuration.rst |
---|
2 | --- old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/configuration.rst 2012-06-17 19:38:52.587628030 -0300 |
---|
3 | +++ new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/configuration.rst 2012-06-17 19:38:52.990959790 -0300 |
---|
4 | @@ -365,10 +365,13 @@ |
---|
5 | mutable-type parameter in the webapi. If you do not specify a value here, |
---|
6 | Tahoe-LAFS will use SDMF for all newly-created mutable files. |
---|
7 | |
---|
8 | - Note that this parameter only applies to mutable files. Mutable |
---|
9 | - directories, which are stored as mutable files, are not controlled by |
---|
10 | - this parameter and will always use SDMF. We may revisit this decision in |
---|
11 | - future versions of Tahoe-LAFS. |
---|
12 | + Note that this parameter applies only to files, not to |
---|
13 | + directories. Mutable directories, which are stored in mutable files, are |
---|
14 | + not controlled by this parameter and will always use SDMF. We may revisit |
---|
15 | + this decision in future versions of Tahoe-LAFS. |
---|
16 | + |
---|
17 | + See `<frontends/specifications/mutable.rst>`_ for details about mutable |
---|
18 | + file formats. |
---|
19 | |
---|
20 | Frontend Configuration |
---|
21 | ====================== |
---|
22 | diff -rN -u old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/performance.rst new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/performance.rst |
---|
23 | --- old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/performance.rst 2012-06-17 19:38:52.624294554 -0300 |
---|
24 | +++ new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/performance.rst 2012-06-17 19:38:52.990959790 -0300 |
---|
25 | @@ -10,8 +10,8 @@ |
---|
26 | 6. `Inserting/Removing B bytes in an A-byte mutable file`_ |
---|
27 | 7. `Adding an entry to an A-entry directory`_ |
---|
28 | 8. `Listing an A entry directory`_ |
---|
29 | -9. `Performing a file-check on an A-byte file`_ |
---|
30 | -10. `Performing a file-verify on an A-byte file`_ |
---|
31 | +9. `Checking an A-byte file`_ |
---|
32 | +10. `Verifying an A-byte file (immutable)`_ |
---|
33 | 11. `Repairing an A-byte file (mutable or immutable)`_ |
---|
34 | |
---|
35 | ``K`` indicates the number of shares required to reconstruct the file |
---|
36 | @@ -191,8 +191,8 @@ |
---|
37 | file. This is relatively inexpensive, compared to the verify and repair |
---|
38 | operations. |
---|
39 | |
---|
40 | -Performing a file-verify on an ``A``-byte file |
---|
41 | -============================================== |
---|
42 | +Verifying an A-byte file (immutable) |
---|
43 | +==================================== |
---|
44 | |
---|
45 | cpu: ~N/K*A |
---|
46 | |
---|
47 | @@ -202,8 +202,24 @@ |
---|
48 | |
---|
49 | notes: To verify a file, Tahoe-LAFS downloads all of the ciphertext shares |
---|
50 | that were originally uploaded to the grid and integrity checks them. This is |
---|
51 | -(for well-behaved grids) more expensive than downloading an A-byte file, |
---|
52 | -since only a fraction of these shares are necessary to recover the file. |
---|
53 | +(for grids with good redundancy) more expensive than downloading an A-byte |
---|
54 | +file, since only a fraction of these shares would be necessary to recover the |
---|
55 | +file. |
---|
56 | + |
---|
57 | +Verifying an A-byte file (mutable) |
---|
58 | +================================== |
---|
59 | + |
---|
60 | +cpu: ~N/K*A |
---|
61 | + |
---|
62 | +network: N/K*A |
---|
63 | + |
---|
64 | +memory footprint: N/K*A |
---|
65 | + |
---|
66 | +notes: To verify a file, Tahoe-LAFS downloads all of the ciphertext shares |
---|
67 | +that were originally uploaded to the grid and integrity checks them. This is |
---|
68 | +(for grids with good redundancy) more expensive than downloading an A-byte |
---|
69 | +file, since only a fraction of these shares would be necessary to recover the |
---|
70 | +file. |
---|
71 | |
---|
72 | Repairing an ``A``-byte file (mutable or immutable) |
---|
73 | =================================================== |
---|
74 | diff -rN -u old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/specifications/mutable.rst new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/specifications/mutable.rst |
---|
75 | --- old-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/specifications/mutable.rst 2012-06-17 19:38:52.657627757 -0300 |
---|
76 | +++ new-ticket1398-make_docs_performance_rst_more_precise_and_accurate/docs/specifications/mutable.rst 2012-06-17 19:38:52.994293110 -0300 |
---|
77 | @@ -2,8 +2,6 @@ |
---|
78 | Mutable Files |
---|
79 | ============= |
---|
80 | |
---|
81 | -This describes the "RSA-based mutable files" which were shipped in Tahoe v0.8.0. |
---|
82 | - |
---|
83 | 1. `Mutable Formats`_ |
---|
84 | 2. `Consistency vs. Availability`_ |
---|
85 | 3. `The Prime Coordination Directive: "Don't Do That"`_ |
---|
86 | @@ -19,33 +17,37 @@ |
---|
87 | 6. `Large Distributed Mutable Files`_ |
---|
88 | 7. `TODO`_ |
---|
89 | |
---|
90 | -Mutable File Slots are places with a stable identifier that can hold data |
---|
91 | -that changes over time. In contrast to CHK slots, for which the |
---|
92 | -URI/identifier is derived from the contents themselves, the Mutable File Slot |
---|
93 | -URI remains fixed for the life of the slot, regardless of what data is placed |
---|
94 | -inside it. |
---|
95 | +Mutable files are places with a stable identifier that can hold data that |
---|
96 | +changes over time. In contrast to immutable slots, for which the |
---|
97 | +identifier/capability is derived from the contents themselves, the Mutable |
---|
98 | +file identifier remains fixed for the life of the slot, regardless of what |
---|
99 | +data is placed inside it. |
---|
100 | |
---|
101 | -Each mutable slot is referenced by two different URIs. The "read-write" URI |
---|
102 | +Each mutable file is referenced by two different caps. The "read-write" cap |
---|
103 | grants read-write access to its holder, allowing them to put whatever |
---|
104 | -contents they like into the slot. The "read-only" URI is less powerful, only |
---|
105 | +contents they like into the slot. The "read-only" cap is less powerful, only |
---|
106 | granting read access, and not enabling modification of the data. The |
---|
107 | -read-write URI can be turned into the read-only URI, but not the other way |
---|
108 | +read-write cap can be turned into the read-only cap, but not the other way |
---|
109 | around. |
---|
110 | |
---|
111 | -The data in these slots is distributed over a number of servers, using the |
---|
112 | -same erasure coding that CHK files use, with 3-of-10 being a typical choice |
---|
113 | -of encoding parameters. The data is encrypted and signed in such a way that |
---|
114 | -only the holders of the read-write URI will be able to set the contents of |
---|
115 | -the slot, and only the holders of the read-only URI will be able to read |
---|
116 | -those contents. Holders of either URI will be able to validate the contents |
---|
117 | -as being written by someone with the read-write URI. The servers who hold the |
---|
118 | -shares cannot read or modify them: the worst they can do is deny service (by |
---|
119 | -deleting or corrupting the shares), or attempt a rollback attack (which can |
---|
120 | -only succeed with the cooperation of at least k servers). |
---|
121 | +The data in these files is distributed over a number of servers, using the |
---|
122 | +same erasure coding that immutable files use, with 3-of-10 being a typical |
---|
123 | +choice of encoding parameters. The data is encrypted and signed in such a way |
---|
124 | +that only the holders of the read-write cap will be able to set the contents |
---|
125 | +of the slot, and only the holders of the read-only cap will be able to read |
---|
126 | +those contents. Holders of either cap will be able to validate the contents |
---|
127 | +as being written by someone with the read-write cap. The servers who hold the |
---|
128 | +shares are not automatically given the ability read or modify them: the worst |
---|
129 | +they can do is deny service (by deleting or corrupting the shares), or |
---|
130 | +attempt a rollback attack (which can only succeed with the cooperation of at |
---|
131 | +least k servers). |
---|
132 | |
---|
133 | Mutable Formats |
---|
134 | =============== |
---|
135 | |
---|
136 | +History |
---|
137 | +------- |
---|
138 | + |
---|
139 | When mutable files first shipped in Tahoe-0.8.0 (15-Feb-2008), the only |
---|
140 | version available was "SDMF", described below. This was a |
---|
141 | limited-functionality placeholder, intended to be replaced with |
---|
142 | @@ -75,8 +77,11 @@ |
---|
143 | handled by the old SDMF code). In the fall of 2011, Kevan's code was finally |
---|
144 | integrated, and first made available in the Tahoe-1.9.0 release. |
---|
145 | |
---|
146 | -The main improvement of MDMF is the use of multiple segments: individual |
---|
147 | -128KiB sections of the file can be retrieved or modified independently. The |
---|
148 | +SDMF vs. MDMF |
---|
149 | +------------- |
---|
150 | + |
---|
151 | +The improvement of MDMF is the use of multiple segments: individual 128 KiB |
---|
152 | +sections of the file can be retrieved or modified independently. The |
---|
153 | improvement can be seen when fetching just a portion of the file (using a |
---|
154 | Range: header on the webapi), or when modifying a portion (again with a |
---|
155 | Range: header). It can also be seen indirectly when fetching the whole file: |
---|
156 | @@ -84,12 +89,14 @@ |
---|
157 | than from an SDMF file, although the overall download will then proceed at |
---|
158 | the same rate. |
---|
159 | |
---|
160 | -We've decided to make it opt-in for the first release while we shake out the |
---|
161 | -bugs, just in case a problem is found which requires an incompatible format |
---|
162 | -change. All new mutable files will be in SDMF format unless the user |
---|
163 | -specifically chooses to use MDMF instead. The code can read and modify |
---|
164 | -existing files of either format without user intervention. We expect to make |
---|
165 | -MDMF the default in a subsequent release, perhaps 2.0. |
---|
166 | +We've decided to make it opt-in for now—mutable files default to SDMF format |
---|
167 | +unless explicitly configured to use MDMF, either in tahoe.cfg (see |
---|
168 | +`<configuration.rst>`_) or in the WUI or CLI command that created a new |
---|
169 | +mutable file. |
---|
170 | + |
---|
171 | +The code can read and modify existing files of either format without user |
---|
172 | +intervention. We expect to make MDMF the default in a subsequent release, |
---|
173 | +perhaps 2.0. |
---|
174 | |
---|
175 | Which format should you use? SDMF works well for files up to a few MB, and |
---|
176 | can be handled by older versions (Tahoe-1.8.3 and earlier). If you do not |
---|