Changes between Initial Version and Version 8 of Ticket #678
- Timestamp:
- 2011-01-29T21:10:17Z (13 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #678
- Property Keywords newcaps space-efficiency performance added
- Property Summary changed from converge same file, same K, different M to converge same file, same K, different N
-
Ticket #678 – Description
initial v8 1 The underlying erasure code is "systematic", which means the first {{{K}}} shares are simply the segments of the file (except that the last one, which contains the end of the file, is padded out to be the same size as the others). The erasure code also has the property (I don't know what it is called) that the "check shares" or "secondary shares" -- the ones after the first {{{K}}} -- are also the same regardless of what {{{ M}}} is.1 The underlying erasure code is "systematic", which means the first {{{K}}} shares are simply the segments of the file (except that the last one, which contains the end of the file, is padded out to be the same size as the others). The erasure code also has the property (I don't know what it is called) that the "check shares" or "secondary shares" -- the ones after the first {{{K}}} -- are also the same regardless of what {{{N}}} is. 2 2 3 Therefore if you upload a file with, e.g. {{{K=13}}}, {{{ M=16}}} and then you __re-upload__ it with {{{K=13}}}, {{{M=26}}}, then the index 0 through the index 15 share that you upload would be exactly the same as they were in the original upload (if you used the same encryption key to encrypt the file before erasure-coding it).3 Therefore if you upload a file with, e.g. {{{K=13}}}, {{{N=16}}} and then you __re-upload__ it with {{{K=13}}}, {{{N=26}}}, then the index 0 through the index 15 share that you upload would be exactly the same as they were in the original upload (if you used the same encryption key to encrypt the file before erasure-coding it). 4 4 5 However, Tahoe currently doesn't take advantage of this coincidence at all, because it doesn't use the same encryption key for those two files. Instead it includes {{{ M}}} in the generation of the encryption key, and in the resulting immutable-file read-cap, so that the two uploads of the same file with the same {{{K}}} and different {{{M}}} result in completely different sets of shares and different read-caps. specs: [source:docs/specifications/file-encoding.txt@20090222054054-4233b-ce16f0d882f804485e792782c1a9527db25114d0 file-encoding.txt]; source code: [source:src/allmydata/immutable/upload.py@20090304013715-4233b-fbbf44eba801fb41795d4421bbcb7a028d45e6ff#L1116 upload.py]5 However, Tahoe currently doesn't take advantage of this coincidence at all, because it doesn't use the same encryption key for those two files. Instead it includes {{{N}}} in the generation of the encryption key, and in the resulting immutable-file read-cap, so that the two uploads of the same file with the same {{{K}}} and different {{{N}}} result in completely different sets of shares and different read-caps. specs: [source:docs/specifications/file-encoding.txt@20090222054054-4233b-ce16f0d882f804485e792782c1a9527db25114d0 file-encoding.txt]; source code: [source:src/allmydata/immutable/upload.py@20090304013715-4233b-fbbf44eba801fb41795d4421bbcb7a028d45e6ff#L1116 upload.py] 6 6 7 7 To fix this ticket, figure out how to retain all of the safety and security properties that Tahoe immutable-file currently have, while also letting those two uploads share their first 16 shares.
