converge same file, same K, different M

[zooko]

The underlying erasure code is "systematic", which means the first K shares are simply the segments of the file (except that the last one, which contains the end of the file, is padded out to be the same size as the others). The erasure code also has the property (I don't know what it is called) that the "check shares" or "secondary shares" -- the ones after the first K -- are also the same regardless of what M is.

Therefore if you upload a file with, e.g. K=13, M=16 and then you re-upload it with K=13, M=26, then the index 0 through the index 15 share that you upload would be exactly the same as they were in the original upload (if you used the same encryption key to encrypt the file before erasure-coding it).

However, Tahoe currently doesn't take advantage of this coincidence at all, because it doesn't use the same encryption key for those two files. Instead it includes M in the generation of the encryption key, and in the resulting immutable-file read-cap, so that the two uploads of the same file with the same K and different M result in completely different sets of shares and different read-caps. specs: file-encoding.txt; source code: upload.py

To fix this ticket, figure out how to retain all of the safety and security properties that Tahoe immutable-file currently have, while also letting those two uploads share their first 16 shares.

(By the way, the reason I was reminded of this is that I'm currently doing an upload exactly like this: ​dogfood tasting report . :-))