Changes between Initial Version and Version 11 of Ticket #928


Ignore:
Timestamp:
2010-01-28T15:44:42Z (15 years ago)
Author:
zooko
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #928 – Description

    initial v11  
    1 The current code ([changeset:2a2cc5c4b8a490ae]) performs immutable file download in two phases.  In phase 1 it sends a {{{get_buckets()}}} remote invocation message to every storage server that it knows about ([source:src/allmydata/immutable/download.py@4164#L871 CiphertextDownloader._get_all_shareholders()]).  In phase 2 it requests the CEB (a.k.a. UEB) from each server in turn until it gets a valid CEB ([source:src/allmydata/immutable/download.py@4164#L956 CiphertextDownloader._obtain_uri_extension()]).  In phase 3 it requests the crypttext hash tree from each server in turn until it gets a valid hash tree ([source:src/allmydata/immutable/download.py@4164#L1002 CiphertextDownloader._get_crypttext_hash_tree()]).  In phase 4 is actually downloads the blocks of data from servers in parallel.  Out of the shares which it learned about in phase 1, which shares does it choose? It chooses "primary shares" first (their sharenum is < K) because those can be erasure-decoded at no computational cost, then it chooses randomly from among secondary shares until it has K.
     1The current code ([changeset:2a2cc5c4b8a490ae]) performs immutable file download in four phases.  In phase 1 it sends a {{{get_buckets()}}} remote invocation message to every storage server that it knows about ([source:src/allmydata/immutable/download.py@4164#L871 CiphertextDownloader._get_all_shareholders()]).  In phase 2 it requests the CEB (a.k.a. UEB) from each server in turn until it gets a valid CEB ([source:src/allmydata/immutable/download.py@4164#L956 CiphertextDownloader._obtain_uri_extension()]).  In phase 3 it requests the crypttext hash tree from each server in turn until it gets a valid hash tree ([source:src/allmydata/immutable/download.py@4164#L1002 CiphertextDownloader._get_crypttext_hash_tree()]).  In phase 4 is actually downloads the blocks of data from servers in parallel.  Out of the shares which it learned about in phase 1, which shares does it choose? It chooses "primary shares" first (their sharenum is < K) because those can be erasure-decoded at no computational cost, then it chooses randomly from among secondary shares until it has K.
    22
    3 Now currently phase 1 does not end, and therefore phase 2 does not begin, until all servers have answered the {{{get_buckets()}}}!  To close this ticket, make it so phase 2 ends as soon as at least {{{K}}} buckets have been found.
     3Now currently phase 1 does not end, and therefore phase 2 does not begin, until all servers have answered the {{{get_buckets()}}}!  To close this ticket, make it so phase 1 ends as soon as at least {{{K}}} buckets have been found.
    44
    55The nice non-invasive way to do this is to replace the DeferredList in [source:src/allmydata/immutable/download.py@4164#L871 CiphertextDownloader._get_all_shareholders()] with an object that acts like a DeferredList but fires once at least {{{K}}} shares (or possibly once at least {{{K}}} servers) have been found.