[tahoe-dev] tahoe over satellite

[Aleksandr Milewski]

And you could *really* improve throughput by doing the bulk transfer  
of erasure coded ciphertext over UDP and ignoring packet loss  
entirely...

Sorry... past lives coming back to haunt me. :D

-ZP

On Jun 20, 2008, at 1:49 PM, zooko wrote:

>
> On Jun 20, 2008, at 12:12 PM, Aleksandr Milewski wrote:
>
>> Those are precisely the sorts of network conditions we were designing
>> for both at DF and later at Aspera. I'm actually very surprised  
>> things
>> are working as well as they are, since TCP is mostly useless under
>> those conditions.
>
> Yeah, the throughput it terrible for large files.  The first large
> file just finished:
>
>
> (more details appended)
>
> We could improve this behavior (as well as the behavior in the more
> common case) by implementing pipelining -- ticket #392.  We can see
> that this makes a difference in the common case because the graphs of
> the upload rate for immutable files changed dramatically when we
> reduced the share size from 1 MiB to 128 KiB, in March:
>
> http://allmydata.org/tahoe-figleaf-graph/hanford.allmydata.com-
> tahoe_speedstats_rate.html
>
> Regards,
>
> Zooko
>
> tickets mentioned in this e-mail:
>
> http://allmydata.org/trac/tahoe/ticket/392 # pipeline upload segments
> to make upload faster
>
>                                File Upload Status
>
>      * Started: 10:54:48 20-Jun-2008
>      * Storage Index: xjnmcvpxwbe33fs5ldyqf2c76u
>      * Helper?: No
>      * Total Size: 27529494
>      * Progress (Hash): 100.0%
>      * Progress (Ciphertext): 100.0%
>      * Progress (Encode+Push): 100.0%
>      * Status: Done
>
> Upload Results
>
>      * Shares Pushed: 10
>      * Shares Already Present: 0
>      * Sharemap:
>           * 0 -> Placed on [7tlov7eg]
>           * 1 -> Placed on [trjdor3o]
>           * 2 -> Placed on [lwkv6cji]
>           * 3 -> Placed on [2y7ldksg]
>           * 4 -> Placed on [xiktf6ok]
>           * 5 -> Placed on [ivjakubr]
>           * 6 -> Placed on [6fyx5u4z]
>           * 7 -> Placed on [sgh4c5cr]
>           * 8 -> Placed on [uf7kq2sv]
>           * 9 -> Placed on [wfninubk]
>      * Servermap:
>           * [ivjakubr] got share: #5
>           * [lwkv6cji] got share: #2
>           * [sgh4c5cr] got share: #7
>           * [trjdor3o] got share: #1
>           * [uf7kq2sv] got share: #8
>           * [wfninubk] got share: #9
>           * [xiktf6ok] got share: #4
>           * [2y7ldksg] got share: #3
>           * [6fyx5u4z] got share: #6
>           * [7tlov7eg] got share: #0
>      * Timings:
>           * File Size: 27529494 bytes
>           * Total: 10325.49s (2.7kBps)
>                * Storage Index: 462ms (59.50MBps)
>                * [Contacting Helper]:
>                     * [Helper Already-In-Grid Check]:
>                * [Upload Ciphertext To Helper]: ()
>                * Peer Selection: 25.50s
>                * Encode And Push: 10299.53s (2.7kBps)
>                     * Cumulative Encoding: 3.64s (7.56MBps)
>                     * Cumulative Pushing: 10080.89s (2.7kBps)
>                     * Send Hashes And Close: 203.30s
>                * [Helper Total]:
>    Return to the Welcome Page
>
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev