[volunteergrid2-l] Network speed on volunteergrid2

[Shawn Willden]

At present the performance is limited by the speed and latency of the
slowest server that you're connected to.  The Tahoe implementation delivers
one share to one server before it starts sending the next to the next
server, so one slow node can slow down your whole upload.  I believe this
has been fixed for downloads, so those are parallelized, and faster.

In practice, it's not a big problem for most people who are using it as a
backup solution.  It takes a long time to make your first backup, but unless
you have a high volume of new/changed data, Tahoe's idempotent puts mean
that subsequent backups go very quickly as the system essentially just
notices for each file that it's already in the grid.  The built-in Tahoe
backup tool also tried to avoid even that much overhead.

2011/9/24 Vlastimil Ovčáčík <v.ovcacik at gmail.com>

> Hi Shawn,
>
> thanks for the reply. It's home computer and for various reasons
> (especially undisturbed sleeping) I can't improve its availability. I plan
> to convert it into home server once I buy new pc, but its not going to
> happen any time soon :( As I see it now, it does not make sense to let
> someone with 55 % availability in the grid if you actually maintain 95 %.
>
> The speed sounds painfully, I reckoned that it could be at least 4 Mbps or
> more likely around 10 Mbps. I am wondering if this is typical speed for
> everyone in the grid? Anyway I read on Tahoe wiki that they improved
> transfer speeds in the latest version - 2 MBps to 4 MBps on LAN. I guess
> over-the-internet speeds will improve also.
>
> Vlastimil
>
> On Fri, Sep 23, 2011 at 16:57, Shawn Willden <shawn at willden.org> wrote:
>
>> Hi Vlastimil!
>>
>> 2011/9/23 Vlastimil Ovčáčík <v.ovcacik at gmail.com>
>>
>>> I'm new to this mailing list and I am thinking of joining volunteergrid2
>>> (if you are opened).
>>
>>
>> We are actively seeking more members/storage nodes.
>>
>>
>>> The idea is to use volunteergrid2 as my backup. I'd like to share/give 1
>>> TB and store 300 GB in (3-10 settings).
>>
>>
>> Sounds perfect.
>>
>>
>>> My ISP will soon upgrade my connection to 20/20 Mbps (which will be
>>> effectively 15/15 Mbps or more).
>>
>>
>> Should be just fine.
>>
>>
>>> The only requirement I can't actually meet is 95 % uptime, it is rather
>>> 50 - 60 %.
>>
>>
>> Hmm.  50-60% uptime is pretty low availability.  One of the key goals of
>> this grid is to achieve high availability for both reads and writes.  Thanks
>> to Tahoe, read availability should be near-perfect even with a few nodes
>> down, but write availability is a different matter; we really need to ensure
>> that most of the grid is always available.  Even with 95% per-host uptime,
>> with 20 hosts at least one host will be down 65% of the time.  If we were to
>> accept 60% uptime we'd always have multiple hosts unavailable.
>>
>> Is there something you can do to improve that uptime number?
>>
>>
>>> I am located in Czech Republic.
>>>
>>
>> Cool!  We like wide geographic distribution.
>>
>>
>>> 300 GB is lot of data and I would like to know what's the average/typical
>>> network speed on volunteergrid2? I would really like to know the speed
>>> regardless you are open or I am fitting candidate.
>>
>>
>> I usually get around 100-110 KBps upload speeds.  My upstream bandwidth is
>> about 50 Mbps, so it's Tahoe inefficiency which is the limiting factor, and
>> you can probably expect the same.  At that rate, it takes about one month of
>> uploading 24x7 to upload 300 GB.
>>
>> I use some software I wrote that allows me to upload multiple files in
>> parallel, which allows Tahoe to make better use of my available bandwidth.
>>
>> --
>> Shawn
>>
>> _______________________________________________
>> volunteergrid2-l mailing list
>> volunteergrid2-l at tahoe-lafs.org
>> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
>> http://bigpig.org/twiki/bin/view/Main/WebHome
>>
>
>
> _______________________________________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> http://bigpig.org/twiki/bin/view/Main/WebHome
>



-- 
Shawn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/cgi-bin/mailman/private/volunteergrid2-l/attachments/20110924/db5d9602/attachment.html>