[volunteergrid2-l] adding files to the grid

[Jody Harris]

Yeah, I noticed that uploads where painfully slow last week. It took hours
to upload a 400+ MB file. There could be lots of bottlenecks. The beauty of
Tahoe is in the distributed redundancy of the system, not the speed of
placing data on the nodes.

Steve, I'll see if I can find the helper config from our old grid. If it's
still in tact, it should be a decent go-by.

Keep in mind that if you are using a helper, You'll be able to get the file
or files to the helper as quickly as you can push the original file to that
machine, but it's still going to take time for the helper to spool the data
out to the nodes.

j
----
- Think carefully.


On Tue, Mar 1, 2011 at 8:06 AM, Steve Dodson <steve.dodson at gmail.com> wrote:

> It is my understanding that, without a helper, you are uploading multiple
> copies of each file.  Which leads to my next question; has anyone
> successfully configured a helper, and if so, what guide did you use?  Tahoe
> documentation is a bit lacking IMHO...
>
>
> On 03/01/2011 06:49 AM, Earney, Billy C. wrote:
>
>> Yesterday I was off work, and I decided I’d place a few GB of files on
>> our grid. This seems to be taking quite a bit of time (over 24 hours for
>> ~10GB). I’m thinking the bottleneck could be my upload bandwidth, since
>> the cpu is mainly idling. If my memory serves my upload speed should be
>> around 1Mbps.
>>
>> Any suggestions from veterans on how to debug this, or maybe how to
>> increase upload throughput?
>>
>> Billy Earney
>>
>> earney at umsystem.edu <mailto:earney at umsystem.edu>
>>
>>
>> Programmer/Analyst-Expert
>>
>> Office of Social and Economic Data Analysis (OSEDA)
>>
>> University of Missouri
>>
>> Phone: 573-882-7396
>>
>> Fax: 573-884-4635
>>
>>
>>
>> _______________________________________________
>> volunteergrid2-l mailing list
>> volunteergrid2-l at tahoe-lafs.org
>> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
>> http://bigpig.org/twiki/bin/view/Main/WebHome
>>
>
> --
>
> soli Deo gloria
> _______________________________________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> http://bigpig.org/twiki/bin/view/Main/WebHome
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/cgi-bin/mailman/private/volunteergrid2-l/attachments/20110301/375cf9bb/attachment.html>