[volunteergrid2-l] adding files to the grid

[Earney, Billy C.]

Shawn,

So let me see if I understand this correctly.  The client uploads the encrypted file to the helper.  The helper then splits the file, and creates the extra metadata to help put the file back together (so that the client only needs N out of S shares).  The helper then takes the shares and distributes them to different nodes on the grid.

Can a single client use more than 1 helper?  If so, how does that fit into the configuration?

I would expect a helper to be beneficial when a cpu cannot keep up with the bandwidth, but my load is very low, but it also doesn't look like my bandwidth is saturated either..  So I'm not sure where the bottleneck is..  Any ideas of commands I can run to find the bottleneck?  If figure there are some tahoe debug commands that would help.  I'll look into those.   Does anyone have any other advice?

I don't mind if it takes some time to upload files, but I figure something isn't right on my end with the level of performance I've seen..

________________________________________
From: volunteergrid2-l-bounces at tahoe-lafs.org [volunteergrid2-l-bounces at tahoe-lafs.org] On Behalf Of Shawn Willden [shawn at willden.org]
Sent: Tuesday, March 01, 2011 10:45 AM
To: a small group of people sharing storage resources with Tahoe-LAFS
Subject: Re: [volunteergrid2-l] adding files to the grid

On Tue, Mar 1, 2011 at 8:42 AM, Billy Earney <billy.earney at gmail.com<mailto:billy.earney at gmail.com>> wrote:
Would it be possible to have more than 1 helper in the grid?  If so, if many
of us ran helpers that would speed up adding files to the grid, but would
the contents of the raw file be viewable by the helper?

There is no problem with having many helpers.  Only the clients that are specifically configured to use a helper will do so, and they'll use the helper they're pointed at (there's no automatic discovery/selection of helpers).

The helper receives the file after AES encryption but before splitting into shares.  So the file uploaded to the helper is the same size as the original, but it's already encrypted.

--
Shawn