[volunteergrid2-l] upload errors

[Marco Tedaldi]

Hi Eric

On 30.06.2012 20:08, erpo41 at gmail.com wrote:
> Would you go into a bit more detail regarding how you worked around the
> problem?
> 
I can try...

> In my case, I have a single directory called backup-T61p containing 1451
> files, the vast majority around 26MB. I can't split those files into
> different directories.
> 
Oh... if you can't split it into subdirectories, than my workaround
won't help you in any way.

Maybe make a script that sorts the files into subdirectoreies just for
the backup? Or don't use the backup-command but just make a script that
uploads the files into the grid...

Or investigate and help the devs to fix the problem in the next version...

> Thanks,
> Eric
> 
> 
> On Sat, Jun 30, 2012 at 11:46 AM, Marco Tedaldi <marco.tedaldi at gmail.com
> <mailto:marco.tedaldi at gmail.com>> wrote:
> 
>     Hello Eric
> 
>     On 30.06.2012 19:22, erpo41 at gmail.com <mailto:erpo41 at gmail.com> wrote:
>     > Hi all,
>     >
>     >
>     >
>     > P.S.
>     > Here's the error I keep getting from the console:
>     > raise
>     >
>     UploadUnhappinessError(msg)\x0aallmydata.interfaces.UploadUnhappinessError:
>     > shares could be placed on only 11 server(s) such that any 5 of
>     them have
>     > enough shares to recover the file, but we were asked to place
>     shares on
>     > at least 13 such servers. (placed all 15 shares, want to place
>     shares on
>     > at least 13 servers such that any 5 of them have enough shares to
>     > recover the file, sent 11 queries to 11 servers, 11 queries placed
>     some
>     > shares, 0 placed none (of which 0 placed none due to the server being
>     > full and 0 placed none due to an error))\x0a"
>     >
>     >
>     I have had that error a lot as well when I wanted to make the whole
>     backup in one go. surprisingly, these errors went away as soon as I
>     started to upload the subdirectories, one by one.
>     I did not have time to investigate is further, but there seem to be
>     problems when trying to upload a lot of files in one process.
> 
>     Since I know that there had been enough servers online and ready to
>     server my request, i guess there is an error in the client. My (quite
>     wild) guess is, that the client does not select a new set of servers for
>     each file uploaded but tries to always use the same ones. And if one of
>     these servers does not respond anymore, you get the happyness error.
> 
>     As a work around you could set the total-servers higher, so it would
>     still be happy when there are some more not accepting the data...
> 
>     Maybe take this to tahoe-dev...
> 
>     best regards
> 
>     Marco