Problems uploading immutable files

[Zooko Wilcox-OHearn]

Hi David:

Thanks for writing to report your problem.

On Wed, Apr 8, 2015 at 2:15 PM, David Schneider
<davidschneider.kontakt at gmail.com> wrote:
>
>> The full error message is:
>> ran out of shares: complete= pending=Share(sh5-on-v3otfjwi) overdue=
>> unused= need 3. Last failure: [Failure instance: Traceback (failure with no
>> frames): <class 'allmydata.immutable.downloader.share.LayoutInvalid'>:
>> unknown version 0 (I understand 1 and 2)

I have not seen this before in the wild, but it means that the share,
stored on disk on the storage server, has been corrupted. Since the
version number in the share is "0", when that number should have been
either "1" or "2", I wonder if the local filesystem filled the share
file with all zero bytes.

You could investigate this more by running a "check" operation on that
file, from the client, and by running the "tahoe debug" command on the
command-line on the storage server:

zooko at spark ~/playground/tahoe/tahoe-lafs/bin $ ./tahoe debug --help

Usage: tahoe debug SUBCOMMAND
Subcommands:
    tahoe debug dump-share      Unpack and display the contents of a share.
    tahoe debug dump-cap        Unpack a read-cap or write-cap.
    tahoe debug find-shares     Locate sharefiles in node directories.
    tahoe debug catalog-shares  Describe all shares in node dirs.
    tahoe debug corrupt-share   Corrupt a share by flipping a bit.
    tahoe debug repl            Open a Python interpreter.
    tahoe debug trial           Run tests using Twisted Trial with the
right imports.
    tahoe debug flogtool        Utilities to access log files.

Please run e.g. 'tahoe debug dump-share --help' for more details on each
subcommand.

You could run:

tahoe debug catalog-shares $NODE_DIRECTORY

NODE_DIRECTORY is probably ~/.tahoe/, unless you configured it to be
somewhere else.

It will print out a line of information about each share therein.

For example, on my system:

zooko at spark ~/playground/tahoe/tahoe-lafs/bin $ ./tahoe debug
catalog-shares ./test/s0/
CHK pxwh2r245lu2j55wiq46vcmiea 1/1 644734
xujcpmqedhhi54ebuierwxuy3z3tj2pzfwirzlizlrmlu6572q7a 2678304
'/home/zooko/playground/tahoe/tahoe-lafs/bin/test/s0/storage/shares/px/pxwh2r245lu2j55wiq46vcmiea/0'

You could then cut and paste the filename of the share into "tahoe
debug dump-share", for example:

zooko at spark ~/playground/tahoe/tahoe-lafs/bin $ ./tahoe debug
dump-share /home/zooko/playground/tahoe/tahoe-lafs/bin/test/s0/storage/shares/px/pxwh2r245lu2j55wiq46vcmiea/0
share filename:
'/home/zooko/playground/tahoe/tahoe-lafs/bin/test/s0/storage/shares/px/pxwh2r245lu2j55wiq46vcmiea/0'
             version: 1
           file_size: 644734
        num_segments: 5
        segment_size: 131072
       needed_shares: 1
        total_shares: 1

          codec_name: crs
        codec_params: 131072-1-1
   tail_codec_params: 120446-1-1

      crypttext_hash: yhcii4w2dkkped4475tqtx2eu323yqxgdninsuvk7i3uth6yminq
 crypttext_root_hash: bgc3blozd4sb6465rvwkstqhpotaux7pullvgxoec7przzm7slgq
     share_root_hash: 2524tnool2py55ay4hxni2gt3xbaz7s7kr5p6m4qikaisiflun3q
            UEB_hash: xujcpmqedhhi54ebuierwxuy3z3tj2pzfwirzlizlrmlu6572q7a
          verify-cap:
URI:CHK-Verifier:pxwh2r245lu2j55wiq46vcmiea:xujcpmqedhhi54ebuierwxuy3z3tj2pzfwirzlizlrmlu6572q7a:1:1:644734

 Size of data within the share:
                data: 644734
       uri-extension: 323
          validation: 1474

 Lease #0: owner=0, expire in 2678245s (30 days)


You could also just use "hexdump" on the file to see if it is all zero bytes!



> As upload result of a immutable file I get the following message:
…
>> 8 -> placed on [y2zlzq7z]
>> 9 -> placed on [u64psdjj]

> It kinda seems that it just placed 2 shares instead of 10?

Yes, it does seem so. What is your "shares.happy" config parameter in
your tahoe.cfg file?

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.