[tahoe-dev] Tahoe performance

[Andrej Falout]

Hello all,

I wonder if anyone can comment on the results of my upload tests, in
terms of upload speed; Please forgive if I missed something and if
this is completely off the mark. Dropbox and Mesh used just for a
quick reality check.

Data: SourceFiles 3465 SourceFileSize 144 MB. Content: music MIDI files

Link: 2.5MB/s down, 0.2MB/s up (According to speedtest.net, this is
faster then 98% of all Internet connections they measured)

Description of tests:

1) Duplicity over Tahoe, including encryption (Thanks, François!) -
Fastest, but not an option for me [1]
2) plain "tahoe cp -r" - No comment
3) "cp -r" to FUSE mount - Not an option: does not work
4) upload using Windows client - Drag'n'Drop to Allmydata drive - even
less comment
5) Drag'n'Drop to Dropbox using Konqueror
6) Drag'n'Drop to Microsoft Mesh on Windows


Test measurement:

      Test dataset       1MB       50GB

1)     22                   0.152m   5d
2)    167                  1.15m    40d
3)    did not complete [2]
4)    239                  1.65       57d
5)    33[3]                0.22m    7.95d
6)    60/15         not calculated [4]


What am I doing wrong here? If copying half full average modern laptop
will take 40 days, I wonder how does this fit in real world use...

I did not try just committed "tahoe backup", but I assume it will
suffer from same limitations in [1]?

Web interface is apparently not capable of uploading directories?

Also, deleting test data from virtual drive on Windows: 83 minutes -
There MUST be something wrong with this?

Thanks,
--
Andrej Falout

[1] Duplicity, or any other tar based backup is unfortunately not an
option (for me) because:
 - full backup file download requirement to restore even 1 file, which makes it
             a) requires free space on target PC (at least) equal to
the size of whole backup file to both backup and restore
             b) Bandwidth is not free, and in anything less then ADSL2
downloading 50GB is irrational, making it useless for
                     restoring important business document while
traveling and on wireless Internet. Heck, downloading 50GB to
                     restore one file will always be irrational. Plus
waiting for 3 days to be able to get a file is just not acceptable.
 - backup running 5 days will most likely get interrupted at some
point and cannot be resumed
 - Makes the whole Web Browser interface to virtual drive virtually
useless, and the whole "virtual drive" concept pointless.

[2]  92 M in 1197m (when I interrupted it)

[3] First about 10 minutes passed in waiting for upload to start, so
the upload actually took only 23 minutes. As I could not tell what was
going on, I added this to time needed for upload.

[4] Approximately; Mash uploads data in short bursts, so only
approximately one quarter of that time was actually spent uploading.
Which from user perspective can is the same as the action took the
full 60 minutes. Then again, this is not a backup service, but a file
sharing service.