[tahoe-dev] speed issues

[erpo41 at gmail.com]

Good afternoon,

I'm a new member of VolunteerGrid2, a tahoe-lafs grid run by volunteers to
pool storage resources. My node is up and running, and other people seem to
be getting acceptable performance storing files on it.

However, when I try to back up my files into the grid, I get very poor
performance: sometimes as low as 1.6kbytes measured by the tahoe client. At
the NIC, I see typical upload rates of perhaps 100kbits/s. This is on a
10mbit/1mbit home cable connection. I would be shocked to discover that
anyone else on the grid has a downstream connection slower than my upstream
connection.

Also, every once in a while, the tahoe client will crash. Here's the
backtrace:

eric at io:/srv/per-user/eric$ tahoe backup backup-T61p/ tahoe:
Traceback (most recent call last):
  File "/home/eric/allmydata-tahoe-1.9.1/support/bin/tahoe", line 9, in
<module>
    load_entry_point('allmydata-tahoe==1.9.1', 'console_scripts', 'tahoe')()
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/runner.py",
line 113, in run
    rc = runner(sys.argv[1:], install_node_control=install_node_control)
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/runner.py",
line 99, in runner
    rc = cli.dispatch[command](so)
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/cli.py",
line 569, in backup
    rc = tahoe_backup.backup(options)
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/tahoe_backup.py",
line 325, in backup
    return bu.run()
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/tahoe_backup.py",
line 118, in run
    new_backup_dircap = self.process(options.from_dir)
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/tahoe_backup.py",
line 194, in process
    childcap, metadata = self.upload(childpath)
  File
"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/scripts/tahoe_backup.py",
line 305, in upload
    raise HTTPError("Error during file PUT", resp)
allmydata.scripts.common_http.HTTPError: Error during file PUT: 500
Internal Server Error
"Traceback (most recent call last):\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/support/lib/python2.6/site-packages/Twisted-12.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py\",
line 551, in _runCallbacks\x0a    current.result = callback(current.result,
*args, **kw)\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/util/pipeline.py\",
line 121, in _call_finished\x0a    d.errback(self.failure)\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/support/lib/python2.6/site-packages/Twisted-12.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py\",
line 397, in errback\x0a    self._startRunCallbacks(fail)\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/support/lib/python2.6/site-packages/Twisted-12.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py\",
line 464, in _startRunCallbacks\x0a    self._runCallbacks()\x0a---
<exception caught here> ---\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/support/lib/python2.6/site-packages/Twisted-12.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py\",
line 551, in _runCallbacks\x0a    current.result = callback(current.result,
*args, **kw)\x0a  File
\"/home/volunteergrid2/allmydata-tahoe-1.9.1/src/allmydata/immutable/encode.py\",
line 501, in _remove_shareholder\x0a    raise
UploadUnhappinessError(msg)\x0aallmydata.interfaces.UploadUnhappinessError:
shares could be placed on only 12 server(s) such that any 5 of them have
enough shares to recover the file, but we were asked to place shares on at
least 13 such servers.: [Failure instance: Traceback (failure with no
frames): <class 'allmydata.util.pipeline.PipelineError'>: <PipelineError
error=([Failure instance: Traceback (failure with no frames): <class
'foolscap.ipb.DeadReferenceError'>: Connection was lost (to tubid=aty4)
(during method=RIBucketWriter:write)\x0a])>\x0a]\x0a"

The folks on the VG2 list suggested that this may be the appropriate place
to report these problems.

Thanks,
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20120422/3dc33b10/attachment.html>