[tahoe-dev] Very low response

[David-Sarah Hopwood]

Wassim Drira wrote:
> Hi all,
> I am using tahoe v1.7 . I use SFTP interface to communicate with one client.
> The remark was that after pushing huge amount of files in a directory, this
> operation becomes very low after some days of use. It began with 0.3s to
> become around 10s.

This is likely to be due to inefficiencies with large mutable files,
which are used to implement directories (see
http://tahoe-lafs.org/trac/tahoe-lafs/ticket/383 and
http://tahoe-lafs.org/trac/tahoe-lafs/ticket/327).

To confirm this and to eliminate any problem specific to the SFTP frontend,
please try adding a file to the same directory via the web interface. This
should also take 10 seconds.

When using the SFTP interface, it is not possible to add multiple files
to a directory at once, since the SFTP protocol doesn't have any way to
express that. (I suppose it would be possible to batch requests, but that
would probably have to be done as part of more general caching for mutable
files.)

If you can use the 'set_children' webapi request, rather than SFTP (see
docs/frontends/webapi.txt), then that does allow adding multiple child
entries at once. This would not change the time taken by each request
(operating on a directory of a given size), but you might be able to use
fewer requests.

> Do you have a solution to this issue?

Not immediately. The new "MDMF" format for mutable files that is planned
to be added in 1.9.0 *may* help us to implement additions to mutable
directories more efficiently, but without additional caching, I think it
would still be necessary to read the whole directory on each request.

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20100914/fa54c533/attachment.pgp>