[tahoe-dev] new stable grid volunteers

[Brian Warner]

Unfortunately, Andrej's concerns are well-founded. sizelimit= was in  
1.2.0, but I removed it from 1.3.0, and reserved_space= does not offer  
exactly the same functionality. This ought to be mentioned in the NEWS  
file (and sizelimit= should be listed as removed from docs/ 
configuration.txt .. if not, please file a docs bug ticket).

I removed sizelimit= because it turned out to be too expensive to use  
on large storage servers, such as the ones we run here at  
allmydata.com, which hold several million shares each. Each time the  
node started, it had to do the python equivalent of /bin/du, to wall  
through all shares, measure their size, add them all together, then  
compare the total against the sizelimit= value. This was causing node  
startup to block for a long time (upwards of 15 minutes), impacting  
server availability and discouraging us from upgrading servers in a  
timely fashion.

In addition, using du to measure space-consumed was inaccurate, as it  
didn't always take minimum block size into account. Consequently the  
server could easily use much more space than you wanted it to.

The new reserved_space= control, added in 1.3.0, uses the python  
equivalent of /bin/df (specifically os.statvfs), which is practically  
instantaneous (a single syscall), because the filesystem keeps track  
of partition-wide space usage continually. However, it doesn't enable  
the kind of limits that Andrej would like to enforce, and we're still  
looking for an os.statvfs equivalent for windows (currently  
reserved_space= is not honored on windows).

With the new share-crawling framework I just added last week, we could  
conceivably bring back sizelimit=. it would probably be applied  
slowly: when first enabled, we do a slow (hours or days) crawl of all  
shares to add up their size in a non-blocking CPU-yielding manner,  
then start enforcing the limit once we'd found out how much space we  
were actually using. We'd persist the results of the crawl to let us  
get moving faster on subsequent restarts.

We could also bring sizelimit= back as is, with a warning that it may  
take a long time to reatart the node when there are a lot of shares.

Or, the Accounting work I'm slowly accomplishing in the background  
would provide a more fine-grained limit, and would include a coarse  
server-wide limit as a side-effect.

Hope that helps,
  -Brian


On Mar 1, 2009, at 1:44 AM, Rogério Schneider <stockrt at gmail.com>  
wrote:

> Andrej, what you want is 'sizelimit'. This configuration limits the  
> utilization of
> storage for a given client.
>
> To share only 2GB, for example:
>
> [storage]
> enabled = true
> sizelimit = 2000000000
>
> http://allmydata.org/source/tahoe/trunk/docs/configuration.txt
>
> Regards,
> Rogério Schneider
>
>
> On Sun, Mar 1, 2009 at 3:09 AM, Andrej Falout <andrej at falout.org>  
> wrote:
> Thanks Rogério,
>
>
> That 'reserved_space' is, as documented, the minimum space tahoe  
> will try to keep free in your disk. Say, when you disk goes to only  
> 2gb (in my config) it will stop storing new chunks. This is exactly  
> what you want.
>
> Unless I'm missing something, not exactly.
>
> On my 3T mount, this would consume all space between currently used  
> 1T and (3T-2G)
>
> I'd like to allocate only a portion of (3T-2G) to Tahoe storage, as  
> I will need it in the future.
>
> therefor the need for something like storage_use_max=xxxx parameter.
>
> IIUC, reserved_space= has the meaning of "dont store any more data  
> if partition space is less then this"
>
> I would like to be able to declare "use maximum x MB, but only if  
> doing so will not reduce available space under y MB"
>
> -- 
> Andrej Falout
>
>
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
>
>
> -- 
> Rogério Schneider
>
> MSN: stockrt at hotmail.com
> GTalk: stockrt at gmail.com
> TerraVoip: stockrt
> Skype: stockrt
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20090301/31430a27/attachment.htm