<html><body bgcolor="#FFFFFF"><div>Unfortunately, Andrej's concerns are well-founded. sizelimit= was in 1.2.0, but I removed it from 1.3.0, and reserved_space= does not offer exactly the same functionality. This ought to be mentioned in the NEWS file (and sizelimit= should be listed as removed from docs/configuration.txt .. if not, please file a docs bug ticket).</div><div><br></div><div>I removed sizelimit= because it turned out to be too expensive to use on large storage servers, such as the ones we run here at allmydata.com, which hold several million shares each. Each time the node started, it had to do the python equivalent of /bin/du, to wall through all shares, measure their size, add them all together, then compare the total against the sizelimit= value. This was causing node startup to block for a long time (upwards of 15 minutes), impacting server availability and discouraging us from upgrading servers in a timely fashion.</div><div><br></div><div>In addition, using du to measure space-consumed was inaccurate, as it didn't always take minimum block size into account. Consequently the server could easily use much more space than you wanted it to.</div><div><br></div><div>The new reserved_space= control, added in 1.3.0, uses the python equivalent of /bin/df (specifically os.statvfs), which is practically instantaneous (a single syscall), because the filesystem keeps track of partition-wide space usage continually. However, it doesn't enable the kind of limits that Andrej would like to enforce, and we're still looking for an os.statvfs equivalent for windows (currently reserved_space= is not honored on windows).</div><div><br></div><div>With the new share-crawling framework I just added last week, we could conceivably bring back sizelimit=. it would probably be applied slowly: when first enabled, we do a slow (hours or days) crawl of all shares to add up their size in a non-blocking CPU-yielding manner, then start enforcing the limit once we'd found out how much space we were actually using. We'd persist the results of the crawl to let us get moving faster on subsequent restarts.</div><div><br></div><div>We could also bring sizelimit= back as is, with a warning that it may take a long time to reatart the node when there are a lot of shares.</div><div><br></div><div>Or, the Accounting work I'm slowly accomplishing in the background would provide a more fine-grained limit, and would include a coarse server-wide limit as a side-effect.</div><div><br></div><div>Hope that helps,</div><div> -Brian</div><div><br></div><div><br>On Mar 1, 2009, at 1:44 AM, Rogério Schneider <<a href="mailto:stockrt@gmail.com">stockrt@gmail.com</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div>Andrej, what you want is 'sizelimit'. This configuration limits the utilization of<br>storage for a given client.<br><br>To share only 2GB, for example:<br><br>[storage]<br>enabled = true<br>sizelimit = 2000000000<br>
<br><a href="http://allmydata.org/source/tahoe/trunk/docs/configuration.txt"><a href="http://allmydata.org/source/tahoe/trunk/docs/configuration.txt">http://allmydata.org/source/tahoe/trunk/docs/configuration.txt</a></a><br><br>Regards,<br>Rogério Schneider<br><br><br><div class="gmail_quote">On Sun, Mar 1, 2009 at 3:09 AM, Andrej Falout <span dir="ltr"><<a href="mailto:andrej@falout.org"><a href="mailto:andrej@falout.org">andrej@falout.org</a></a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Thanks Rogério,<div class="Ih2E3d"><br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
That 'reserved_space' is, as documented, the minimum space tahoe will try to keep free in your disk. Say, when you disk goes to only 2gb (in my config) it will stop storing new chunks. This is exactly what you want.<br>
</blockquote></div><br></div>Unless I'm missing something, not exactly.<br><br>On my 3T mount, this would consume all space between currently used 1T and (3T-2G)<br><br clear="all">I'd like to allocate only a portion of (3T-2G) to Tahoe storage, as I will need it in the future. <br>
<br>therefor the need for something like storage_use_max=xxxx parameter.<br><br>IIUC, reserved_space= has the meaning of "dont store any more data if partition space is less then this"<br><br>I would like to be able to declare "use maximum x MB, but only if doing so will not reduce available space under y MB"<br>
<br>-- <br><font color="#888888">Andrej Falout<br><br>
</font><br>_______________________________________________<br>
tahoe-dev mailing list<br>
<a href="mailto:tahoe-dev@allmydata.org"><a href="mailto:tahoe-dev@allmydata.org">tahoe-dev@allmydata.org</a></a><br>
<a href="http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev" target="_blank"><a href="http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev">http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev</a></a><br>
<br></blockquote></div><br><br>-- <br>Rogério Schneider<br><br>MSN: <a href="mailto:stockrt@hotmail.com"><a href="mailto:stockrt@hotmail.com">stockrt@hotmail.com</a></a><br>GTalk: <a href="mailto:stockrt@gmail.com"><a href="mailto:stockrt@gmail.com">stockrt@gmail.com</a></a><br>TerraVoip: stockrt<br>
Skype: stockrt<br>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>tahoe-dev mailing list</span><br><span><a href="mailto:tahoe-dev@allmydata.org">tahoe-dev@allmydata.org</a></span><br><span><a href="http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev">http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev</a></span><br></div></blockquote></body></html>