[tahoe-dev] pubgrid share size distribution

[Greg Troxel]

I wrote a quick script to see what the size distribution is of shares on
pubgrid, and ran it on a pubgrid node that I run.  Each file contains
lists of shares that are from [N, 2N) disk blocks.

wc SHARES.*
       0       0       0 SHARES.00001
    1606    1606   63742 SHARES.00002
    6056    6056  238036 SHARES.00004
    1464    1464   57285 SHARES.00008
     719     719   28133 SHARES.00016
     476     476   18608 SHARES.00032
     418     418   16346 SHARES.00064
     356     356   13901 SHARES.00128
     192     192    7503 SHARES.00256
     224     224    8758 SHARES.00512
      92      92    3607 SHARES.01024
      93      93    3645 SHARES.02048
      27      27    1060 SHARES.04096
      76      76    2972 SHARES.08192
       3       3     117 SHARES.16384
      30      30    1191 SHARES.32768
       0       0       0 SHARES.65536
       3       3     117 SHARES.big
   11835   11835  465021 total

In particular, I'm finding lots of shares that fit this find command,
more than I would expect:

  find shares -type f -size +10324 -size -10346

The total usage is about 2.3GB.  Interestingly, almost 800MB was
uploaded in the last 24 hours (net of expirations).   The uploads are
coming from fairly few hosts, including one that sent 345 MB in two
bursts, one around 0030EST/0530Z and one around 1000EST/0500Z.
Discounting about 3 hosts, upload volume is neglible.


----------------------------------------
#!/bin/sh

size=1

while [ "$size" -le 65536 ]; do
    dsize=`expr $size \* 2`
    sprint=`printf "%05d" $size`
    find shares -type f \! -size -$size -size -$dsize > SHARES.$sprint
    size=$dsize
done

find shares -type f \! -size -$size > SHARES.big
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20110223/57ce7ac4/attachment.pgp>