[tahoe-lafs-trac-stream] [tahoe-lafs] #699: rebalance during repair or upload

[tahoe-lafs]

#699: rebalance during repair or upload
-------------------------+-------------------------------------------------
     Reporter:  zooko    |      Owner:
         Type:  defect   |     Status:  new
     Priority:  major    |  Milestone:  eventually
    Component:  code-    |    Version:  1.4.1
  peerselection          |   Keywords:  upload repair preservation test
   Resolution:           |  anti-censorship
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Old description:

> In [http://allmydata.org/pipermail/tahoe-dev/2009-May/001729.html this
> mailing list message], Humberto Ortiz-Zuazaga asks how to rebalance the
> shares of a file.  To close this ticket, ensure (either as an option or
> unconditionally) that repairing or uploading a file attempts to
> "rebalance" its shares, making it healthy as defined by #614.
>
> See also related tickets #481 (build some share-migration tools), #231
> (good handling of small numbers of servers, or strange choice of
> servers), #232 (Peer selection doesn't rebalance shares on overwrite of
> mutable file.), and #543 ('rebalancing manager')

New description:

 In [http://allmydata.org/pipermail/tahoe-dev/2009-May/001729.html this
 mailing list message], Humberto Ortiz-Zuazaga asks how to rebalance the
 shares of a file.  To close this ticket, ensure (either as an option or
 unconditionally) that repairing or uploading a file attempts to
 "rebalance" its shares, making it healthy as defined by #614.

 See also related tickets #481 (build some share-migration tools), #231
 (good handling of small numbers of servers, or strange choice of servers),
 #232 (Peer selection doesn't rebalance shares on overwrite of mutable
 file.), and #543 ('rebalancing manager')

--

Comment (by sickness):

 Replying to [comment:14 amontero]:
 > Replying to [comment:12 sickness]:
 > > I've found the same behaviour experimenting with 1.8.0 on 10 nodes
 with 3-7-10.
 > > I've tried to upload a file with one of the servers shut down, it said
 ok and put:
 > > 1 share on 8 servers, 2 shares on 1 server, 0 shares on the shut down
 server (obviously).
 > > Then when I've powered the shutdown server back on, I've tried to do a
 repair on that file, it
 > > said that it needed rebalance but didn't do it no matter what.
 > > The solution was following zooko's advice in this mail:
 > > http://tahoe-lafs.org/pipermail/tahoe-dev/2009-May/001735.html
 > > and
 > > http://tahoe-lafs.org/pipermail/tahoe-dev/2009-May/001739.html
 > > So basically I've deleted 1 of the 2 shares of the server with 2
 shares and then repaired the file.
 > > Now all the servers had 1 share.
 > > I'd like to have an option to tell the uploader to never put more than
 N shares per server "no matter what" (I'd set it at 1 because I don't want
 one server to hold more "responsability" that I've planned to...)
 > > tnx! :)
 >
 > Hi sickness.
 >
 > I'm following zooko's advice, since I have the same problem. My use case
 is described in #1657 if you're interested.
 > I've created a [http://pastebin.com/6w8AibEf bash script] that may help
 you. It's not the most efficient way of doing it, but saves me a lot of
 time.
 > If some maintainer can upload it to some misc tools folder in the repo,
 maybe others will be able to use it as starting point and hopefully
 improve it, since my bash skills are somewhat limited.
 >
 > Hope you find it useful.
 yeah! tnx for this script!
 it would be really useful, but I've found it just now :/
 and it spells out this error:
 tahoe-prune.sh: line 49: read: -i: invalid option
 read: usage: read [-ers] [-u fd] [-t timeout] [-p prompt] [-a array] [-n
 nchars] [-d delim] [name ...]
 tahoe-prune.sh: line 55: read: -i: invalid option
 read: usage: read [-ers] [-u fd] [-t timeout] [-p prompt] [-a array] [-n
 nchars] [-d delim] [name ...]
 (this on a debian with bash)
 then I've also found this tool:
 http://killyourtv.i2p.to/tahoe-lafs/rebalance-shares.py/
 it seems to basically do the same but in python

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/699#comment:15>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage