[tahoe-dev] Help uploading when file exists but needs repair

[Brian Warner]

On 11/30/10 10:06 PM, Kyle Markley wrote:
> 
> I have a fun problem and need some advice.

Yay fun problems! :)

> I'm having trouble uploading an immutable file (via "tahoe backup")
> because the file already exists on the grid, but is unhealthy
> (UploadUnhappinessError).

Huh? Shouldn't the new upload just put new shares in place? I know our
uploader isn't particularly clever in the face of existing shares (it
will put multiple shares on one server, and in general not achieve the
ideal diversity), but it shouldn't just fail.

Has something changed recently? Did the "servers of happiness" change
make this sort of thing start to fail?

Kyle: could you do a 'check' on the file that is causing this problem?
I'd like to know k, N, and which shares were where. Ideally I'd like to
reproduce the problem by uploading a file to a local testgrid and then
deleting the right number of shares.

The 'tahoe backup' step is supposed to do a file-check every once in a
while (I think every 4-8 weeks, on average), and if it sees a problem,
it is supposed to upload the file anyways to make it healthy again.

> Oops - I can't!  Due to some poor planning on my part, some significant
> chunks of the data on my grid have already expired.  In particular, I
> have a few unrecoverable directories that prevent me from navigating
> much in my grid.

Almost all of the directories created by modern 'tahoe backup' (1.7.0 or
later, I think) are immutable. Only the top-most 'Archives' directory
(and maybe its sibling) are mutable. So the next time you do a backup,
it should just regenerate any that can't be recovered.

  This means I don't have the caps for the file(s)
> giving me the UploadUnhappinessError.  (I don't think I can get them
> from the backupdb either -- (1) I don't know how, and (2) they might
> only exist in the backupdb in the grid, which might be unreachable.)

The backupdb is strictly local: ~/.tahoe/private/backupdb.sqlite (I
think). Use /usr/bin/sqlite3 to issue a '.schema' command. There should
be a pretty simple mapping from absolute pathname to filecap.

> Help!  What can I do to recover from this?  (I would prefer not to
> change the convergence secret or wipe out all the data in the grid.)

Absolutely! Clients should never fail to *upload* because of existing
shares. We'll get to the bottom of this.

cheers,
 -Brian