the timers used by tahoe backup to trigger repair for unhealthy files should be configurable

[davidsarah]

Reported by Kyle Markley:

I am nervous about synchronization between backupdb.sqlite and the grid. As happened recently, if a large chunk of the grid goes down, some files may go below their minimum number of shares. I could repair those files by re-uploading them, but the backupdb assumes those files are fine. So those files won't be reuploaded and they'll stay broken on the grid -- possibly forever. I'll think they're backed up, but they aren't! I'd like to have some way for a deep-check to inject a dose of reality into my backupdb.

For extra credit, I'd like behavior that's a hybrid between backup and deep-check. Run a backup operation, but check every file as we go along, and repair or re-upload the file if the grid doesn't have enough shares.

(Actually it should be if there are insufficient servers of happiness for the file, as per #614.)

[warner]

Take a look at source:docs/backupdb.txt . Clients of the backupdb (like "tahoe backup") are required to eventually do a filecheck on the files that they skip uploading. It uses an age-since-last-check -based probability. The timer values are set such that all files will be checked at least every two months. If the check indicates unhealthy, the file is re-uploaded.

It would probably be useful to make these timers configurable. Setting them to "zero" would give you the check-each-time behavior you'd like. source:src/allmydata/scripts/backupdb.py#L163​ (NO_CHECK_BEFORE and ALWAYS_CHECK_AFTER in BackupDB_v2) are the values that would need configuring.