#643 new enhancement

Automatically schedule repair process (and backups?)

Reported by: mmore Owned by: somebody
Priority: major Milestone: eventually
Component: operational Version: 1.3.0
Keywords: repair availability preservation usability scheduling tahoe-backup Cc: mahmoud.morshedy@…, tahoe-dev@…
Launchpad Bug:

Description (last modified by zhan0903)

1.have Tahoe an automatic repair, i.e. , the introducer monitor the replicas numbers, for example if we have configuration of 10 /3 , so introducer trigger repair process when the replicas became only 4 to keep the file available?

  1. what are the threshold that triggers the repair process?

Change History (17)

comment:1 follow-up: Changed at 2009-02-25T15:21:27Z by terrell

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

comment:2 in reply to: ↑ 1 Changed at 2009-02-26T15:15:46Z by mmore

  • Cc mahmoud.morshedy@… added

Replying to terrell:

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

indeed , my explanation was: currently has Tahoe automatic repair ?, i.e the lost share can be rebuild in order to keep the file available.

comment:3 Changed at 2009-03-05T04:14:26Z by zooko

  • Cc tahoe-dev@… added

It is currently not "automatic" in the sense that there is an established process which you can tell it: "Please repair the following set of files on the following repeating schedule.". Such a process could be built, for example with a bash script or a short Python program, using the wapi or the cli to trigger the repair processes.

Brian Warner has been writing code like that, to trigger repair processes at certain times. I think it isn't quite documented yet but already runs. Also, you could write your own, which might actually be better as the exact set of file to repair and the exact schedule of when to repair them might fit your use better.

comment:4 Changed at 2009-08-04T15:24:58Z by zooko

This introductory article in arstechnica.com reminds me that I've had many conversations with people introducing them to Tahoe-LAFS in which I have to explain that the repair process is not automatic. Anytime I have to repeatedly explain something to newcomers, this makes me think it ought to be changed so that the newcomers's assumptions are true. In this case, that means making a scheduler which automatically kicks off a repair process and runs it in the background.

(The comment by thraxil after the arstechnica.com article incorrectly states that this is currently done automatically. In fact, the user has to learn a special command-line tool and run it themselves or configure it to run as a cron job or whatever.)

comment:6 Changed at 2009-08-27T06:47:18Z by warner

  • Keywords reliability added

comment:7 Changed at 2009-10-27T04:35:23Z by zooko

Kevin Reid posted a cron job to do this http://allmydata.org/pipermail/tahoe-dev/2009-October/003012.html and I linked to it from the RelatedProjects page.

comment:8 Changed at 2009-10-27T04:36:30Z by zooko

So, even though Kevin has demonstrated how to do it with a cron job, this ticket still remains open to make it an integrated and easy-to-configure feature of Tahoe-LAFS itself.

comment:11 Changed at 2009-12-04T04:31:15Z by davidsarah

  • Keywords repair availability dataloss usability added; Repair process reliability removed

comment:12 Changed at 2009-12-13T03:28:50Z by davidsarah

  • Keywords preservation added; dataloss removed

comment:13 Changed at 2009-12-29T19:43:58Z by davidsarah

  • Summary changed from Tahoe Repair process to Automatically schedule repair process

comment:14 Changed at 2009-12-29T19:46:38Z by davidsarah

An earlier ticket discussing a repair service (now partly overtaken by other changes) was #483, now marked as a duplicate of #543.

Last edited at 2013-07-03T09:41:49Z by daira (previous) (diff)

comment:15 in reply to: ↑ 9 Changed at 2010-12-12T23:45:04Z by davidsarah

Replying to davidsarah:

To schedule tasks from Python under [...] Windows:

http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/

However, see #1274 (eliminate pywin32 dependency). The code on that page could be translated to use ctypes, although it's slightly more complicated than usual because it uses COM.

comment:16 Changed at 2011-12-12T04:43:46Z by davidsarah

  • Keywords scheduling tahoe-backup added
  • Milestone changed from undecided to eventually
  • Summary changed from Automatically schedule repair process to Automatically schedule repair process (and backups?)

amiller had some difficulty configuring tahoe backup to run from cron. If we were to add automatic scheduling of repair, we could make it schedule tahoe backup as well, with little extra work.

comment:17 Changed at 2013-07-03T16:49:52Z by zhan0903

  • Description modified (diff)

Is there some implementation about automatical repair? I found it is not convenient to repair only one file or directory every time you run the repair tool

sometimes, you even donnot know which file should be repaired, so I think it will be good to make this tool run as a service and check files automatically based on some principles or algorithm

actually, I would like to do something about that

Note: See TracTickets for help on using tickets.