[tahoe-dev] GridBackup moved to Github

[Shawn Willden]

On Saturday 30 May 2009 07:58:31 am Arthur Lutz wrote:
> I tried it out, here a few things :

Thanks for looking at it.  As you can plainly see, it's far from ready for 
use.  However, if you're willing to do some testing, I'd like to see your 
results.

> * no README at the root of project, it would be nice to have a small
> description of how to use it

Or, in this case, why you shouldn't expect it to work :-)

> * the tests are hard wired to the default python, so since i'm on 
> ubuntu and only python2.5 (and not python2.6 which is default) has
> pycrypto, I couldn't run them

I develop on Ubuntu 8.04 (with python 2.5.2) and 9.04 (with python 2.6), and 
the tests work on my systems.

I just pushed a new commit that uses /usr/bin/env so if you use virtualenv you 
can change the python version.  But it does run fine on 2.5.2 for me.

> * I initially had an import error on _librsync which I did not find after a
> "python setup.py build" (since I guessed it was a python-c module), then I
> found it on my system from the duplicity package (locate _librsync.so)
> copied it and it worked

Yes.  I snitched it from rdiff-backup, but duplicity uses it as well.

> * it's not entirely clear what is code and what is not in the documentation

Yeah, I really need to write some real documentation.

> I'll be watching this project, since the description is what I've been
> looking for. Since I know python I might even try to find time to code a
> bit.

Cool!

I'd love to get some more help with this.  I'll get there eventually on my 
own, but I could get there faster with some help.  The time I have available 
for hacking is sadly limited, and I've recently joined another project (the 
OLPC Math4 project) which is further limiting my available time.  I care 
enough about GridBackup that I will get it going, but it may be months yet 
before there's a really usable version if I'm the only one working on it.

To that end, I'll summarize the current status here, and in a README:

1.  The rudiments of a configuration system are in place, so you can control 
what gets scanned.  If you look in ~/.GridBackup you'll find a config.ini 
file.

2.  The scanner is pretty solid.  It scans the configured portions (by default 
the root partition) and generates signatures, a backup log and an upload job 
queue.  Later scans generate additional backup logs and job queues, 
documenting content and metadata changes (and generating new signatures).

3.  The job queue works fine, but I decided I'm going to replace the flat file 
approach with a sqlite database.  That's my current project.

4.  The uploader, which will read the job queue and actually pushes changes 
into the grid, doesn't exist at all.  A basic uploader should be relatively 
quick to implement once all of the foundations are in place.

5.  The restore engine doesn't exist at all.

6.  The GUI doesn't exist.  However, another contributor is looking into 
building at least a bare bones implementation.  Like all proper software, 
GridBackup will be usable from the command line, but will have a GUI built on 
top of it.

7.  As you mentioned, the whole thing needs a lot of polishing to make it 
usable.

What does exist has been tested fairly well on Linux (Ubuntu 8.04, Ubuntu 9.04 
and Debian Lenny) and a little on Mac OS X (10.5).  I really need to find 
some time to do some serious testing of the scanner on Windows.

Another thing that you could do easily to help out, if you like, is to file 
issues for any problems that you notice on the github issues tab.

Thanks for looking!

	Shawn