[tahoe-dev] timestamps on mutable files

[Jody Harris]

I have a script that:
 - iterates though my MySQL tables
 - dumps each table
 - pipes the dump through gzip
 - write the file to a temp file
 - copies the file to [remote storage] (now Tahoe grid)
 - deletes temp file

This gives me a daily snapshot of all of the active databases on my server
every night.

The current backup is ~ 300 MB/night, which isn't bad, but after one year,
that's 110+GB x 3.3 and counting of consumed tahoe space. Unless tahoe has a
better garbage collection system than I have come to believe, that's going
to be a problem on a 500 GB grid.

In my reading of the tahoe docs, I have not come across any alarms
concerning mutable files. Is there something I should be made aware of?

jody
----
- Think carefully.
- Contra mundum - "Against the world" (St. Athanasius)
- Credo ut intelliga - "I believe that I may know" (St. Augustin of Hippo)


On Sun, Jan 3, 2010 at 10:50 PM, Zooko O'Whielacronx <zookog at gmail.com>wrote:

> On Sun, Jan 3, 2010 at 10:33 PM, Jody Harris <havoc at harrisdev.com> wrote:
> > I have set up my web server to store backup files on my tahoe grid,
> shifting
> > from the use of a server at my house. One thing that initially alarmed me
> > was that the timestamps of the backed up files is not updated in the
> > tahoe-lafs interface. I did confirm that the shares timestamps are
> updated.
>
> Tahoe-LAFS doesn't store timestamps or other metadata on files, only
> on links to files, which links are stored in directories.  So if you
> have a directory which contains a link named "README.txt" to version 1
> of your file, and then you upload version 2 of your file and you
> change that directory to link to version 2, then Tahoe-LAFS will
> update the timestamp on that link named "README.txt" when it updates
> the link to point to the new file.
>
> Did you say "mutable files"?  You should probably not be using mutable
> files for anything.  Immutable files (plus directories) probably fit
> all your needs for backups, and mutable files have significant
> performance problems.
>
> How do you trigger these backups -- are you using "tahoe cp" or "tahoe
> backup" or sending HTTP requests to the WAPI or something else?
>
> Regards,
>
> Zooko
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20100103/9f718565/attachment-0001.htm