[tahoe-dev] timestamps on mutable files

[Jody Harris]

Ack!

So, I'm NOT backing up my data!? (I'm just backing up the first 2MB of each
compressed table?)

Okay, if immutable files are my only choice, how do I do garbage collection
to insure that a minimum of dead space is taken up by "yesterday's" files?

jody
----
- Think carefully.
- Contra mundum - "Against the world" (St. Athanasius)
- Credo ut intelliga - "I believe that I may know" (St. Augustin of Hippo)


On Mon, Jan 4, 2010 at 11:05 AM, Peter Secor <secorp at allmydata.com> wrote:

> Mutable files currently have a small (2MB?) limit and cannot be used for
> larger files. Immutable files (the default) have a much higher limit
> (12GB now?) and are much better suited for the purpose you described of
> large snapshots of data that won't change later.
>
> There are ideas behind larger mutable files (#393) but they are not
> implemented yet.
>
> Ps
>
> On 1/4/10 7:37 AM, Jody Harris wrote:
> > Oh, I forgot to answer the "how?"
> >
> > ... put --mutable ...
> >
> > jody
> > ----
> > - Think carefully.
> > - Contra mundum - "Against the world" (St. Athanasius)
> > - Credo ut intelliga - "I believe that I may know" (St. Augustin of
> Hippo)
> >
> >
> > On Sun, Jan 3, 2010 at 10:59 PM, Jody Harris <imhavoc at gmail.com
> > <mailto:imhavoc at gmail.com>> wrote:
> >
> >     I have a script that:
> >       - iterates though my MySQL tables
> >       - dumps each table
> >       - pipes the dump through gzip
> >       - write the file to a temp file
> >       - copies the file to [remote storage] (now Tahoe grid)
> >       - deletes temp file
> >
> >     This gives me a daily snapshot of all of the active databases on my
> >     server every night.
> >
> >     The current backup is ~ 300 MB/night, which isn't bad, but after one
> >     year, that's 110+GB x 3.3 and counting of consumed tahoe space.
> >     Unless tahoe has a better garbage collection system than I have come
> >     to believe, that's going to be a problem on a 500 GB grid.
> >
> >     In my reading of the tahoe docs, I have not come across any alarms
> >     concerning mutable files. Is there something I should be made aware
> of?
> >
> >     jody
> >     ----
> >     - Think carefully.
> >     - Contra mundum - "Against the world" (St. Athanasius)
> >     - Credo ut intelliga - "I believe that I may know" (St. Augustin of
> >     Hippo)
> >
> >
> >     On Sun, Jan 3, 2010 at 10:50 PM, Zooko O'Whielacronx
> >     <zookog at gmail.com <mailto:zookog at gmail.com>> wrote:
> >
> >         On Sun, Jan 3, 2010 at 10:33 PM, Jody Harris
> >         <havoc at harrisdev.com <mailto:havoc at harrisdev.com>> wrote:
> >          > I have set up my web server to store backup files on my tahoe
> >         grid, shifting
> >          > from the use of a server at my house. One thing that
> >         initially alarmed me
> >          > was that the timestamps of the backed up files is not updated
> >         in the
> >          > tahoe-lafs interface. I did confirm that the shares
> >         timestamps are updated.
> >
> >         Tahoe-LAFS doesn't store timestamps or other metadata on files,
> only
> >         on links to files, which links are stored in directories.  So if
> you
> >         have a directory which contains a link named "README.txt" to
> >         version 1
> >         of your file, and then you upload version 2 of your file and you
> >         change that directory to link to version 2, then Tahoe-LAFS will
> >         update the timestamp on that link named "README.txt" when it
> updates
> >         the link to point to the new file.
> >
> >         Did you say "mutable files"?  You should probably not be using
> >         mutable
> >         files for anything.  Immutable files (plus directories) probably
> fit
> >         all your needs for backups, and mutable files have significant
> >         performance problems.
> >
> >         How do you trigger these backups -- are you using "tahoe cp" or
> >         "tahoe
> >         backup" or sending HTTP requests to the WAPI or something else?
> >
> >         Regards,
> >
> >         Zooko
> >         _______________________________________________
> >         tahoe-dev mailing list
> >         tahoe-dev at allmydata.org <mailto:tahoe-dev at allmydata.org>
> >         http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
> >
> >
> >
> >
> >
> > _______________________________________________
> > tahoe-dev mailing list
> > tahoe-dev at allmydata.org
> > http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20100104/e3502f06/attachment-0001.htm