[tahoe-dev] timestamps on mutable files

[Peter Secor]

You are correct Kevan, the hard limit has been removed, sorry I didn't 
realize that had been fixed already!

So, the limit is removed, but Jody you may want to read the ticket #694 
notes, there is a good discussion of why we originally had a hard-limit.

Ps

On 1/4/10 10:24 AM, Kevan Carstensen wrote:
> I'm sure someone will correct me if I'm wrong, but I think #694 removed
> the hard limit on mutable files. So your data should be backed up.
>
> (apologies for formatting - I'm on a phone)
> --
> Kevan Carstensen | <kevan at isnotajoke.com <mailto:kevan at isnotajoke.com>>
>
> On Jan 4, 2010, at 10:10 AM, Jody Harris <imhavoc at gmail.com
> <mailto:imhavoc at gmail.com>> wrote:
>
>> Ack!
>>
>> So, I'm NOT backing up my data!? (I'm just backing up the first 2MB of
>> each compressed table?)
>>
>> Okay, if immutable files are my only choice, how do I do garbage
>> collection to insure that a minimum of dead space is taken up by
>> "yesterday's" files?
>>
>> jody
>> ----
>> - Think carefully.
>> - Contra mundum - "Against the world" (St. Athanasius)
>> - Credo ut intelliga - "I believe that I may know" (St. Augustin of Hippo)
>>
>>
>> On Mon, Jan 4, 2010 at 11:05 AM, Peter Secor <
>> <mailto:secorp at allmydata.com>secorp at allmydata.com
>> <mailto:secorp at allmydata.com>> wrote:
>>
>>     Mutable files currently have a small (2MB?) limit and cannot be
>>     used for
>>     larger files. Immutable files (the default) have a much higher limit
>>     (12GB now?) and are much better suited for the purpose you
>>     described of
>>     large snapshots of data that won't change later.
>>
>>     There are ideas behind larger mutable files (#393) but they are not
>>     implemented yet.
>>
>>     Ps
>>
>>     On 1/4/10 7:37 AM, Jody Harris wrote:
>>     > Oh, I forgot to answer the "how?"
>>     >
>>     > ... put --mutable ...
>>     >
>>     > jody
>>     > ----
>>     > - Think carefully.
>>     > - Contra mundum - "Against the world" (St. Athanasius)
>>     > - Credo ut intelliga - "I believe that I may know" (St. Augustin
>>     of Hippo)
>>     >
>>     >
>>     > On Sun, Jan 3, 2010 at 10:59 PM, Jody Harris <
>>     <mailto:imhavoc at gmail.com>imhavoc at gmail.com <mailto:imhavoc at gmail.com>
>>     > <mailto: <mailto:imhavoc at gmail.com>imhavoc at gmail.com
>>     <mailto:imhavoc at gmail.com>>> wrote:
>>     >
>>     > I have a script that:
>>     > - iterates though my MySQL tables
>>     > - dumps each table
>>     > - pipes the dump through gzip
>>     > - write the file to a temp file
>>     > - copies the file to [remote storage] (now Tahoe grid)
>>     > - deletes temp file
>>     >
>>     > This gives me a daily snapshot of all of the active databases on my
>>     > server every night.
>>     >
>>     > The current backup is ~ 300 MB/night, which isn't bad, but after one
>>     > year, that's 110+GB x 3.3 and counting of consumed tahoe space.
>>     > Unless tahoe has a better garbage collection system than I have come
>>     > to believe, that's going to be a problem on a 500 GB grid.
>>     >
>>     > In my reading of the tahoe docs, I have not come across any alarms
>>     > concerning mutable files. Is there something I should be made
>>     aware of?
>>     >
>>     > jody
>>     > ----
>>     > - Think carefully.
>>     > - Contra mundum - "Against the world" (St. Athanasius)
>>     > - Credo ut intelliga - "I believe that I may know" (St. Augustin of
>>     > Hippo)
>>     >
>>     >
>>     > On Sun, Jan 3, 2010 at 10:50 PM, Zooko O'Whielacronx
>>     > < <mailto:zookog at gmail.com>zookog at gmail.com
>>     <mailto:zookog at gmail.com> <mailto:
>>     <mailto:zookog at gmail.com>zookog at gmail.com
>>     <mailto:zookog at gmail.com>>> wrote:
>>     >
>>     > On Sun, Jan 3, 2010 at 10:33 PM, Jody Harris
>>     > < <mailto:havoc at harrisdev.com>havoc at harrisdev.com
>>     <mailto:havoc at harrisdev.com> <mailto:
>>     <mailto:havoc at harrisdev.com>havoc at harrisdev.com
>>     <mailto:havoc at harrisdev.com>>> wrote:
>>     > > I have set up my web server to store backup files on my tahoe
>>     > grid, shifting
>>     > > from the use of a server at my house. One thing that
>>     > initially alarmed me
>>     > > was that the timestamps of the backed up files is not updated
>>     > in the
>>     > > tahoe-lafs interface. I did confirm that the shares
>>     > timestamps are updated.
>>     >
>>     > Tahoe-LAFS doesn't store timestamps or other metadata on files, only
>>     > on links to files, which links are stored in directories. So if you
>>     > have a directory which contains a link named "README.txt" to
>>     > version 1
>>     > of your file, and then you upload version 2 of your file and you
>>     > change that directory to link to version 2, then Tahoe-LAFS will
>>     > update the timestamp on that link named "README.txt" when it updates
>>     > the link to point to the new file.
>>     >
>>     > Did you say "mutable files"? You should probably not be using
>>     > mutable
>>     > files for anything. Immutable files (plus directories) probably fit
>>     > all your needs for backups, and mutable files have significant
>>     > performance problems.
>>     >
>>     > How do you trigger these backups -- are you using "tahoe cp" or
>>     > "tahoe
>>     > backup" or sending HTTP requests to the WAPI or something else?
>>     >
>>     > Regards,
>>     >
>>     > Zooko
>>     > _______________________________________________
>>     > tahoe-dev mailing list
>>     > <mailto:tahoe-dev at allmydata.org>tahoe-dev at allmydata.org
>>     <mailto:tahoe-dev at allmydata.org> <mailto:
>>     <mailto:tahoe-dev at allmydata.org>tahoe-dev at allmydata.org
>>     <mailto:tahoe-dev at allmydata.org>>
>>     >
>>     <http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev>http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>>     >
>>     >
>>     >
>>     >
>>     >
>>     > _______________________________________________
>>     > tahoe-dev mailing list
>>     > <mailto:tahoe-dev at allmydata.org>tahoe-dev at allmydata.org
>>     <mailto:tahoe-dev at allmydata.org>
>>     >
>>     <http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev>http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>>     _______________________________________________
>>     tahoe-dev mailing list
>>     <mailto:tahoe-dev at allmydata.org>tahoe-dev at allmydata.org
>>     <mailto:tahoe-dev at allmydata.org>
>>     <http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev>http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>>
>>
>> _______________________________________________
>> tahoe-dev mailing list
>> tahoe-dev at allmydata.org <mailto:tahoe-dev at allmydata.org>
>> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
>
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev