[volunteergrid2-l] Backing up Files

Wed Feb 8 20:12:45 UTC 2012

Nope, they'll stay around until they're GC'd.

More precisely, they'll stay around as long as they've been re-leased
within the timeout period, and they'll get re-leased every time you
re-lease your root dircap, unless you have removed the archive directory
that references them.  I don't know if old archive directories get removed
automatically eventually, but I doubt it.  You probably have to do it
manually.

On Wed, Feb 8, 2012 at 12:43 PM, Brad Rupp <bradrupp at gmail.com> wrote:

> Thanks Shawn.
>
> One more question for the group.  Since tahoe backup does a versioned
> backup, is there any way to remove old files from the grid that have been
> deleted on the source?  For example, say I backup my pictures each night
> using tahoe backup.  Now, a month later, I do some house cleaning and
> remove a bunch of old pictures.  My understanding is that on the grid those
> files will not be in the current version of my backup, but they will be in
> the version from 30 days ago.  Is there a way to remove them so I'm not
> wasting space on the grid with files I no longer care about?
>
> I know this will eventually be taken care of by the garbage collector. I
> just don't want to waste space for a year with old files.
>
> Thanks,
>
> Brad
>
>
> On 2/7/2012 3:47 PM, Shawn Willden wrote:
>
>> The best way to handle that, I think, is to do it in small bites.  The
>> actual data files are stored as immutable files, and immutable uploads
>> are idempotent... meaning that once a file is in the grid, subsequent
>> attempts to upload it will succeed quickly when Tahoe notices that the
>> file is already present.
>>
>> So, pick a reasonably-sized subdir and run tahoe backup on that.  Then
>> another, and another, etc.  Once you've got everything uploaded, then
>> you can run a top-level backup.  That will be somewhat time-consuming
>> because your Tahoe node will still have to do all of the computations on
>> every file, but it'll go much faster because when it starts doing the
>> actual upload if it finds that there are already enough shares present
>> it won't bother with uploading.
>>
>> After your first big top-level backup, then you can re-run the backup
>> whenever, and it should be pretty fast.  Tahoe backup keeps a local
>> database so it can recognize when files haven't change and skip them
>> (completely skip them, not encode-then-notice-it's-**unnecessary).
>>
>> On Tue, Feb 7, 2012 at 3:28 PM, Brad Rupp <bradrupp at gmail.com
>> <mailto:bradrupp at gmail.com>> wrote:
>>
>>    Oh boy, I definitely have more data than can be uploaded in a few
>>    days.  That is part of the fun I am facing with getting this done.
>>
>>    Brad
>>
>>
>>    On 2/7/2012 2:11 PM, Shawn Willden wrote:
>>
>>        For relatively small amounts of data, tahoe backup works fine.
>>          If you
>>        have more than can reasonably be uploaded in a few days, you can
>>        still
>>        use tahoe backup, but it's trickier.
>>
>>        On Tue, Feb 7, 2012 at 11:52 AM, Brad Rupp <bradrupp at gmail.com
>>        <mailto:bradrupp at gmail.com>
>>        <mailto:bradrupp at gmail.com <mailto:bradrupp at gmail.com>>> wrote:
>>
>>            I am curious how everyone is backing up their data to the
>>        grid.  I
>>            assume most people are using tahoe backup, but I'm wondering if
>>            there are other solutions that have been hacked together.
>>
>>            As for me, I'm just getting started on this.  My server
>>        going up in
>>            smoke last month has caused a sense of urgency regarding
>>        backups.
>>              Of course, that is why I am still working on this a month
>>        later... :)
>>
>>            My preferred backup method would be to push a weekly
>>        snapshot to the
>>            grid.  I can live with losing a weeks worth of data and I
>>        don't need
>>            versioned backups.  Basically the last weekly snapshot is
>>        all that I
>>            need.  Now to make it happen...
>>
>>            Regards,
>>
>>            Brad
>>
>>
>>            ______________________________**_____________________
>>            volunteergrid2-l mailing list
>>            volunteergrid2-l at tahoe-lafs.__**__org
>>        <mailto:volunteergrid2-l at __tah**oe-lafs.org<http://tahoe-lafs.org>
>>        <mailto:volunteergrid2-l@**tahoe-lafs.org<volunteergrid2-l at tahoe-lafs.org>
>> >>
>>        http://tahoe-lafs.org/cgi-bin/**____mailman/listinfo/____**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/____mailman/listinfo/____volunteergrid2-l>
>>        <http://tahoe-lafs.org/cgi-**bin/__mailman/listinfo/__**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/__mailman/listinfo/__volunteergrid2-l>
>> >
>>        <http://tahoe-lafs.org/cgi-__**bin/mailman/listinfo/__**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-__bin/mailman/listinfo/__volunteergrid2-l>
>>        <http://tahoe-lafs.org/cgi-**bin/mailman/listinfo/**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l>
>> >>
>>        http://bigpig.org/twiki/bin/__**__view/Main/WebHome<http://bigpig.org/twiki/bin/____view/Main/WebHome>
>>        <http://bigpig.org/twiki/bin/_**_view/Main/WebHome<http://bigpig.org/twiki/bin/__view/Main/WebHome>
>> >
>>        <http://bigpig.org/twiki/bin/_**_view/Main/WebHome<http://bigpig.org/twiki/bin/__view/Main/WebHome>
>>        <http://bigpig.org/twiki/bin/**view/Main/WebHome<http://bigpig.org/twiki/bin/view/Main/WebHome>
>> >>
>>
>>
>>
>>
>>        --
>>        Shawn
>>
>>
>>
>>        ______________________________**___________________
>>        volunteergrid2-l mailing list
>>        volunteergrid2-l at tahoe-lafs.__**org
>>        <mailto:volunteergrid2-l@**tahoe-lafs.org<volunteergrid2-l at tahoe-lafs.org>
>> >
>>        http://tahoe-lafs.org/cgi-bin/**__mailman/listinfo/__**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/__mailman/listinfo/__volunteergrid2-l>
>>        <http://tahoe-lafs.org/cgi-**bin/mailman/listinfo/**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l>
>> >
>>        http://bigpig.org/twiki/bin/__**view/Main/WebHome<http://bigpig.org/twiki/bin/__view/Main/WebHome>
>>        <http://bigpig.org/twiki/bin/**view/Main/WebHome<http://bigpig.org/twiki/bin/view/Main/WebHome>
>> >
>>
>>    ______________________________**___________________
>>    volunteergrid2-l mailing list
>>    volunteergrid2-l at tahoe-lafs.__**org
>>    <mailto:volunteergrid2-l@**tahoe-lafs.org<volunteergrid2-l at tahoe-lafs.org>
>> >
>>    http://tahoe-lafs.org/cgi-bin/**__mailman/listinfo/__**
>> volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/__mailman/listinfo/__volunteergrid2-l>
>>    <http://tahoe-lafs.org/cgi-**bin/mailman/listinfo/**volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l>
>> >
>>    http://bigpig.org/twiki/bin/__**view/Main/WebHome<http://bigpig.org/twiki/bin/__view/Main/WebHome>
>>    <http://bigpig.org/twiki/bin/**view/Main/WebHome<http://bigpig.org/twiki/bin/view/Main/WebHome>
>> >
>>
>>
>>
>>
>> --
>> Shawn
>>
>>
>> ______________________________**_________________
>> volunteergrid2-l mailing list
>> volunteergrid2-l at tahoe-lafs.**org <volunteergrid2-l at tahoe-lafs.org>
>> http://tahoe-lafs.org/cgi-bin/**mailman/listinfo/**volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l>
>> http://bigpig.org/twiki/bin/**view/Main/WebHome<http://bigpig.org/twiki/bin/view/Main/WebHome>
>>
> ______________________________**_________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.**org <volunteergrid2-l at tahoe-lafs.org>
> http://tahoe-lafs.org/cgi-bin/**mailman/listinfo/**volunteergrid2-l<http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l>
> http://bigpig.org/twiki/bin/**view/Main/WebHome<http://bigpig.org/twiki/bin/view/Main/WebHome>
>

-- 
Shawn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/cgi-bin/mailman/private/volunteergrid2-l/attachments/20120208/006a4b7b/attachment.html>