[tahoe-dev] [tahoe-lafs] #1354: compression (e.g. to efficiently store sparse files)

[tahoe-lafs]

#1354: compression (e.g. to efficiently store sparse files)
---------------------------+------------------------------------------------
 Reporter:  zooko          |           Owner:           
     Type:  enhancement    |          Status:  new      
 Priority:  major          |       Milestone:  undecided
Component:  code-encoding  |         Version:  1.8.2    
 Keywords:                 |   Launchpad Bug:           
---------------------------+------------------------------------------------
 I was just re-reading
 [http://doc.cat-v.org/plan_9/4th_edition/papers/fossil/ the Fossil/Venti
 tech docs] and realized that their method of omitting trailing zeroes from
 each block might be adapted to Tahoe-LAFS to efficiently store sparse
 files. (Tahoe-LAFS would do this per Tahoe-LAFS "segment" where Venti did
 it per Venti "block".)

 This is of course a very specific form of compression and might be
 subsumed by more general compression instead.

 On the other hand compression -- either the specific "remove runs of
 zeroes" compression from Venti or more general compression -- is often a
 lose at this layer since the biggest files are almost always
 uncompressable (because they are already compressed).

 Are sparse files common?

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1354>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage