[tahoe-lafs-trac-stream] [tahoe-lafs] #1833: storage server deletes garbage shares itself instead of waiting for crawler to notice them

[tahoe-lafs]

#1833: storage server deletes garbage shares itself instead of waiting for crawler
to notice them
-------------------------+-------------------------------------------------
     Reporter:  zooko    |      Owner:
         Type:           |     Status:  new
  enhancement            |  Milestone:  undecided
     Priority:  normal   |    Version:  1.9.2
    Component:  code-    |   Keywords:  leases garbage-collection
  storage                |  accounting
   Resolution:           |
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Old description:

> Currently, the "lease crawler" or "accounting crawler" is responsible for
> deleting shares that have lost all their leases (by cancellation or
> expiry).
>
> I propose that this be done instead by the storage server maintaining a
> persistent set of shares to be deleted. When lease-updating step (which,
> in #666, is synchronous and fast) has identified a share that has no more
> leases, the share's id gets added to the persistent set of shares to
> delete. A long-running, persistent, duty-cycle-limited processes deletes
> those shares from the backend and removes their ids from the set of
> shares-to-delete. This is cleaner and more efficient than using a
> crawler, which has to visit ''all'' shares and which never stops
> twitching, since this has to visit only shares that have been marked as
> to-delete, and it quiesces when there is nothing to delete.
>
> This is part of an "overarching ticket" to eliminate most uses of crawler
> — ticket #1834.

New description:

 Currently, the "lease crawler" or "accounting crawler" is responsible for
 deleting shares that have lost all their leases (by cancellation or
 expiry).

 I propose that this be done instead by the storage server maintaining a
 persistent set of shares to be deleted. When lease-updating step (which,
 in #666, is synchronous and fast) has identified a share that has no more
 leases, the share's id gets added to the persistent set of shares to
 delete. A long-running, persistent, duty-cycle-limited processes deletes
 those shares from the backend and removes their ids from the set of
 shares-to-delete. This is cleaner and more efficient than using a crawler,
 which has to visit ''all'' shares and which never stops twitching, since
 this has to visit only shares that have been marked as to-delete, and it
 quiesces when there is nothing to delete.

 This is part of an "overarching ticket" to eliminate most uses of crawler
 — ticket #1834.

--

Comment (by zooko):

 This would fix #1987.

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1833#comment:4>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage