LiteralFileURI instance has no attribute 'storage_index'

[francois]

A daily cronjob is running 'tahoe deep-check --repair' on files which were created by 'tahoe backup'. It began to fail with the following exception sometimes around the implementation of immutable directories.

As soon as this failure occurs, the deep-check operation is aborted. This might be dangerous because the remaining files are not being correctly checked.

$ tahoe deep-check -v --repair --add-lease tahoe:
[...]
ERROR: AttributeError(LiteralFileURI instance has no attribute 'storage_index')
[Failure instance: Traceback: <type 'exceptions.AttributeError'>: LiteralFileURI instance has no attribute 'storage_index'
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:285:unpause
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:328:_runCallbacks
/home/francois/dev/tahoe/src/allmydata/dirnode.py:676:<lambda>
/home/francois/dev/tahoe/src/allmydata/dirnode.py:631:_deep_traverse_dirnode
--- <exception caught here> ---
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:106:maybeDeferred
/home/francois/dev/tahoe/src/allmydata/web/directory.py:1114:add_node
/home/francois/dev/tahoe/src/allmydata/dirnode.py:349:get_storage_index

$ tahoe --version
allmydata-tahoe: 1.5.0-r4228, foolscap: 0.4.2, pycryptopp: 0.5.15, zfec: 1.4.5, Twisted: 8.1.0, Nevow: 0.9.33-r17222, zope.interface: 3.3.1, python: 2.5.2, platform: Linux-debian_5.0.4-i686-32bit, sqlite: 3.5.9, simplejson: 1.9.2, argparse: 0.9.1, pyOpenSSL: 0.7, pyutil: 1.3.34, zbase32: 1.1.1, setuptools: 0.6c12dev, pysqlite: 2.3.2

[davidsarah]

get_storage_index on DirectoryNode​ is implemented like this:

    def get_storage_index(self):
        return self._uri._filenode_uri.storage_index

which makes unwarranted assumptions about attributes of URI objects. It should be

    def get_storage_index(self):
        return self._uri.get_storage_index()

(This delegates to get_storage_index on _DirectoryBaseURI​, which delegates to get_storage_index on LiteralFileURI​, which correctly returns None. The bug probably occurred as a result of trying to skip one of these delegation steps, violating abstraction.)

Please check whether this is sufficient to fix the problem -- I haven't checked whether tahoe check handles a storage index of None correctly, although it should if it handles LIT files.

[davidsarah]

(I meant tahoe deep-check, not tahoe check.)

This is a regression relative to 1.5.0, since tahoe backup didn't use immutable directories then, and only immutable directories can wrap a LiteralFileURI.

[davidsarah]

Looking at where get_storage_index, it seems that several other operations on empty immutable directories that are small enough to fit in a LIT file are likely to fail. Viewing them in the WUI certainly provokes the same error:

• ​http://testgrid.allmydata.org:3567/uri/URI:DIR2-LIT:ge3dumj2mewdcotyfqydulbshj5x2lbm/
• ​http://testgrid.allmydata.org:3567/uri/URI:DIR2-LIT:/

We should probably put out a point release (although there's time to fix other stuff and perhaps include #778.)

[warner]

In general, a storage_index of None is used to indicate a "non-distributed file", which the Checker ought to skip entirely (non-distributed files are always healthy). I'd guess that we used to have about 90% support for these (i.e. of the code that I wrote, I probably made it tolerate LITs about 90% of the time, and/or we have test coverage of this for about 90% of the code paths). But it sounds like something changed recently.

[davidsarah]

As far as I can see, we have no tests for DIR2-LIT URIs, which is why this escaped notice in the 1.6.0 release cycle.

[zooko]

I'm writing tests.

[davidsarah]

Zooko is writing tests for test_dirnode.py. I'll write them for test_web.py, and write the actual fix. Note that there are other cases -- lots in test code and a few in non-test code -- where we directly access .storage_index on an URI object rather than calling .get_storage_index().

[davidsarah]

Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948)

[davidsarah]

Additions to test_web.py for #948

[davidsarah]

Additional fix for abbrev_si, with test (includes previous two patches)

[zooko]

add tests of literal dirnodes (the current code already passes these tests) This patch replaces the previous patch by the same name, which was buggy.

[zooko]

We should add a test of deepcheck (where this issue was first detected). I think all we need to do is add a DIR2:LIT to DeepCheckWebGood.set_up_tree().

[davidsarah]

DeepCheckWebGood?.set_up_tree: agreed. I also want to check the WUI output for a DIR2:LIT, which is in test_web.py.

[davidsarah]

dirnode: add tests of literal dirnodes (current and fix for #948)

[davidsarah]

Nevow traceback showing bug when following More Info link of a DIR2-LIT directory (​http://127.0.0.1:3456/uri/URI%3ADIR2-LIT%3Agqytunj2onug64tufqzdcosvkjetutcjkq5gw4tvm5vwszdgnz5hgyzufqydulbshj5x2lbm/?t=info)

[zooko]

add tests of literal dirnodes to test_deepcheck.py (the current trunk fails these tests and so does the current all-948-darcspatch.txt! Of course I'm not sure if it is the tests or the code that is wrong)

[davidsarah]

directories: add DIR2-LIT directories to test_deepcheck.py (#948) -- fixes test bugs

[davidsarah]

Additional fixes for DIR2-LIT More Info page and deep-check/manifest operations (#948)

[davidsarah]

The patch bundles that need review are:

• all-948-darcspatch.txt
• test-dirnode-lit-2-darcspatch.txt
• test-dirnode-lit-in-deep-check-2-darcspatch.txt
• additional-fixes-for-948-darcspatch.txt

This does not include the test of WUI output, which is currently failing (the functionality seems to work, so I think it's a test problem -- will investigate tomorrow).

[davidsarah]

Updates to NEWS relevant to this ticket and #577

[zooko]

I think we need to add to some of these tests -- at least to test_deepcheck.py -- a large immutable directory. Large enough so that it isn't a LIT.

[davidsarah]

Replying to zooko:

I think we need to add to some of these tests -- at least to test_deepcheck.py -- a large immutable directory. Large enough so that it isn't a LIT.

I agree that would be useful, but strictly speaking this ticket is about DIR2-LITs, so let's get these patches reviewed first.

[davidsarah]

Additional test for DIR2-LIT directories in test_web.py, fixed version (#948). Corrects a mistake in the use of Deferreds.

[zooko]

Okay I'm reviewing

• test-dirnode-lit-in-wui-darcspatch.txt
• updates-to-NEWS-darcspatch.txt
• all-948-darcspatch.txt
• test-dirnode-lit-2-darcspatch.txt
• test-dirnode-lit-in-deep-check-2-darcspatch.txt
• additional-fixes-for-948-darcspatch.txt

Is that the right set to review?

[zooko]

I reviewed test-dirnode-lit-in-wui-darcspatch.txt​ and it looks good.

[zooko]

I reviewed test-dirnode-lit-in-deep-check-2-darcspatch.txt​ and it is good.

[zooko]

Replying to zooko:

I reviewed test-dirnode-lit-in-deep-check-2-darcspatch.txt​ and it is good.

Whoops, sorry I meant updates-to-NEWS-darcspatch.txt​ not test-dirnode-lit-in-deep-check-2-darcspatch.txt​.

[zooko]

Okay, now I've reviewed test-dirnode-lit-in-deep-check-2-darcspatch.txt​ and approved it. Technically, it would be better if someone else reviewed test-dirnode-lit-in-deep-check-2-darcspatch.txt​ since I'm part author of it, but Brian's not around on IRC right now and I'm hoping to announce v1.6.1 tomorrow! :-) If Kevan or anyone else is reading this and wants to read over test-dirnode-lit-in-deep-check-2-darcspatch.txt​ and look for goofs, that would be great. It will probably take only a few minutes, especially if you don't mind being uncertain about exactly what all the tests do and just look for things that look suspiciously like mistakes to you. If you develop a confident understanding of what the tests actually do here, all the better!

[zooko]

Okay, I've reviewed test-dirnode-lit-2-darcspatch.txt​ and I like it. Again, I'm not the perfect reviewer since I'm the author of most of this one, so if you are reading this and you can double-check that patch, that would be great.

[zooko]

Okay, I've reviewed all-948-darcspatch.txt​ and could find nothing in it to change!

[zooko]

Okay I'm looking at additional-fixes-for-948-darcspatch.txt​, which is the last patch from the set of six patches that constitute the fix for this ticket (per comment:15). For this one I have some questions:

• Is this correct:

  +            self.root_storage_index_s = "<none>"  # is this correct?
  

(Elsewhere we use either "" or "<LIT>" to be the string encoding of a null storage index.)

• Should we add tests of the changes in this patch, mainly the changes to emit "" when a storage index is None, or there already unit tests that fail when those changes are not applied?
• The construction base32.b2a(m.origin_si or "") feels slightly weird to me because it feels like a type error—we really mean that the expression should evaluate to "" not that the argument to b2a() should be "". However, the alternative is bigger and uglier: m.origin_si and base32.b2a(m.origin_si) or "". Maybe we should define a function:

  def b2a_or_empty(si):
      if si:
          return base32.b2a(si)
      else:
          return ""
  

[davidsarah]

All patches to be applied for #948

[davidsarah]

all-darcspatch.txt includes a change to avoid base32.b2a(... or ""), although not using a separate function. This change has been reviewed by bigpig.

The changes to emit "" when the storage index is None are tested by test_deepcheck.py. We decided not to change "<none>" for 1.6.1.

[davidsarah]

Committed as 973f0afdd32285a7, d29ec184a6e67623, fec9185f2a866523, 187d837c1db532f6, e6aee33bb77f6f88, d7b50a3b86acf8c6, f8e2bab41ea6c38d, a2ed17f2a0d69ec3, 40edf8f419947b80.