Ticket #999: storagemocktest01.darcs.patch

Mon Mar 21 23:28:10 MDT 2011  wilcoxjg@gmail.com
  * storage: add test of read share data
  There are already tests in test_download.py which exercise the whole stack including the client and the real operating system's filesystem access functions; this test mocks out the filesystem access functions and invokes the StorageServer interface.

New patches:

[storage: add test of read share data
wilcoxjg@gmail.com**20110322052810
 Ignore-this: 19e9fc3864a65dd4f1eb4460f3be4344
 There are already tests in test_download.py which exercise the whole stack including the client and the real operating system's filesystem access functions; this test mocks out the filesystem access functions and invokes the StorageServer interface.
] {
addfile ./src/allmydata/test/test_server.py
hunk ./src/allmydata/test/test_server.py 1
+from twisted.trial import unittest
+
+from StringIO import StringIO
+
+from allmydata.test.common_util import ReallyEqualMixin
+
+import mock
+
+# This is the code that we're going to be testing.
+from allmydata.storage.server import StorageServer 
+
+class TestServer(unittest.TestCase, ReallyEqualMixin):
+    @mock.patch('os.path.getsize')
+    @mock.patch('__builtin__.open')
+    @mock.patch('os.listdir')
+    def test_reads_old_share_from_disk(self, mocklistdir, mockopen, mockgetsize):
+        """ This tests whether the code correctly finds and reads
+        shares written out by old (Tahoe-LAFS <= v1.8.2)
+        servers. There is a similar test in test_download, but that one
+        is from the perspective of the client and exercises a deeper
+        stack of code. This one is for exercising just the
+        StorageServer object. """
+
+        mocklistdir.return_value = ['0']
+
+        # The following share file contents was generated with
+        # storage.immutable.ShareFile from Tahoe-LAFS v1.8.2
+        # with share data == 'a'.
+        share_data = 'a\x00\x00\x00\x00xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\x00\x00\x00\x07'
+        share_file_data = '\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01' + share_data
+
+        sharefname = 'testdir/shares/or/orsxg5dtorxxeylhmvpws3temv4a/0'
+        def call_open(fname, mode):
+            if fname == 'testdir/bucket_counter.state':
+                raise IOError(2, "No such file or directory: 'testdir/bucket_counter.state'")
+            elif fname == 'testdir/lease_checker.state':
+                raise IOError(2, "No such file or directory: 'testdir/lease_checker.state'")
+            elif fname == 'testdir/lease_checker.history':
+                return StringIO()
+            else:
+                self.failUnlessReallyEqual(fname, sharefname)
+                self.failUnless('r' in mode, mode)
+                self.failUnless('b' in mode, mode)
+
+                return StringIO(share_file_data)
+        mockopen.side_effect = call_open
+
+        datalen = len(share_file_data)
+        def call_getsize(fname):
+            self.failUnlessReallyEqual(fname, sharefname)
+            return datalen
+        mockgetsize.side_effect = call_getsize
+
+        
+        # Now begin the test.
+        s = StorageServer('testdir', 'testnodeidxxxxxxxxxx')
+
+        bs = s.remote_get_buckets('teststorage_index')
+
+        self.failUnlessEqual(len(bs), 1)
+        b = bs[0]
+        self.failUnlessReallyEqual(b.remote_read(0, datalen), share_data)
+        # If you try to read past the end you get the as much data as is there.
+        self.failUnlessReallyEqual(b.remote_read(0, datalen+20), share_data)
+        # If you start reading past the end of the file you get the empty string.
+        self.failUnlessReallyEqual(b.remote_read(datalen+1, 3), '')
}

Context:

[test: increase timeout on a network test because Francois's ARM machine hit that timeout
zooko@zooko.com**20110317165909
 Ignore-this: 380c345cdcbd196268ca5b65664ac85b
 I'm skeptical that the test was proceeding correctly but ran out of time. It seems more likely that it had gotten hung. But if we raise the timeout to an even more extravagant number then we can be even more certain that the test was never going to finish.
] 
[docs/configuration.rst: add a "Frontend Configuration" section
Brian Warner <warner@lothar.com>**20110222014323
 Ignore-this: 657018aa501fe4f0efef9851628444ca
 
 this points to docs/frontends/*.rst, which were previously underlinked
] 
[web/filenode.py: avoid calling req.finish() on closed HTTP connections. Closes #1366
"Brian Warner <warner@lothar.com>"**20110221061544
 Ignore-this: 799d4de19933f2309b3c0c19a63bb888
] 
[Add unit tests for cross_check_pkg_resources_versus_import, and a regression test for ref #1355. This requires a little refactoring to make it testable.
david-sarah@jacaranda.org**20110221015817
 Ignore-this: 51d181698f8c20d3aca58b057e9c475a
] 
[allmydata/__init__.py: .name was used in place of the correct .__name__ when printing an exception. Also, robustify string formatting by using %r instead of %s in some places. fixes #1355.
david-sarah@jacaranda.org**20110221020125
 Ignore-this: b0744ed58f161bf188e037bad077fc48
] 
[Refactor StorageFarmBroker handling of servers
Brian Warner <warner@lothar.com>**20110221015804
 Ignore-this: 842144ed92f5717699b8f580eab32a51
 
 Pass around IServer instance instead of (peerid, rref) tuple. Replace
 "descriptor" with "server". Other replacements:
 
  get_all_servers -> get_connected_servers/get_known_servers
  get_servers_for_index -> get_servers_for_psi (now returns IServers)
 
 This change still needs to be pushed further down: lots of code is now
 getting the IServer and then distributing (peerid, rref) internally.
 Instead, it ought to distribute the IServer internally and delay
 extracting a serverid or rref until the last moment.
 
 no_network.py was updated to retain parallelism.
] 
[TAG allmydata-tahoe-1.8.2
warner@lothar.com**20110131020101] 
Patch bundle hash:
01bcb40d26aa13847d26001cf0550614e9b22a47