Ticket #1118: fix-1118.dpatch

File fix-1118.dpatch, 12.6 KB (added by davidsarah, at 2010-07-19T04:26:55Z)
  • upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct.
Line 
1Mon Jul 19 05:08:52 GMT Daylight Time 2010  david-sarah@jacaranda.org
2  * upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct.
3
4New patches:
5
6[upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct.
7david-sarah@jacaranda.org**20100719040852
8 Ignore-this: 2d49faa35b37372643f0af8e5e6a21eb
9] {
10hunk ./src/allmydata/immutable/upload.py 138
11         return (alreadygot, set(b.keys()))
12 
13 
14-    def abort(self):
15+    def abort(self, sharenums=None):
16         """
17         I abort the remote bucket writers for the share numbers in
18         sharenums. This is a good idea to conserve space on the storage
19hunk ./src/allmydata/immutable/upload.py 144
20         server.
21         """
22-        for writer in self.buckets.itervalues(): writer.abort()
23+        if not sharenums:
24+            sharenums = self.buckets.keys()
25+
26+        for sharenum in sharenums:
27+            if sharenum in self.buckets:
28+                self.buckets[sharenum].abort()
29+                del self.buckets[sharenum]
30 
31 
32 class Tahoe2PeerSelector:
33hunk ./src/allmydata/immutable/upload.py 365
34                             if not self.preexisting_shares[share]:
35                                 del self.preexisting_shares[share]
36                             items.append((server, sharelist))
37+                        for writer in self.use_peers:
38+                            writer.abort(self.homeless_shares)
39                     return self._loop()
40                 else:
41                     # Redistribution won't help us; fail.
42hunk ./src/allmydata/immutable/upload.py 375
43                                           self.needed_shares,
44                                           self.servers_of_happiness,
45                                           effective_happiness)
46+                    log.msg("server selection unsuccessful for %r: %s (%s), merged=%r"
47+                            % (self, msg, self._get_progress_message(), merged), level=log.INFREQUENT)
48                     return self._failed("%s (%s)" % (msg, self._get_progress_message()))
49 
50         if self.uncontacted_peers:
51}
52
53Context:
54
55[trivial: fix unused import (sorry about that, pyflakes)
56zooko@zooko.com**20100718215133
57 Ignore-this: c2414e443405072b51d552295f2c0e8c
58] 
59[docs/logging.txt: document that _trial_temp/test.log does not receive messages below level=OPERATIONAL, due to <http://foolscap.lothar.com/trac/ticket/154>.
60david-sarah@jacaranda.org**20100718230420
61 Ignore-this: aef40f2e74ddeabee5e122e8d80893a1
62] 
63[tests, NEWS, CREDITS re: #1117
64zooko@zooko.com**20100718203225
65 Ignore-this: 1f08be2c692fb72cc0dd023259f11354
66 Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117.
67 fixes #1117
68] 
69[test/test_upload.py: test to see that aborted buckets are ignored by the storage server
70Kevan Carstensen <kevan@isnotajoke.com>**20100716001046
71 Ignore-this: cc075c24b1c86d737f3199af894cc780
72] 
73[test/test_storage.py: test for the new remote_abort semantics.
74Kevan Carstensen <kevan@isnotajoke.com>**20100715232148
75 Ignore-this: d3d6491f17bf670e770ca4b385007515
76] 
77[storage/immutable.py: make remote_abort btell the storage server about aborted buckets.
78Kevan Carstensen <kevan@isnotajoke.com>**20100715232105
79 Ignore-this: 16ab0090676355abdd5600ed44ff19c9
80] 
81[test/test_upload.py: changes to test plumbing for #1117 tests
82Kevan Carstensen <kevan@isnotajoke.com>**20100715231820
83 Ignore-this: 78a6d359d7bf8529d283e2815bf1e2de
84 
85     - Add a callRemoteOnly method to FakeBucketWriter.
86     - Change the abort method in FakeBucketWriter to not return a
87       RuntimeError.
88] 
89[immutable/upload.py: abort buckets if peer selection fails
90Kevan Carstensen <kevan@isnotajoke.com>**20100715231714
91 Ignore-this: 2a0b643a22284df292d8ed9d91b1fd37
92] 
93[test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable.
94david-sarah@jacaranda.org**20100718151420
95 Ignore-this: af050955f623fbc0e4d78e15a0a8a144
96] 
97[NEWS: Forward-compatibility improvements for non-ASCII caps (#1051).
98david-sarah@jacaranda.org**20100718143622
99 Ignore-this: 1edfebc4bd38a3b5c35e75c99588153f
100] 
101[test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON.
102david-sarah@jacaranda.org**20100718142915
103 Ignore-this: c4e78ef4b1478dd400da71cf077ffa4a
104] 
105[docs: add comment clarifying #1051
106zooko@zooko.com**20100718053250
107 Ignore-this: 6cfc0930434cbdbbc262dabb58f1505d
108] 
109[docs: update NEWS
110zooko@zooko.com**20100718053225
111 Ignore-this: 63d5c782ef84812e6d010f0590866831
112] 
113[Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected.
114david-sarah@jacaranda.org**20100711200252
115 Ignore-this: c2f193352369d32e06865f8f3e951894
116] 
117[Debian documentation update
118jacob@appelbaum.net**20100305003004] 
119[debian-docs-patch-final
120jacob@appelbaum.net**20100304085955] 
121[M-x whitespace-cleanup
122zooko@zooko.com**20100718032739
123 Ignore-this: babfd4af6ad2fc885c957fd5c8b10c3f
124] 
125[docs: tidy up NEWS a little
126zooko@zooko.com**20100718032434
127 Ignore-this: 54f2820fd1a37c8967609f6bfc4e5e18
128] 
129[benchmarking: update bench_dirnode.py to reflect the new directory interfaces
130zooko@zooko.com**20100718031710
131 Ignore-this: 368ba523dd3de80d9da29cd58afbe827
132] 
133[test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock.
134david-sarah@jacaranda.org**20100718125412
135 Ignore-this: 4bf373a5e2dfe4209e5e364124af29a3
136] 
137[iputil.py: Add support for FreeBSD 7,8 and 9
138francois@ctrlaltdel.ch**20100718022832
139 Ignore-this: 1829b4cf4b91107f4cf87841e6167e99
140 committed by: zooko@zooko.com
141 date: 2010-07-17
142 and I also patched: NEWS and CREDITS
143] 
144[NEWS: add snippet about #1083
145zooko@zooko.com**20100718020653
146 Ignore-this: d353a9d93cbc5a5e6ba4671f78d1e22b
147] 
148[fileutil: docstrings for non-obvious usage restrictions on methods of EncryptedTemporaryFile.
149david-sarah@jacaranda.org**20100717054647
150 Ignore-this: 46d8fc10782fa8ec2b6c5b168c841943
151] 
152[Move EncryptedTemporaryFile from SFTP frontend to allmydata.util.fileutil, and make the FTP frontend also use it (fixing #1083).
153david-sarah@jacaranda.org**20100711213721
154 Ignore-this: e452e8ca66391aa2a1a49afe0114f317
155] 
156[NEWS: reorder NEWS snippets to be in descending order of interestingness
157zooko@zooko.com**20100718015929
158 Ignore-this: 146c42e88a9555a868a04a69dd0e5326
159] 
160[test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename.
161david-sarah@jacaranda.org**20100718030333
162 Ignore-this: c114d92c17714a5d4ae005c15267d60c
163] 
164[Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error.
165david-sarah@jacaranda.org**20100718013435
166 Ignore-this: c8940c4e1aa2e9acc80cd4fe54753cd8
167] 
168[test_cli.py: fix error that crept in when rebasing the patch for #1072.
169david-sarah@jacaranda.org**20100718000123
170 Ignore-this: 3e8f6cc3a27b747c708221dd581934f4
171] 
172[stringutils: add test for when sys.stdout has no encoding attribute (fixes #1099).
173david-sarah@jacaranda.org**20100717045816
174 Ignore-this: f28dce6940e909f12f354086d17db54f
175] 
176[CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility.
177david-sarah@jacaranda.org**20100717220411
178 Ignore-this: 3ecdde7f2d0498514cef32e118e0b855
179] 
180[minor code clean-up in dirnode.py
181zooko@zooko.com**20100714060255
182 Ignore-this: bb0ab2783203e605024b3e2f798256a1
183 Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring.
184 fixes #967
185] 
186[Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil.
187david-sarah@jacaranda.org**20100712003015
188 Ignore-this: 103b809d180df17a7283077c3104c7be
189] 
190[Allow URIs passed in the initial JSON for t=mkdir-with-children, t=mkdir-immutable to be Unicode. Also pass the name of each child into nodemaker.create_from_cap for error reporting.
191david-sarah@jacaranda.org**20100711195525
192 Ignore-this: deac32d8b91ba26ede18905d3f7d2b93
193] 
194[docs: CREDITS and NEWS
195zooko@zooko.com**20100714060150
196 Ignore-this: dc83e612f77d69e50ee975f07f6b16fe
197] 
198[CREDITS: more creds for Kevan, plus utf-8 BOM
199zooko@zooko.com**20100619045503
200 Ignore-this: 72d02bdd7a0f324f1cee8cd399c7c6de
201] 
202[cli.py: make command descriptions consistently end with a full stop.
203david-sarah@jacaranda.org**20100714014538
204 Ignore-this: 9ee7fa29ca2d1631db4049c2a389a97a
205] 
206[SFTP: address some of the comments in zooko's review (#1106).
207david-sarah@jacaranda.org**20100712025537
208 Ignore-this: c3921638a2d4f1de2a776ae78e4dc37e
209] 
210[docs/logging.txt: note that setting flogging vars might affect tests with race conditions.
211david-sarah@jacaranda.org**20100712050721
212 Ignore-this: fc1609d215fcd5561a57fd1226206f27
213] 
214[test_storage.py: potential fix for failures when logging is enabled.
215david-sarah@jacaranda.org**19700713040546
216 Ignore-this: 5815693a0df3e64c52c3c6b7be2846c7
217] 
218[upcase_since_on_welcome
219terrellrussell@gmail.com**20100708193903] 
220[server_version_on_welcome_page.dpatch.txt
221freestorm77@gmail.com**20100605191721
222 Ignore-this: b450c76dc875f5ac8cca229a666cbd0a
223 
224 
225 - The storage server version is 0 for all storage nodes in the Welcome Page
226 
227 
228] 
229[NEWS: add NEWS snippets about two recent patches
230zooko@zooko.com**20100708162058
231 Ignore-this: 6c9da6a0ad7351a960bdd60f81532899
232] 
233[directory_html_top_banner.dpatch
234freestorm77@gmail.com**20100622205301
235 Ignore-this: 1d770d975e0c414c996564774f049bca
236 
237 The div tag with the link "Return to Welcome page" on the directory.xhtml page is not correct
238 
239] 
240[tahoe_css_toolbar.dpatch
241freestorm77@gmail.com**20100622210046
242 Ignore-this: 5b3ebb2e0f52bbba718a932f80c246c0
243 
244 CSS modification to be correctly diplayed with Internet Explorer 8
245 
246 The links on the top of page directory.xhtml are not diplayed in the same line as display with Firefox.
247 
248] 
249[runnin_test_tahoe_css.dpatch
250freestorm77@gmail.com**20100622214714
251 Ignore-this: e0db73d68740aad09a7b9ae60a08c05c
252 
253 Runnin test for changes in tahoe.css file
254 
255] 
256[runnin_test_directory_xhtml.dpatch
257freestorm77@gmail.com**20100622201403
258 Ignore-this: f8962463fce50b9466405cb59fe11d43
259 
260 Runnin test for diretory.xhtml top banner
261 
262] 
263[stringutils.py: tolerate sys.stdout having no 'encoding' attribute.
264david-sarah@jacaranda.org**20100626040817
265 Ignore-this: f42cad81cef645ee38ac1df4660cc850
266] 
267[quickstart.html: python 2.5 -> 2.6 as recommended version
268david-sarah@jacaranda.org**20100705175858
269 Ignore-this: bc3a14645ea1d5435002966ae903199f
270] 
271[SFTP: don't call .stopProducing on the producer registered with OverwriteableFileConsumer (which breaks with warner's new downloader).
272david-sarah@jacaranda.org**20100628231926
273 Ignore-this: 131b7a5787bc85a9a356b5740d9d996f
274] 
275[docs/how_to_make_a_tahoe-lafs_release.txt: trivial correction, install.html should now be quickstart.html.
276david-sarah@jacaranda.org**20100625223929
277 Ignore-this: 99a5459cac51bd867cc11ad06927ff30
278] 
279[setup: in the Makefile, refuse to upload tarballs unless someone has passed the environment variable "BB_BRANCH" with value "trunk"
280zooko@zooko.com**20100619034928
281 Ignore-this: 276ddf9b6ad7ec79e27474862e0f7d6
282] 
283[trivial: tiny update to in-line comment
284zooko@zooko.com**20100614045715
285 Ignore-this: 10851b0ed2abfed542c97749e5d280bc
286 (I'm actually committing this patch as a test of the new eager-annotation-computation of trac-darcs.)
287] 
288[docs: about.html link to home page early on, and be decentralized storage instead of cloud storage this time around
289zooko@zooko.com**20100619065318
290 Ignore-this: dc6db03f696e5b6d2848699e754d8053
291] 
292[docs: update about.html, especially to have a non-broken link to quickstart.html, and also to comment out the broken links to "for Paranoids" and "for Corporates"
293zooko@zooko.com**20100619065124
294 Ignore-this: e292c7f51c337a84ebfeb366fbd24d6c
295] 
296[TAG allmydata-tahoe-1.7.0
297zooko@zooko.com**20100619052631
298 Ignore-this: d21e27afe6d85e2e3ba6a3292ba2be1
299] 
300Patch bundle hash:
301df848c673c94a407759e90d3c9029a6bf824354e