Ticket #567: add_tahoe_version_to_json.darcs.txt

File add_tahoe_version_to_json.darcs.txt, 20.0 KB (added by freestorm, at 2010-06-04T22:47:55Z)
Line 
1Thu Jun  3 22:01:34 CEST 2010  freestorm77@gmail.com
2  * add_tahoe_version_to_json.darcs.txt
3 
4  - Add "tahoe_version" field to the JSON output from operations deep-stats, deep-manifest and deep-check.
5  - See ticket #567 for more information
6 
7
8New patches:
9
10[add_tahoe_version_to_json.darcs.txt
11freestorm77@gmail.com**20100603200134
12 Ignore-this: 9997f4ff326a7ea9272c9dec6be52e22
13 
14 - Add "tahoe_version" field to the JSON output from operations deep-stats, deep-manifest and deep-check.
15 - See ticket #567 for more information
16 
17] {
18hunk ./src/allmydata/dirnode.py 25
19 from allmydata.uri import LiteralFileURI, from_string, wrap_dirnode_cap
20 from pycryptopp.cipher.aes import AES
21 from allmydata.util.dictutil import AuxValueDict
22+from allmydata._version import __version__
23 
24 
25 # TODO: {Deleter,MetadataSetter,Adder}.modify all start by unpacking the
26hunk ./src/allmydata/dirnode.py 724
27         self.histograms = {}
28         for k in ["size-files-histogram"]:
29             self.histograms[k] = {} # maps (min,max) to count
30+        self.stats["tahoe_version"] = str(__version__)
31         self.buckets = [ (0,0), (1,3)]
32         self.root = math.sqrt(10)
33 
34hunk ./src/allmydata/web/check_results.py 10
35 from allmydata.web.operations import ReloadMixin
36 from allmydata.interfaces import ICheckAndRepairResults, ICheckResults
37 from allmydata.util import base32, idlib
38+from allmydata._version import __version__
39 
40 def json_check_counts(d):
41     r = {}
42hunk ./src/allmydata/web/check_results.py 14
43+    r["tahoe_version"] = str(__version__)
44     r["count-shares-good"] = d["count-shares-good"]
45     r["count-shares-needed"] = d["count-shares-needed"]
46     r["count-shares-expected"] = d["count-shares-expected"]
47hunk ./src/allmydata/web/check_results.py 350
48     def json(self, ctx):
49         inevow.IRequest(ctx).setHeader("content-type", "text/plain")
50         data = {}
51+        data["tahoe_version"] = str(__version__)
52         data["finished"] = self.monitor.is_finished()
53         res = self.monitor.get_status()
54         data["root-storage-index"] = res.get_root_storage_index_string()
55hunk ./src/allmydata/web/check_results.py 506
56         inevow.IRequest(ctx).setHeader("content-type", "text/plain")
57         res = self.monitor.get_status()
58         data = {}
59+        data["tahoe_version"] = str(__version__)
60         data["finished"] = self.monitor.is_finished()
61         data["root-storage-index"] = res.get_root_storage_index_string()
62         c = res.get_counters()
63hunk ./src/allmydata/web/directory.py 36
64 from allmydata.web.operations import ReloadMixin
65 from allmydata.web.check_results import json_check_results, \
66      json_check_and_repair_results
67+from allmydata._version import __version__
68 
69 class BlockingFileError(Exception):
70     # TODO: catch and transform
71hunk ./src/allmydata/web/directory.py 932
72         else:
73             origin_base32 = ""
74         status = { "stats": s["stats"],
75+                   "tahoe_version": str(__version__),
76                    "finished": m.is_finished(),
77                    "origin": origin_base32,
78                    }
79}
80
81Context:
82
83[Document reason for the trialcoverage version requirement being 0.3.3.
84david-sarah@jacaranda.org**20100525004444
85 Ignore-this: 2f9f1df6882838b000c063068f258aec
86]
87[Downgrade version requirement for trialcoverage to 0.3.3 (from 0.3.10), to avoid needing to compile coveragepy on Windows.
88david-sarah@jacaranda.org**20100524233707
89 Ignore-this: 9c397a374c8b8017e2244b8a686432a8
90]
91[Suppress deprecation warning for twisted.web.error.NoResource when using Twisted >= 9.0.0.
92david-sarah@jacaranda.org**20100516205625
93 Ignore-this: 2361a3023cd3db86bde5e1af759ed01
94]
95[docs: CREDITS for Jeremy Visser
96zooko@zooko.com**20100524081829
97 Ignore-this: d7c1465fd8d4e25b8d46d38a1793465b
98]
99[test: show stdout and stderr in case of non-zero exit code from "tahoe" command
100zooko@zooko.com**20100524073348
101 Ignore-this: 695e81cd6683f4520229d108846cd551
102]
103[setup: upgrade bundled zetuptoolz to zetuptoolz-0.6c15dev and make it unpacked and directly loaded by setup.py
104zooko@zooko.com**20100523205228
105 Ignore-this: 24fb32aaee3904115a93d1762f132c7
106 Also fix the relevant "make clean" target behavior.
107]
108[setup: remove bundled zipfile egg of setuptools
109zooko@zooko.com**20100523205120
110 Ignore-this: c68b5f2635bb93d1c1fa7b613a026f9e
111 We're about to replace it with bundled unpacked source code of setuptools, which is much nicer for debugging and evolving under revision control.
112]
113[setup: remove bundled copy of setuptools_trial-0.5.2.tar
114zooko@zooko.com**20100522221539
115 Ignore-this: 140f90eb8fb751a509029c4b24afe647
116 Hopefully it will get installed automatically as needed and we won't bundle it anymore.
117]
118[setup: remove bundled setuptools_darcs-1.2.8.tar
119zooko@zooko.com**20100522015333
120 Ignore-this: 378b1964b513ae7fe22bae2d3478285d
121 This version of setuptools_darcs had a bug when used on Windows which has been fixed in setuptools_darcs-1.2.9. Hopefully we will not need to bundle a copy of setuptools_darcs-1.2.9 in with Tahoe-LAFS and can instead rely on it to be downloaded from PyPI or bundled in the "tahoe deps" separate tarball.
122]
123[tests: fix pyflakes warnings in bench_dirnode.py
124zooko@zooko.com**20100521202511
125 Ignore-this: f23d55b4ed05e52865032c65a15753c4
126]
127[setup: if the string '--reporter=bwverbose-coverage' appears on sys.argv then you need trialcoverage
128zooko@zooko.com**20100521122226
129 Ignore-this: e760c45dcfb5a43c1dc1e8a27346bdc2
130]
131[tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__')
132zooko@zooko.com**20100521122052
133 Ignore-this: 96144a412250d9bbb5fccbf83b8753b8
134]
135[tests: increase timeout to give François's ARM buildslave a chance to complete the tests
136zooko@zooko.com**20100520134526
137 Ignore-this: 3dd399fdc8b91149c82b52f955b50833
138]
139[run_trial.darcspath
140freestorm77@gmail.com**20100510232829
141 Ignore-this: 5ebb4df74e9ea8a4bdb22b65373d1ff2
142]
143[docs: line-wrap README.txt
144zooko@zooko.com**20100518174240
145 Ignore-this: 670a02d360df7de51ebdcf4fae752577
146]
147[Hush pyflakes warnings
148Kevan Carstensen <kevan@isnotajoke.com>**20100515184344
149 Ignore-this: fd602c3bba115057770715c36a87b400
150]
151[setup: new improved misc/show-tool-versions.py
152zooko@zooko.com**20100516050122
153 Ignore-this: ce9b1de1b35b07d733e6cf823b66335a
154]
155[Improve code coverage of the Tahoe2PeerSelector tests.
156Kevan Carstensen <kevan@isnotajoke.com>**20100515032913
157 Ignore-this: 793151b63ffa65fdae6915db22d9924a
158]
159[Remove a comment that no longer makes sense.
160Kevan Carstensen <kevan@isnotajoke.com>**20100514203516
161 Ignore-this: 956983c7e7c7e4477215494dfce8f058
162]
163[docs: update docs/architecture.txt to more fully and correctly explain the upload procedure
164zooko@zooko.com**20100514043458
165 Ignore-this: 538b6ea256a49fed837500342092efa3
166]
167[Fix up the behavior of #778, per reviewers' comments
168Kevan Carstensen <kevan@isnotajoke.com>**20100514004917
169 Ignore-this: 9c20b60716125278b5456e8feb396bff
170 
171   - Make some important utility functions clearer and more thoroughly
172     documented.
173   - Assert in upload.servers_of_happiness that the buckets attributes
174     of PeerTrackers passed to it are mutually disjoint.
175   - Get rid of some silly non-Pythonisms that I didn't see when I first
176     wrote these patches.
177   - Make sure that should_add_server returns true when queried about a
178     shnum that it doesn't know about yet.
179   - Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set
180     of peerids, alter dependencies to deal with that.
181   - Remove upload.should_add_servers, because it is no longer necessary
182   - Move upload.shares_of_happiness and upload.shares_by_server to a utility
183     file.
184   - Change some points in Tahoe2PeerSelector.
185   - Compute servers_of_happiness using a bipartite matching algorithm that
186     we know is optimal instead of an ad-hoc greedy algorithm that isn't.
187   - Change servers_of_happiness to just take a sharemap as an argument,
188     change its callers to merge existing_shares and used_peers before
189     calling it.
190   - Change an error message in the encoder to be more appropriate for
191     servers of happiness.
192   - Clarify the wording of an error message in immutable/upload.py
193   - Refactor a happiness failure message to happinessutil.py, and make
194     immutable/upload.py and immutable/encode.py use it.
195   - Move the word "only" as far to the right as possible in failure
196     messages.
197   - Use a better definition of progress during peer selection.
198   - Do read-only peer share detection queries in parallel, not sequentially.
199   - Clean up logging semantics; print the query statistics whenever an
200     upload is unsuccessful, not just in one case.
201 
202]
203[Alter the error message when an upload fails, per some comments in #778.
204Kevan Carstensen <kevan@isnotajoke.com>**20091230210344
205 Ignore-this: ba97422b2f9737c46abeb828727beb1
206 
207 When I first implemented #778, I just altered the error messages to refer to
208 servers where they referred to shares. The resulting error messages weren't
209 very good. These are a bit better.
210]
211[Change "UploadHappinessError" to "UploadUnhappinessError"
212Kevan Carstensen <kevan@isnotajoke.com>**20091205043037
213 Ignore-this: 236b64ab19836854af4993bb5c1b221a
214]
215[Alter the error message returned when peer selection fails
216Kevan Carstensen <kevan@isnotajoke.com>**20091123002405
217 Ignore-this: b2a7dc163edcab8d9613bfd6907e5166
218 
219 The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError
220 for a variety of error conditions that weren't informatively described by them.
221 This patch creates a new error, UploadHappinessError, replaces uses of
222 NoSharesError and NotEnoughSharesError with it, and alters the error message
223 raised with the errors to be more in line with the new servers_of_happiness
224 behavior. See ticket #834 for more information.
225]
226[Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things.
227Kevan Carstensen <kevan@isnotajoke.com>**20091118014542
228 Ignore-this: a6cb032cbff74f4f9d4238faebd99868
229]
230[Change stray "shares_of_happiness" to "servers_of_happiness"
231Kevan Carstensen <kevan@isnotajoke.com>**20091116212459
232 Ignore-this: 1c971ba8c3c4d2e7ba9f020577b28b73
233]
234[Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778
235Kevan Carstensen <kevan@isnotajoke.com>**20091116192805
236 Ignore-this: 15289f4d709e03851ed0587b286fd955
237]
238[Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness.
239Kevan Carstensen <kevan@isnotajoke.com>**20091104111222
240 Ignore-this: abb3283314820a8bbf9b5d0cbfbb57c8
241]
242[Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness.
243Kevan Carstensen <kevan@isnotajoke.com>**20091104033241
244 Ignore-this: b3a6649a8ac66431beca1026a31fed94
245]
246[Alter CiphertextDownloader to work with servers_of_happiness
247Kevan Carstensen <kevan@isnotajoke.com>**20090924041932
248 Ignore-this: e81edccf0308c2d3bedbc4cf217da197
249]
250[Revisions of the #778 tests, per reviewers' comments
251Kevan Carstensen <kevan@isnotajoke.com>**20100514012542
252 Ignore-this: 735bbc7f663dce633caeb3b66a53cf6e
253 
254 - Fix comments and confusing naming.
255 - Add tests for the new error messages suggested by David-Sarah
256   and Zooko.
257 - Alter existing tests for new error messages.
258 - Make sure that the tests continue to work with the trunk.
259 - Add a test for a mutual disjointedness assertion that I added to
260   upload.servers_of_happiness.
261 - Fix the comments to correctly reflect read-onlyness
262 - Add a test for an edge case in should_add_server
263 - Add an assertion to make sure that share redistribution works as it
264   should
265 - Alter tests to work with revised servers_of_happiness semantics
266 - Remove tests for should_add_server, since that function no longer exists.
267 - Alter tests to know about merge_peers, and to use it before calling
268   servers_of_happiness.
269 - Add tests for merge_peers.
270 - Add Zooko's puzzles to the tests.
271 - Edit encoding tests to expect the new kind of failure message.
272 - Edit tests to expect error messages with the word "only" moved as far
273   to the right as possible.
274 - Extended and cleaned up some helper functions.
275 - Changed some tests to call more appropriate helper functions.
276 - Added a test for the failing redistribution algorithm
277 - Added a test for the progress message
278 - Added a test for the upper bound on readonly peer share discovery.
279 
280]
281[Alter various unit tests to work with the new happy behavior
282Kevan Carstensen <kevan@isnotajoke.com>**20100107181325
283 Ignore-this: 132032bbf865e63a079f869b663be34a
284]
285[Replace "UploadHappinessError" with "UploadUnhappinessError" in tests.
286Kevan Carstensen <kevan@isnotajoke.com>**20091205043453
287 Ignore-this: 83f4bc50c697d21b5f4e2a4cd91862ca
288]
289[Add tests for the behavior described in #834.
290Kevan Carstensen <kevan@isnotajoke.com>**20091123012008
291 Ignore-this: d8e0aa0f3f7965ce9b5cea843c6d6f9f
292]
293[Re-work 'test_upload.py' to be more readable; add more tests for #778
294Kevan Carstensen <kevan@isnotajoke.com>**20091116192334
295 Ignore-this: 7e8565f92fe51dece5ae28daf442d659
296]
297[Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers
298Kevan Carstensen <kevan@isnotajoke.com>**20091109003735
299 Ignore-this: 12f9b4cff5752fca7ed32a6ebcff6446
300]
301[Add more tests for comment:53 in ticket #778
302Kevan Carstensen <kevan@isnotajoke.com>**20091104112849
303 Ignore-this: 3bb2edd299a944cc9586e14d5d83ec8c
304]
305[Add a test for upload.shares_by_server
306Kevan Carstensen <kevan@isnotajoke.com>**20091104111324
307 Ignore-this: f9802e82d6982a93e00f92e0b276f018
308]
309[Minor tweak to an existing test -- make the first server read-write, instead of read-only
310Kevan Carstensen <kevan@isnotajoke.com>**20091104034232
311 Ignore-this: a951a46c93f7f58dd44d93d8623b2aee
312]
313[Alter tests to use the new form of set_shareholders
314Kevan Carstensen <kevan@isnotajoke.com>**20091104033602
315 Ignore-this: 3deac11fc831618d11441317463ef830
316]
317[Refactor some behavior into a mixin, and add tests for the behavior described in #778
318"Kevan Carstensen" <kevan@isnotajoke.com>**20091030091908
319 Ignore-this: a6f9797057ca135579b249af3b2b66ac
320]
321[Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes.
322Kevan Carstensen <kevan@isnotajoke.com>**20091018013013
323 Ignore-this: e12cd7c4ddeb65305c5a7e08df57c754
324]
325[Update 'docs/architecture.txt' to reflect readonly share discovery
326kevan@isnotajoke.com**20100514003852
327 Ignore-this: 7ead71b34df3b1ecfdcfd3cb2882e4f9
328]
329[Alter the wording in docs/architecture.txt to more accurately describe the servers_of_happiness behavior.
330Kevan Carstensen <kevan@isnotajoke.com>**20100428002455
331 Ignore-this: 6eff7fa756858a1c6f73728d989544cc
332]
333[Alter wording in 'interfaces.py' to be correct wrt #778
334"Kevan Carstensen" <kevan@isnotajoke.com>**20091205034005
335 Ignore-this: c9913c700ac14e7a63569458b06980e0
336]
337[Update 'docs/configuration.txt' to reflect the servers_of_happiness behavior.
338Kevan Carstensen <kevan@isnotajoke.com>**20091205033813
339 Ignore-this: 5e1cb171f8239bfb5b565d73c75ac2b8
340]
341[Clarify quickstart instructions for installing pywin32
342david-sarah@jacaranda.org**20100511180300
343 Ignore-this: d4668359673600d2acbc7cd8dd44b93c
344]
345[web: add a simple test that you can load directory.xhtml
346zooko@zooko.com**20100510063729
347 Ignore-this: e49b25fa3c67b3c7a56c8b1ae01bb463
348]
349[setup: fix typos in misc/show-tool-versions.py
350zooko@zooko.com**20100510063615
351 Ignore-this: 2181b1303a0e288e7a9ebd4c4855628
352]
353[setup: show code-coverage tool versions in show-tools-versions.py
354zooko@zooko.com**20100510062955
355 Ignore-this: 4b4c68eb3780b762c8dbbd22b39df7cf
356]
357[docs: update README, mv it to README.txt, update setup.py
358zooko@zooko.com**20100504094340
359 Ignore-this: 40e28ca36c299ea1fd12d3b91e5b421c
360]
361[Dependency on Windmill test framework is not needed yet.
362david-sarah@jacaranda.org**20100504161043
363 Ignore-this: be088712bec650d4ef24766c0026ebc8
364]
365[tests: pass z to tar so that BSD tar will know to ungzip
366zooko@zooko.com**20100504090628
367 Ignore-this: 1339e493f255e8fc0b01b70478f23a09
368]
369[setup: update comments and URLs in setup.cfg
370zooko@zooko.com**20100504061653
371 Ignore-this: f97692807c74bcab56d33100c899f829
372]
373[setup: reorder and extend the show-tool-versions script, the better to glean information about our new buildslaves
374zooko@zooko.com**20100504045643
375 Ignore-this: 836084b56b8d4ee8f1de1f4efb706d36
376]
377[CLI: Support for https url in option --node-url
378Francois Deppierraz <francois@ctrlaltdel.ch>**20100430185609
379 Ignore-this: 1717176b4d27c877e6bc67a944d9bf34
380 
381 This patch modifies the regular expression used for verifying of '--node-url'
382 parameter.  Support for accessing a Tahoe gateway over HTTPS was already
383 present, thanks to Python's urllib.
384 
385]
386[backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error
387Brian Warner <warner@lothar.com>**20100428050803
388 Ignore-this: 1fca7b8f364a21ae413be8767161e32f
389 
390 This handles the case where we upload a new tahoe directory for a
391 previously-processed local directory, possibly creating a new dircap (if the
392 metadata had changed). Now we replace the old dirhash->dircap record. The
393 previous behavior left the old record in place (with the old dircap and
394 timestamps), so we'd never stop creating new directories and never converge
395 on a null backup.
396]
397["tahoe webopen": add --info flag, to get ?t=info
398Brian Warner <warner@lothar.com>**20100424233003
399 Ignore-this: 126b0bb6db340fabacb623d295eb45fa
400 
401 Also fix some trailing whitespace.
402]
403[docs: install.html http-equiv refresh to quickstart.html
404zooko@zooko.com**20100421165708
405 Ignore-this: 52b4b619f9dde5886ae2cd7f1f3b734b
406]
407[docs: install.html -> quickstart.html
408zooko@zooko.com**20100421155757
409 Ignore-this: 6084e203909306bed93efb09d0e6181d
410 It is not called "installing" because that implies that it is going to change the configuration of your operating system. It is not called "building" because that implies that you need developer tools like a compiler. Also I added a stern warning against looking at the "InstallDetails" wiki page, which I have renamed to "AdvancedInstall".
411]
412[Fix another typo in tahoe_storagespace munin plugin
413david-sarah@jacaranda.org**20100416220935
414 Ignore-this: ad1f7aa66b554174f91dfb2b7a3ea5f3
415]
416[Add dependency on windmill >= 1.3
417david-sarah@jacaranda.org**20100416190404
418 Ignore-this: 4437a7a464e92d6c9012926b18676211
419]
420[licensing: phrase the OpenSSL-exemption in the vocabulary of copyright instead of computer technology, and replicate the exemption from the GPL to the TGPPL
421zooko@zooko.com**20100414232521
422 Ignore-this: a5494b2f582a295544c6cad3f245e91
423]
424[munin-tahoe_storagespace
425freestorm77@gmail.com**20100221203626
426 Ignore-this: 14d6d6a587afe1f8883152bf2e46b4aa
427 
428 Plugin configuration rename
429 
430]
431[setup: add licensing declaration for setuptools (noticed by the FSF compliance folks)
432zooko@zooko.com**20100309184415
433 Ignore-this: 2dfa7d812d65fec7c72ddbf0de609ccb
434]
435[setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division
436zooko@zooko.com**20100309163736
437 Ignore-this: c0623d27e469799d86cabf67921a13f8
438]
439[CREDITS to Jacob Appelbaum
440zooko@zooko.com**20100304015616
441 Ignore-this: 70db493abbc23968fcc8db93f386ea54
442]
443[desert-island-build-with-proper-versions
444jacob@appelbaum.net**20100304013858]
445[docs: a few small edits to try to guide newcomers through the docs
446zooko@zooko.com**20100303231902
447 Ignore-this: a6aab44f5bf5ad97ea73e6976bc4042d
448 These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.
449]
450[TAG allmydata-tahoe-1.6.1
451david-sarah@jacaranda.org**20100228062314
452 Ignore-this: eb5f03ada8ea953ee7780e7fe068539
453]
454Patch bundle hash:
455bc98cd5fb3432029698cfcc0df00a8fab93128b8