Ticket #321: NOTES

File NOTES, 2.1 KB (added by warner, at 2008-02-29T04:07:57Z)

some timing notes from our logfile analysis

Line 
1http://webapi.allmydata.com:8123/uri/URI%3ADIR2%3Atdpubdsjyuf4elbb6b7o6shqfa%3Arr4hyetavbujscw3kkn65uurdclagv2ecd5idporw6uxp7bwzz6a/
2URI:DIR2:tdpubdsjyuf4elbb6b7o6shqfa:rr4hyetavbujscw3kkn65uurdclagv2ecd5idporw6uxp7bwzz6a
3SI: rrbnfbtklzx3b3wyzmfpsswwbi
4
5The test dumped 353 files (each 1024 bytes long) into this directory, using
6the winfuse plugin.
7
8#The second file (000002) was URI:CHK:7eea67hqshtp66t5cxk527b7n4:66jakdueukpk6r772b3bhtcontpx73cpdbztpcuqa3asyj3vzj2q:3:10:1024
9#SI: qfzynyjjyjgzry2pwqeza3e5sq
10#From the FUSE plugins point of view, it took
11#into FUSE    out of FUSE   into tahoe   out of tahoe
12#open         close         start PUT    PUT returns
13#04:58:00.019 04:58:00.039  04:58:08.474 04:58:13.492  5.018s
14
15000006 was URI:CHK:42yikc6m4y4dyx6yz6s5bf5caq:3r4cjmvir5fyujjcxw4oc7wfqwaew4y5235xaupdvwwwkdwiwjua:3:10:1024
16SI: oqmlqeu6dwy5uufrt4fw6on5ui
1704:58:03.874 04:58:03.894  04:58:26.951 04:58:29.825  2.874s
18f31048
19CHK took 920ms (using helper)
20a t=json query that read the directory took 210ms
21 there were 3 such queries during the CHK upload.. too many!
22directory read took 390ms
23serialize takes 20ms
24unnecessary dir read took 280ms
25write takes 812ms
26 query results take 111ms-301ms
27 getting privkey takes 50ms
28 generate shares takes 10ms
29 sending shares takes 140ms-441ms
30
31
32The 350th file (000350) was URI:CHK:pmo6dl3ro6embt7v3d6rn42pje:3s55yzaxspuw7zm2pteur5jd3g6oerfshrvyr43y2b6vkjlw62bq:3:10:1024
33SI: edwrvhelj5hqnlv7lb2j2fg64a
3405:02:14.201 05:02:14.221  05:54:47.233 05:54:57.958   10.725s
35CHK took 791ms
36dirread took 1.763s
37serialize takes 500ms
38unnecessary dirread took 1.613
39write takes 4.066
40 query results take 200ms-270ms
41 privkey takes 120ms
42 generating shares 20ms
43 sending shares took 1.813-3.646
44
45dirnode.add_file:
46 CHK Upload
47  "upload finished, building readcap"
48 Retrieve
49 Retrieve
50 Publish
51 "webish upload complete"
52 web log
53
54353 files
55114305 bytes
56324 bytes per file
57
58TODO:
59 write-batching:
60  expose dirnode.set_uris() to HTTP
61  get Mike to:
62   upload and get URI, then update directory
63   nagle the directory updates, for 350 entries that's 85%
64 remove unnecessary Retrieve from dirnode._update  15%
65  expose seqnum*, or split overwrite from replace
66