Opened at 2015-11-16T18:36:24Z
Closed at 2015-12-07T15:55:12Z
#2577 closed defect (fixed)
Magic Folder: files transferred multiple times
| Reported by: | meejah | Owned by: | daira |
|---|---|---|---|
| Priority: | normal | Milestone: | undecided |
| Component: | code-frontend-magic-folder | Version: | n/a |
| Keywords: | blocks-magic-folder-merge reliability performance review-needed | Cc: | |
| Launchpad Bug: |
Description
I've several times seen a client get into a state whereby it downloads the same file more than once; you can "usually" get this to happen if you restart the node while it's currently downloading at least one file.
I'm still not sure if it's the uploader re-uploading something or the downloader deciding to re-download something or even just a bug in the very-new status code.
It could also be related to a "big" download that takes > the scan-interval (3 seconds) to completely process.
Change History (11)
comment:1 Changed at 2015-11-19T10:05:06Z by dawuud
comment:2 Changed at 2015-11-19T11:07:47Z by dawuud
i tried to fork off of stable.5 and just replace the time.time() with clock.seconds() but failed to make on of the unit tests to pass: https://github.com/david415/tahoe-lafs/tree/2438.magic-folder-stable.5-add-pending-set.0
comment:3 Changed at 2015-11-19T12:00:23Z by dawuud
hey i think i fixed this bug here in this branch: https://github.com/david415/tahoe-lafs/tree/2569.magic-status-cli.2
this branch adds a pending set to the downloader in order to deduplicate file objects that are downloaded. this should now handle the case where an single download may take longer than the downloader turn deque delay. i did try it locally with a huge file and it worked.
please review.
comment:4 Changed at 2015-11-19T12:00:38Z by dawuud
- Owner changed from daira to meejah
comment:5 Changed at 2015-11-19T16:37:54Z by meejah
dawuud: looks plausible, thanks for looking at this!
comment:6 Changed at 2015-12-02T02:54:30Z by daira
- Component changed from unknown to code-frontend-magic-folder
- Keywords reliability added
comment:7 Changed at 2015-12-02T02:54:54Z by daira
- Keywords performance added
comment:8 Changed at 2015-12-02T02:55:18Z by daira
- Keywords review-needed added
comment:9 Changed at 2015-12-03T20:45:05Z by daira
- Keywords blocks-magic-folder-merge added; magic-folder removed
- Owner changed from meejah to daira
- Status changed from new to assigned
- Summary changed from files transferred multiple times to Magic Folder: files transferred multiple times
Will review.
comment:10 Changed at 2015-12-04T10:14:42Z by dawuud
i made a newer version of this pending set for the downloader
https://github.com/david415/tahoe-lafs/tree/2438.magic-folder-stable.5-add-pending-set.1
please review? https://github.com/tahoe-lafs/tahoe-lafs/pull/207
comment:11 Changed at 2015-12-07T15:55:12Z by daira
- Resolution set to fixed
- Status changed from assigned to closed
- Version changed from 1.10.1 to n/a
This is merged to 2438.magic-folder-stable.5.
i tested for this bug on branch 2438.magic-folder-stable.5, and found it to not produce the bug. i'd guess this was somehow introduced with this status feature... although i'm not sure how.