id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	launchpad_bug
437	automatically schedule tests of large files	zooko	somebody	"#435 (automate testing of large files) is to create an automated test which, once launched, can determine if a given version of Tahoe correctly uploads and downloads large files.  For starters, we need to test files > 4 GiB, and also we need to test files > 12 GiB and probably also even larger in the future.

This ticket is to automatically schedule those tests to get run under certain conditions.  Obviously running such a test can be expensive in terms of disk, CPU, and network, so we don't necessarily want to run such a test on the normal schedule of all of our unit tests, which are done on every darcs commit on all test builders.

Also running such a test takes a long time, so we don't want to habitually block other processes (such as running other unit tests or building packages) on such a large-file test completing.

Brian has already expressed a desire that large-file tests are not automatically launched at all, and instead just get launched by the specific decision of a human user.  I disagree and would like for us to have an automated policy for running large-file tests.  If we go with Brian's preference then we'll just close this ticket as ""invalid"" or ""wontfix"".

Here are some policies which are easily implemented in buildbot which can reduce the problems of an expensive test like this:

1.  We can limit the large-file test to only certain buildslaves.

2.  We can make it so that other processes go ahead without waiting to see the result of the large-file test (such as other unit tests, buildbot results e-mails, building of packages, etc.)

3.  Buildbot normally always makes it so that a new test won't launch if one is already in progress, and so that if multiple triggers for a test accumulate then only one test is needed to satisfy all those triggers.  That can help with this.

4.  We could make it so that large-file tests are not triggered by darcs commit activity, but instead are triggered once per day, or once per week, or whatever.

We could implement more detailed policies, such as ""do it every day at midnight in UTC-7 if there was any darcs commit that day"" or ""if it takes X hours to run the test, then wait at least 3 * X hours before starting the next test"", but I don't know off the top of my head how easy it is to program buildbot for such a policy."	defect	new	major	eventually	dev-infrastructure	1.0.0		test large