port memory usage tests to windows

[zooko]

Make the following also work on Windows: Measure the memory usage in different operations (upload, download, sustained server operation, ...) and graph it with a munin plugin. (Issue #29)

[zooko]

tips and tricks for measuring CPU and RAM on Windows

[zooko]

See #29 for the general memory-usage tests and fixing the memory-usage problems.

[zooko]

bumping this issue out of the v0.6.1 milestone because there are too many other more urgent things for this milestone

[zooko]

If you love this ticket (#54), then you might like tickets #227 (our automated memory measurements might be measuring the wrong thing), #419 (pycryptopp uses up too much RAM), #478 (add memory-usage to stats-provider numbers), and #97 (reducing memory footprint in share reception).

[Grumpf]

Here is a patch proposal. It also makes "make check-memory" works under MinGW, with or without installed Twisted.

The "VmData?" value doesn't seems to have any equivalence in GetProcessMemoryInfo?() nor any use in current code, so might be stripped out.

[davidsarah]

Comments on the patch:

• on non-Windows a failure of the code in the try: block will cause the values to be left at zero, but on Windows a failure will propagate the exception. Make it consistent.
• the duplication of code in startstop_node.py and check_memory.py is undesirable -- move it to pyutil, perhaps.

[davidsarah]

Replying to davidsarah:

• the duplication of code in startstop_node.py and check_memory.py is undesirable -- move it to pyutil, perhaps.

Seems like a suitable place would be find_exe.find_twistd.

[zooko]

Grumpf: thank you for the patch!

So let's see, since David-Sarah's review indicated that this patch isn't ready to be committed then we should unset the reviewed keyword, right? By the way, ​http://allmydata.org/trac/pyutil/browser/pyutil/pyutil/memutil.py would be an ideal place for this functionality to live, eventually, but this doesn't directly help Tahoe-LAFS until and unless #47 (use pyutil as a separate package and contribute src/allmydata/util/* into pyutil) is closed.

Okay unsetting reviewed and assigning it to Grumpf to write a patch that fixes the issues raised by David-Sarah. Grumpf: I think David-Sarah is right that using src/allmydata/util/find_exe.py is a good way to do the "find twistd" behavior.

[zooko]

Oh sorry, I didn't read your patch and see that you were already using find_exe. I guess the code duplication that David-Sarah was talking about was the part that comes after find_exe about looking for an executable that was installed in ./support relative to the basedir.

It is the code in src/allmydata/control.py which could potentially be merged with the comparable code in ​http://allmydata.org/trac/pyutil/browser/pyutil/pyutil/memutil.py .

[zooko]

Grumpf: if you are going to work on this patch then please "accept" this ticket (using the control panel at the bottom of this page). If you don't then anyone else should feel free to update the patch to fix the details noted by David-Sarah's review.