#1472 closed defect (wontfix)

buildbot is unable to delete directories under _trial_temp* that have non-ASCII names under Windows

Reported by: davidsarah Owned by: somebody
Priority: major Milestone: eventually
Component: dev-infrastructure Version: 1.8.2
Keywords: buildbot unicode windows Cc:
Launchpad Bug:

Description (last modified by exarkun)

The drop-upload tests added in 32a7717205ed824a create a subdirectory under _trial_temp that has a non-ASCII name (locāl_dir). On Windows, this causes an error subsequent builds when buildbot tries to delete the "workdir" for the build, which contains _trial_temp. This seems to happen because it ASCIIfies the name to "local_dir" when trying to change its permissions to ensure it is deleteable.

An ugly workaround would be for the tests to delete any non-ASCII directories they create. This should really be fixed in buildbot, though.

The traceback is:

starting darcs operation

remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 1425, in start
    self.maybeClobber(d)
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 1444, in maybeClobber
    d.addCallback(self.doClobber, self.workdir)
  File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 297, in addCallback
    callbackKeywords=kw)
  File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 286, in addCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 1600, in doClobber
    rmdirRecursive(d)
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 127, in rmdirRecursive
    rmdirRecursive(full_name)
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 127, in rmdirRecursive
    rmdirRecursive(full_name)
  File "c:\python27\lib\site-packages\buildbot-0.7.12-py2.7.egg\buildbot\slave\commands.py", line 124, in rmdirRecursive
    os.chmod(full_name, 0600)
exceptions.WindowsError: [Error 2] Le fichier spécifié est introuvable: 'C:\\buildbot_tahoe\\FreeStorm_WinXP-x86_py2.6\\build\\_trial_temp_old228388\\drop_upload.MockTest.test_drop_upload\\local_dir'
]

Change History (13)

comment:1 Changed at 2011-08-09T02:53:02Z by davidsarah

70f03bd6158168ed should work around this.

comment:2 Changed at 2011-08-09T03:08:18Z by davidsarah

  • Description modified (diff)

Bug filed against buildbot: http://trac.buildbot.net/ticket/2073

comment:3 Changed at 2011-08-10T04:26:34Z by david-sarah@…

In db22fdc20dc93a3e:

Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472

comment:4 Changed at 2011-08-10T05:48:53Z by david-sarah@…

In 2deba3319cbf02c8:

(The changeset message doesn't reference this ticket)

comment:5 Changed at 2011-08-10T06:25:17Z by david-sarah@…

In 0dc56daaf9540910:

test/common_util.py: correct fix to mkdir_nonascii. refs #1472

comment:6 Changed at 2011-08-10T17:28:13Z by david-sarah@…

In [5163/ticket393-MDMF-2]:

Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472

comment:17 Changed at 2011-08-10T17:28:14Z by david-sarah@…

In [5165/ticket393-MDMF-2]:

test/common_util.py: fix a typo. refs #1472

comment:18 Changed at 2011-08-10T17:28:15Z by david-sarah@…

In [5166/ticket393-MDMF-2]:

test/common_util.py: correct fix to mkdir_nonascii. refs #1472

comment:19 Changed at 2011-08-12T15:15:56Z by davidsarah

  • Milestone changed from 1.9.0 to eventually
  • Priority changed from critical to major

This is strictly speaking invalid now for Tahoe-LAFS, since the actual bug is in buildbot and we've worked around the immediate problem. However, when buildbot is fixed we'll need to decide whether to remove the workaround.

In the meantime, other tests that create non-ASCII directory names should mix in NonASCIIPathMixin from src/allmydata/test/common_util.py and use self.mkdir_nonascii.

comment:20 follow-up: Changed at 2011-08-14T03:45:45Z by zooko

FWIW, there is a class in fileutil.py that does something similar: trunk/src/allmydata/util/fileutil.py. There is also a more featureful version of that class in pyutil: pyutil/fileutil.py.

Version 2, edited at 2011-08-14T03:53:30Z by davidsarah (previous) (next) (diff)

comment:21 in reply to: ↑ 20 ; follow-up: Changed at 2011-08-14T04:02:59Z by davidsarah

Replying to zooko:

FWIW, there is a class in fileutil.py that does something similar: src/allmydata/util/fileutil.py@4957#L90.

NamedTemporaryDirectory deletes the directory tree when the NamedTemporaryDirectory object is __del__'d, which is the wrong time. We need to delete it when the test has finished (which, because of Deferreds, will be some time after the test method returns). There's no simpler way to do this than using addCleanup, as the current code does.

comment:22 in reply to: ↑ 21 Changed at 2011-11-22T12:28:44Z by zooko

Replying to davidsarah:

Replying to zooko:

FWIW, there is a class in fileutil.py that does something similar: src/allmydata/util/fileutil.py@4957#L90.

NamedTemporaryDirectory deletes the directory tree when the NamedTemporaryDirectory object is __del__'d, which is the wrong time.

For future reference, you can also invoke the NamedTemporaryDirectory object's shutdown method at a specific time if desired.

comment:23 Changed at 2019-07-25T12:50:10Z by exarkun

  • Description modified (diff)
  • Resolution set to wontfix
  • Status changed from new to closed

Buildbot has been decommissioned.

Note: See TracTickets for help on using tickets.