[tahoe-dev] String encoding in tahoe

[Francois Deppierraz]

zooko wrote:

> Okay, after testing on my Macbook Pro, I committed François's patch  
> [1], and some related patches of my own [2, 3, 4].  This fixed the  
> cli tests on Ubuntu Feisty -- hooray!  But it broke the test on  
> cygwin, GNU/OpenSolaris, Windows, and ArchLinux -- boo!  See the  
> buildbot for details [5].

Looking at builbot logs, I see two different issues.

On Cygwin, OpenSolaris and ArchLinux with LANG=C, os.path.exists() fails
because the filesystem doesn't support UTF-8 encoded filenames. Under
Linux it depends on locale settings as Dan pointed out.

Before applying my patch to convert sys.argv to unicode, we were sending
out UTF-8 encoded filenames which certainly had strange effects on
filesystems without support for UTF-8 filenames.

The meaning of this failing test is probably "You're trying to
manipulate a file whose name cannot be represented on this filesystem"
which sounds reasonable and should raise an application error.

Maybe, we should simply avoid running this specific test on platforms
which cannot represent filenames with anything else than ascii ? Because
it doesn't make sense anyway in such case.

On Windows, it fails to find the file it just created and tries to print
an error message containing the incriminated filename as unicode which
raises UnicodeEncodeError. Perhaps stdout only supports ascii ? This one
needs further investigation.

François