connecting to sftp frontend using sshfs fails from linux client

[azazel]

Strange error, i've tried 3 linux clients, and to specify two commandline variations:

sshfs -C -p 8022 server: local_dir

and

sshfs -C -p 8022 server:/ local_dir

boh fail in setting up the mount with errors like:

server:.: Operation not permitted

on the client and tracebacks like this on the server:

2009-02-26 15:37:57+0100 [SSHChannel session (0) on SSHService ssh-connection on SSHServerTransport,0,88.1 49.141.217] GETATTRS . 1 2009-02-26 15:37:57+0100 [SSHChannel session (0) on SSHService ssh-connection on SSHServerTransport,0,88.1 49.141.217] Unhandled Error

Traceback (most recent call last):

File "/home/azazel/wip/tahoe/darcs/tahoe/Twisted-8.2.0-py2.5-linux-i686.egg/twisted/conch/ssh/se

ssion.py", line 105, in dataReceived

self.client.transport.write(data)

File "/home/azazel/wip/tahoe/darcs/tahoe/Twisted-8.2.0-py2.5-linux-i686.egg/twisted/conch/ssh/se

ssion.py", line 156, in write

self.proto.dataReceived(data)

File "/home/azazel/wip/tahoe/darcs/tahoe/Twisted-8.2.0-py2.5-linux-i686.egg/twisted/conch/ssh/fi

letransfer.py", line 53, in dataReceived

f(data)

File "/home/azazel/wip/tahoe/darcs/tahoe/Twisted-8.2.0-py2.5-linux-i686.egg/twisted/conch/ssh/fi

letransfer.py", line 321, in packet_STAT

d = defer.maybeDeferred(self.client.getAttrs, path, followLinks)

--- <exception caught here> ---

File "/home/azazel/wip/tahoe/darcs/tahoe/Twisted-8.2.0-py2.5-linux-i686.egg/twisted/internet/def

er.py", line 106, in maybeDeferred

result = f(*args, kw)

File "/home/azazel/wip/tahoe/lib/python2.5/site-packages/allmydata_tahoe-1.3.0_r3745-py2.5.egg/a

llmydata/frontends/sftpd.py", line 305, in getAttrs

d = self._get_node_and_metadata_for_path(self._convert_sftp_path(path))

File "/home/azazel/wip/tahoe/lib/python2.5/site-packages/allmydata_tahoe-1.3.0_r3745-py2.5.egg/a

llmydata/frontends/sftpd.py", line 317, in _convert_sftp_path

assert pathstring[0] == "/"

exceptions.AssertionError?:

The same sshfs command does work when connecting to the allmydata's production sftp service.

[azazel]

I'm sorry for the ugly formatting of my last post, i 've clicked submit instead of preview. Attached there's a patch that fix this bug. I'm not a test expert so if anyone can suggest an idea about how implement a test for this please do it.

[warner]

Well, I don't think that will hurt anything, so if it makes sshfs connect correctly, I'll apply it.

As for tests, wow, I haven't really thought about that yet. Well, Twisted has an SSH client implementation too (we're using the server code here), so maybe it'd be possible to exercise some of the code that way. These tests would have to be skipped gracefully if conch or pycrypto were unavailable.

[warner]

Applied, in 3035dfb8ed70ae4b. Thanks!

[rheimbuch]

Path 3035dfb8ed70ae4b causes the contents of the 'root' directory to be displayed in all sub-directories. Example:

sftp> ls /
/foobar    /test-dir  
sftp> ls /foobar
/foobar/foobar      /foobar/test-dir    
sftp> ls /foobar/foobar
/foobar/foobar/foobar     /foobar/foobar/test-dir   
sftp> 

I get this behavior in both the sftp command line and sshfs.Reverting the patch seems to fix the it, but breaks sshfs compatability. The problem seems to be with the returning an empty path array when the path == "."

[azazel]

I don't get the same behavior, here is mine:

sftp> ls /
/documenti     /personale      /test
sftp> ls /test                                              
/test/Archives  /test/Latest                                
sftp> version                                                                   
SFTP protocol version 3
azazel@lizard:~$ dpkg -S sftp
openssh-client: /usr/bin/sftp
azazel@lizard:~$ dpkg -l openssh-client
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name                   Version                Description
+++-======================-======================-===========
ii  openssh-client         1:4.6p1-2             

Please add details of your environment to this ticket, so that i can reproduce it, maybe.

[rheimbuch]

Just to clarify: I get the correct behavior using the 1.3.0 release. I get recursive display of the root directory when I build the trunk with changeset 3035dfb8ed70ae4b.

The original run was on a Ubuntu Hardy LTS server:

rheimbuch@swoop:~$ uname -a
Linux swoop 2.6.24-19-generic #1 SMP Wed Jun 18 14:15:37 UTC 2008 x86_64 GNU/Linux

rheimbuch@swoop:~$ dpkg -l openssh-client
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-f/Unpacked/Failed-cfg/Half-inst/t-aWait/T-pend
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name                   Version                Description
+++-======================-======================-============================================================
ii  openssh-client         1:4.7p1-8ubuntu1.2     secure shell client, an rlogin/rsh/rcp replacement

To reproduce I repeated on my laptop with OSX 10.5.6 with fresh installs of the 1.3.0 release and the trunk with changeset 3035dfb8ed70ae4b. v1.3.0 works correctly through sftp. The trunk build has the broken behavior:

ryan-heimbuchs-macbook41:Volumes ryan$ cd
ryan-heimbuchs-macbook41:~ ryan$ sftp -oPort=8022 test2@localhost
Connecting to localhost...
test2@localhost's password: 
sftp> ls
test1  test2  
sftp> ls test1
test1/test1  test1/test2  
sftp> ls test2
test2/test1  test2/test2  
sftp> exit

ryan-heimbuchs-macbook41:~ ryan$ ssh -V
OpenSSH_5.1p1, OpenSSL 0.9.7l 28 Sep 2006

ryan-heimbuchs-macbook41:~ ryan$ which ssh
/usr/bin/ssh

ryan-heimbuchs-macbook41:~ ryan$ Projects/tahoe/trunk/bin/tahoe -V
allmydata-tahoe: 1.3.0-r3759, foolscap: 0.3.2, pycryptopp: 0.5.12, zfec: 1.4.4, Twisted: 8.2.0, Nevow: 0.9.32, zope.interface: 3.3.0, python: 2.5.1, platform: Darwin-9.6.0-i386-32bit, simplejson: 2.0.9, argparse: 0.8.0, pyOpenSSL: 0.6, pyutil: 1.3.30, zbase32: 1.1.1, setuptools: 0.6c12dev

[zooko]

Could we test this code without going all the way through the SSH interface, just by invoking the appropriate Python methods of the server? Going through SSH would be fine, too, but we do need tests for this.

[warner]

I suppose the first step would be to build up a catalog of how ssh clients behave. I wrote the convert_sftp_path method empirically, watching what a few clients I had available to me did. With the required inputs defined, then yeah, a non-round-trip test which creates an sftpd.SFTPHandler instance and invokes methods like openFile would be a good test. (it would still need to be skipped gracefully if conch were not available).

[zooko]

Oh, hey, this is a regression from 1.3.0? Should we revert 3035dfb8ed70ae4b before 1.4.0 release?

[zooko]

So my understanding of this issue is that Alberto's patch makes it start working for Alberto's use case but stop working for Ryan's use case, and that there are no unit tests. I'm going to revert this patch for Tahoe-1.4.0 release on the principle that Alberto's use case didn't work in Tahoe-1.3.0 but Ryan's did, and a regression is worse than "it still doesn't work".

The next step should be for someone (Alberto?) to write unit tests.

Moving this ticket to Milestone 1.4.1 and planning to revert the patch within the next 24 hours or so.

[zooko]

azazel: the next step on this would be for you to write a unit test. I would be willing to show you how to do so.

[davidsarah]

From the patch:

if pathstring == "" or  ".":

This is clearly wrong, since "." is truthy. It was probably supposed to be

if pathstring == "" or pathstring == ".":

However, I'm confused as to why sshfs worked when the path was always set to [].

[davidsarah]

Corrected potential fix for sftpd path handling (ticket #645)

[davidsarah]

Corrected potential fix for sftpd path handling (ticket #645) as unified diff

[davidsarah]

Assertions for size attribute, to track down the bug described in ​http://allmydata.org/pipermail/tahoe-dev/2010-February/003831.html

[davidsarah]

This is intended to be fixed by the new SFTP implementation in #1037.

[davidsarah]

azazel: please check out the ticket1037 branch using

darcs get --lazy http://allmydata.org/source/tahoe-lafs/ticket1037 tahoe-sftp

and try this again.

[davidsarah]

This bug has essentially been obsoleted by the new SFTP implementation. bj0 has been testing with sshfs on Linux, and josipl with sshfs on OS X; neither have seen the bugs mentioned here.