#53 closed enhancement (fixed)

command-line interface

Reported by: zooko Owned by: zooko
Priority: blocker Milestone: 0.5.0
Component: code-frontend Version: 0.2.0
Keywords: Cc:
Launchpad Bug:


I met a nice guy named Jeff Hehner at Caffe Sole tonight, who said that he wanted a command-line interface to a tool like Tahoe.

I agreed with him that command-line interfaces are very good to have.

I'm marking this one as "priority: major". Along with the XML-RPC interface (#48) this might become my next primary focus. Basically, Tahoe already works well enough for several valuable use cases, and now we just need to enable people to use it. (Meaning, of course, people who are enabled by CLI and XML-RPC... People who are enabled by Windows filesystem integration come next...)

This is already mentioned in source:roadmap.txt .

Change History (20)

comment:1 Changed at 2007-05-28T23:11:31Z by zooko

  • Owner changed from somebody to zooko
  • Priority changed from major to critical
  • Status changed from new to assigned

I'm motivated to work on this. The first question is: does the process that you execute block until the operation completes or do we do some funky trick such as it producing a randomly-named temporary file that it later populates when the operation completes? Tahoe (and all of our thinking) is strongly in the non-blocking, event-based paradigm, but I think this is an exception. The purpose of this component is to interface with Unix users and programmers, and Unix users and programmers expect their long-running command-lines to block.

Raising the priority, not because it is "critical", but to indicate that I'm more motivated by this than by some other tickets. Actually perhaps it is "critical". A word like "critical" begs the question "critical for *what*?", and this ticket is critical for getting certain Unix system administrators to give tahoe a spin.

comment:2 Changed at 2007-05-29T19:46:50Z by warner

I think the CLI process should block until the operation completes. I'd like to see a 'tahoe put LOCALFILE VDRIVEFILENAME', 'tahoe get VDRIVEFILENAME LOCALFILE', and then a collection of mv/rm/cp type operations. Actually an interface like scp's would be great (which would imply 'tahoe-cp ~/stuff.txt vdrive:here.txt', or something).

I'm assuming that there is a long-running daemon, and that the CLI tool is a short-lived process that connects to the daemon, tells it to do something, waits for it to indicate completion, emits a message, then quits. If this is the case, both Foolscap and XMLRPC are useful approaches (with either a FURL or a webfront-style "account handle" as the secret authentication bootstrapping pieces). Another technique I've used the past (on buildbot) is to have a special "command directory", like an inbox, which is a Maildir where you can drop in specially-formatted uniquely-named files. A file might have "upload" on the first line, a local filename on the second line, and a vdrive filename on the third line. The daemon could poll this directory every few seconds (or use inotify), execute the commands it finds there, and delete the file when it is done. This reduces the authentication-bootstrap issue to writability of a specific directory, and doesn't require any network for the CLI tool at all.

(incidentally, I'm unsure about using trac's 'priority' field to indicate which tickets we're currently interested in.. we aren't confined to resolving these tickets in any particular order. maybe we need another field named 'coolness' or 'sexyness' for this purpose..)

comment:3 Changed at 2007-05-30T16:43:01Z by zooko

Re: trac "priority" field, let's discuss it in #60.

Re: scp interface: Perfect! Thanks.

Re: command-line backend -- how about if we implement #48 -- XML-RPC interface -- and then use that for the command-line client?

comment:4 Changed at 2007-06-06T22:39:32Z by warner

in thinking about this more, I think I want to see the client provide a virtual drive (composed of a variety of filetrees) in exactly the same way that a kernel provides a single filesystem (composed of a variety of volumes) to applications. The basic ls/cp/rm/mkdir interface can treat this vdrive as a single filesystem, without being too aware of the pieces that it comprises.

More advanced tools can use additional APIs which expose the filetree pieces, and provide ways to manipulate them (splitting a single filetree into multiple pieces so one can be shared without the other, creating a "virtual CD" snapshot from a given root, etc). But frontends that want to think of the vdrive as just a regular filesystem are free to do so.

comment:5 Changed at 2007-06-07T18:12:51Z by warner

robk rightly points out that we should look at xuilcli (from amdlib), and see if we can adapt that.

comment:6 Changed at 2007-07-02T19:44:17Z by warner

  • Milestone set to release 0.5.0

comment:7 Changed at 2007-07-13T03:10:41Z by warner

we have 'tahoe ls' and 'tahoe get' working right now. I think we should declare our 0.5.0 goals to be 'tahoe get -r' and 'tahoe put -r' as well, and then call it a release. Maybe add 'rm' as well. But 'cp' and 'mv' can probably wait for 0.5.1 .

comment:8 Changed at 2007-07-23T18:09:52Z by zooko

  • Priority changed from critical to blocker

This is a blocker for the v0.5 release.

comment:9 Changed at 2007-08-03T16:52:09Z by zooko

Let's make it so that "allmydata-tahoe ls foo/bar.txt" lists bar.txt (if it exists). Currently ls ignores arguments which aren't directories.

comment:10 Changed at 2007-08-07T21:35:34Z by zooko

Nathan had a good question: why do we have the "-r" flag? If someone asks to "tahoe get" a directory, then what else could they mean other than that they want to recursively get the directory's contents?

comment:11 Changed at 2007-08-07T22:26:02Z by warner

I suspect the usual answer is that there's a significant probability that they referred to a directory by mistake, and might be surprised to find themselves downloading hundreds of files instead of just one.

A weaker answer is "because all the other tools do it", and consistency has some merits.

And the historical answer is probably that you can actually open and read directories, although the results are not normally useful, and I think more recent unix kernels removed this ability.

comment:12 Changed at 2007-08-08T02:23:23Z by zooko

Good enough for me.

comment:13 Changed at 2007-08-09T20:49:36Z by zooko

Interesting, "tahoe get ADIRECTORY" currently emits an html page describing that directory. :-)

comment:14 Changed at 2007-08-14T18:52:21Z by warner

  • Component changed from code to code-frontend

comment:15 Changed at 2007-08-15T18:46:27Z by zooko

  • Owner changed from zooko to warner
  • Status changed from assigned to new

Okay, so far we have:

allmydata-tahoe ls
allmydata-tahoe get

We would like to add

allmydata-tahoe put

And that will be good enough for v0.5! (Note that we won't yet support "-r", nor a bunch of features for "ls", etc.)

comment:16 Changed at 2007-08-15T19:17:03Z by zooko

  • Owner changed from warner to zooko

comment:17 Changed at 2007-08-15T19:17:07Z by zooko

  • Status changed from new to assigned

comment:18 Changed at 2007-08-16T21:51:43Z by zooko

Almost done with "put"...

comment:19 Changed at 2007-08-17T19:31:35Z by zooko

Fixing some UI issues concerning --node-url.

Probably going to add "rm" real quick...

comment:20 Changed at 2007-08-17T20:30:32Z by zooko

  • Resolution set to fixed
  • Status changed from assigned to closed

Implemented rm. 0c22044f7edb16b4

Now doing extra manual testing before v0.5 release!

Note: See TracTickets for help on using tickets.