Uploading files with http put

Tue Nov 26 18:07:09 UTC 2019

Jean-Paul,

Does that mean you want to be able to do an HTTP PUT to start an upload and then poll the upload status with HTTP GET?

I checked the Nextcloud code to see how it handles HTTP Put. It looks like Nextcloud files_external applications that use HTTP (like the Amazon S3 plugin) use a combination of the icewinds php stream wrapper and guzzle http to monitor a file upload via http put. Guzzle http is a php package that support asynchronous http operation. The S3 plugin uses a combination of the icewinds php stream wrapper and guzzle http to monitor the number of bytes transferred from a local file to the web server via HTTP put.

Using the method I described above, all I would need is a way to do a synchronous HTTP put of a local file to a URL representing an immutable file resource on the Tahoe-LAFS server. 

Note that this approach is extremely complicated in php but is likely doable.

If not, what does your ideal interaction between Nextcloud and Tahoe-LAFS look like?

The simplest approach would be to make progress functionality available via the tahoe CLI.

One implementation approach would be to have the Tahoe “cp” CLI send progress indications to stdout as it is copying. The progress indications could be in terms of bytes transferred, % complete, etc. If this type of functionality was available, then I could simple use this php code snippet to monitor progress in PHP:

https://stackoverflow.com/questions/45953/php-execute-a-background-process#45966

                Bruce T

From: Jean-Paul Calderone <jean-paul+tahoe-dev at leastauthority.com>
Date: Tuesday, November 26, 2019 at 8:26 AM
To: Bruce Thompson <brucet.cisco at gmail.com>
Cc: "tahoe-dev at tahoe-lafs.org" <tahoe-dev at tahoe-lafs.org>
Subject: Re: Uploading files with http put

On Tue, Nov 26, 2019 at 9:23 AM brucet <brucet.cisco at gmail.com> wrote:

Jean-Paul,

Can you elaborate on what you mean by "asynchronous"?

PHP does not directly support asynchronous operations so I used the term loosely. 

Nextcloud includes a stream package that does callbacks on php streams. This is a link to the icewinds php stream wrapper which is included in the nextcloud code:

https://github.com/icewind1991/Streams

This library supports callbacks to provide updated status in the web UI when performing reads / writes on a php stream. This functionality works with http put / get. I am using this functionality for file downloads from Tahoe-LAFS by using http get. I would like to do the same for file uploads to Tahoe-LAFS by using http put.

Does that mean you want to be able to do an HTTP PUT to start an upload and then poll the upload status with HTTP GET?

If not, what does your ideal interaction between Nextcloud and Tahoe-LAFS look like?

                Bruce T

From: Jean-Paul Calderone <jean-paul+tahoe-dev at leastauthority.com>
Date: Monday, November 25, 2019 at 6:38 PM
To: Bruce Thompson <brucet.cisco at gmail.com>
Cc: "tahoe-dev at tahoe-lafs.org" <tahoe-dev at tahoe-lafs.org>
Subject: Re: Uploading files with http put

Can you elaborate on what you mean by "asynchronous"?

On Mon, Nov 25, 2019 at 7:28 PM brucet <brucet.cisco at gmail.com> wrote:

Never saw a response to this. Does anyone have any pointer of where to look for an answer to my question below?

Thanks,

                Bruce T

From: Bruce Thompson <brucet.cisco at gmail.com>
Date: Friday, November 22, 2019 at 9:08 AM
To: "tahoe-dev at tahoe-lafs.org" <tahoe-dev at tahoe-lafs.org>
Subject: Uploading files with http put

I am writing a Tahoe-LAFS plugin for Nextcloud and close to having it completed. The plugin puts Tahoe-LAFS on even footing with other cloud storage systems that have been integrated with Nextcloud such as: Amazon S3 and Swift.

The problem I am having is I need to support asynchronous uploading of local files to Tahoe-LAFS. I have not found a way to do this using the “tahoe cp” command. However, if I use an http put to perform the upload then I can support asynchronous uploads in Nextcloud.

I am using the “tahoe” command from the plugin so I can use any tahoe command that completes in a relatively short period of time. Is there a simple way for me to create a placeholder tahoe URI that I can use to perform an http put with to perform an upload?

Thanks in advance,

                Bruce T

_______________________________________________
tahoe-dev mailing list
tahoe-dev at tahoe-lafs.org
https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20191126/308d6386/attachment-0001.html>