[tahoe-lafs-trac-stream] [Tahoe-LAFS] #3956: Uploading a 20MB mutable fails in HTTP, but succeeds in Foolscap

[Tahoe-LAFS]

#3956: Uploading a 20MB mutable fails in HTTP, but succeeds in Foolscap
--------------------------+-----------------------------------
     Reporter:  itamarst  |      Owner:
         Type:  defect    |     Status:  new
     Priority:  normal    |  Milestone:  HTTP Storage Protocol
    Component:  unknown   |    Version:  n/a
   Resolution:            |   Keywords:
Launchpad Bug:            |
--------------------------+-----------------------------------
Description changed by itamarst:

Old description:

> To reproduce: use benchmarks/upload_download.py from https://github.com
> /tahoe-lafs/tahoe-lafs/tree/3952-benchmarks, with a 20MB file. It'll run
> with Foolscap, but in HTTP:
>
> ```
> >               result = await
> self.clients[0].create_mutable_file(MutableData(DATA))
> E               allmydata.mutable.common.NotEnoughServersError: ("Publish
> ran out of good servers, last failure was: [Failure instance: Traceback:
> <class 'allmydata.storage.http_client.ClientException'>: (413,
> b'')\n/home/itamarst/devel/tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:734:errback\n/home/itamarst/devel
> /tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:797:_startRunCallbacks\n/home/itamarst/devel
> /tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:891:_runCallbacks\n/home/itamarst/devel
> /tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:1791:gotResult\n--- <exception caught
> here> ---\n/home/itamarst/devel/tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:1692:_inlineCallbacks\n/home/itamarst/devel
> /tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/python/failure.py:518:throwExceptionIntoGenerator\n/home/itamarst/devel
> /tahoe-
> lafs/src/allmydata/storage_client.py:1445:slot_testv_and_readv_and_writev\n/home/itamarst/devel
> /tahoe-lafs/venv/lib/python3.10/site-
> packages/twisted/internet/defer.py:1696:_inlineCallbacks\n/home/itamarst/devel
> /tahoe-
> lafs/src/allmydata/storage/http_client.py:868:read_test_write_chunks\n]",
> None)
> ```
>
> That is, we're hitting a hard-coded size limit in the client code which
> presumably shouldn't be enforced given this works for Foolscap?

New description:

 To reproduce: use benchmarks/upload_download.py from https://github.com
 /tahoe-lafs/tahoe-lafs/tree/3952-benchmarks, with a 20MB file. It'll run
 with Foolscap, but in HTTP:

 {{{
 >               result = await
 self.clients[0].create_mutable_file(MutableData(DATA))
 E               allmydata.mutable.common.NotEnoughServersError: ("Publish
 ran out of good servers, last failure was: [Failure instance: Traceback:
 <class 'allmydata.storage.http_client.ClientException'>: (413,
 b'')\n/home/itamarst/devel/tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:734:errback\n/home/itamarst/devel
 /tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:797:_startRunCallbacks\n/home/itamarst/devel
 /tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:891:_runCallbacks\n/home/itamarst/devel
 /tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:1791:gotResult\n--- <exception caught
 here> ---\n/home/itamarst/devel/tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:1692:_inlineCallbacks\n/home/itamarst/devel
 /tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/python/failure.py:518:throwExceptionIntoGenerator\n/home/itamarst/devel
 /tahoe-
 lafs/src/allmydata/storage_client.py:1445:slot_testv_and_readv_and_writev\n/home/itamarst/devel
 /tahoe-lafs/venv/lib/python3.10/site-
 packages/twisted/internet/defer.py:1696:_inlineCallbacks\n/home/itamarst/devel
 /tahoe-
 lafs/src/allmydata/storage/http_client.py:868:read_test_write_chunks\n]",
 None)
 }}}

 That is, we're hitting a hard-coded size limit in the client code which
 presumably shouldn't be enforced given this works for Foolscap?

--

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3956#comment:1>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage