Are you guys doing these as Hangouts-on-Air? If so, where's the YouTube channel? If not, please do!<div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Dec 7, 2012 at 3:45 PM, Paul Rabahy <span dir="ltr"><<a href="mailto:prabahy@gmail.com" target="_blank">prabahy@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font face="courier new, monospace">First off, let me say that I enjoyed listening in to the hangout even though I couldn't participate. For the last year or 2 I have been thinking about building a distributed, untrusted storage system. When I found Tahoe-lafs, I was ecstatic that it had already implemented about 90% of the ideas that I had thought of and that it sounds like the remaining 10% are being worked on.</font><div>
<font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">I have some comments on Zooko's <span style="white-space:pre-wrap">Proof-of-Retrievability paper.</span></font></div>
<div><span style="white-space:pre-wrap"><font face="courier new, monospace">1. Great job writing this. It was very easy to read and get up to speed without having to read 10 other whitepapers first to understand the basics. (I have some background in cryptography/secure computing from college)</font></span></div>
<div><span style="white-space:pre-wrap"><font face="courier new, monospace"><br></font></span></div><div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">2. I completely agree with the 3 levels of bad behavior (Greed, Malice, and Adaptive Malice). In addition, I believe there should be a fourth level which I will call "Accidental Greed". In this case, the sever stores the data, responds to all requests properly, but one day fails (either a POR or GetData request) for some unknown reason. This server will acknowledge their mistake and attempt to reverse it once they are notified (restore backups, patch bug, etc.).</span></font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">2a. For POR, Zooko nailed this. We don't have to care about "Accidental Greed" at the protocol level because if we cover our-self for Malice or Adaptive Malice we already have a solution.</span></font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap"><br></span></font></div><div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">3. I am convinced that to prevent Malice or Adaptive Malice, there cannot be a difference between running a POR or GetData. If there is either type of attacker could respond correctly to the POR but incorrectly to the GetData.</span></font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">3a. For my use cases I feel that having POR and GetData can have different traffic patterns and not affect my experience as a customer. I realize that will introduce a gap in the protocol so that Adaptive Malice could defeat the POR. I would like for POR to cover me 90% of the time, but occasionally I will actually download a file and will be able to catch the Adaptive Malice server at that point. (This might seem like a contradiction, but Tahoe already has the enormously powerful feature of erasure coding to protect me from an occasional malicious server even if POR fails.)</span></font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap"><br></span></font></div><div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">4. Unfortunately Zooko lost me in Part 2b and 3. </span></font><font face="courier new, monospace">I understand the trade-off between "(a) reduced performance for downloads, and/or (b) increased bandwidth usage for verification", but I was never able to understand how Tahoe is supposed to be convinced that a share is retrievable without even contacting the server containing the share.</font></div>
<div><font face="courier new, monospace">4a. Several times during the hangout, it was mentioned that increasing N and K would help POR to work better. I don't follow that argument. I agree that setting N higher increases the retrievability of a file (because it can withstand more malicious servers), but I don't see how increasing either of these will help me single out the malicious server.</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">5. I need to do more reading on the current Tahoe verify system, but I don't understand how Tahoe can verify a file using B Bandwidth where B is less than F Filesize.</font></div>
<div><font face="courier new, monospace">5a. Using the Tahoe defaults (K = 3, N = 10) and assuming that F = 1(MB) it will take 3.33(MB) to store all ten shares. Each share will take .333(MB) to store. To verify the file, wouldn't you have to retrieve at least K shares therefor B would equal .333(MB) * 3(K) = 1MB(F). To me, it seems we didn't save any bandwidth.</font></div>
<div><font face="courier new, monospace">5b. (Ah, just thought of this as I was writing). Does Tahoe maintain some sort of tree/share based hash so that it can verify individual shares or parts of a share without verifying the entire file? If so, I can see the bandwidth savings.</font></div>
<div><br></div><div><font face="courier new, monospace">6. I agree that TOR/distributed verification could help in the case of an Adaptive Malicious server, but until I have a clearer understanding of my points 5 and 6, I'm not sure if this description of POR will be have a benefit for my use case.</font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap"><br></span></font></div><div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">Hopefully these points make sense. Let me know if I made anything confusing.</span></font></div>
<div><font color="#000000" face="courier new, monospace"><span style="white-space:pre-wrap">PRabahy</span></font></div><div class="HOEnZb"><div class="h5">
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Dec 6, 2012 at 3:42 PM, Zooko Wilcox-O'Hearn <span dir="ltr"><<a href="mailto:zooko@zooko.com" target="_blank">zooko@zooko.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In attendance: Brian, David-Sarah, Zooko (scribe), Andrew, PRabahy (silent)<br>
<br>
The meeting started about 10 minutes late and ran more than 30 minutes<br>
past its scheduled stop-time. (Because we were too engaged to stop at<br>
the stop-time since we were sorting out the question of whether<br>
Zooko's "Strong Proof-of-Retrievability" concept was inherently as<br>
inefficient as simply downloading the whole file.)<br>
<br>
Caveat Lector! I might have forgotten some stuff. I haven't taken the<br>
time to add explanations for most of what follows. My own biases shine<br>
through willy nilly.<br>
<br>
<br>
* The LAFS-PoR.rst text file was cleverly hidden behind an obstacle course.<br>
<br>
* 'Ephemeral Elliptic Curve Diffie-Hellman‽ My friend Zooko excels at<br>
redefining "What 'everyone' or what 'no-one' uses."'<br>
<br>
* leasedb+cloud-backend<br>
* LeastAuthority.com has at long last delivered Milestone 3 to<br>
DARPA. Milestone 1 was a design document Milestone 2 was Cloud/S3<br>
backend, and Milestone 3 was leasedb.<br>
* <a href="https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1818" target="_blank">https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1818</a> /<br>
<a href="https://github.com/davidsarah/tahoe-lafs/tree/1818-leasedb" target="_blank">https://github.com/davidsarah/tahoe-lafs/tree/1818-leasedb</a> is the<br>
implementation of leasedb against trunk (disk backend)<br>
* <a href="https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1819" target="_blank">https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1819</a> /<br>
<a href="https://github.com/davidsarah/tahoe-lafs/tree/1819-cloud-merge" target="_blank">https://github.com/davidsarah/tahoe-lafs/tree/1819-cloud-merge</a> is the<br>
merge of that with the cloud backend<br>
* The 1819-cloud-merge branch passes all unit tests, and passes<br>
manual testing by David-Sarah. It is currently being evaluated on<br>
behalf of DARPA by their contractors, BITSYS.<br>
* next steps:<br>
* Keep 1818-leasedb and 1819-cloud-merge out of Tahoe-LAFS v1.10.<br>
* Let Brian review them.<br>
* David-Sarah is still re-recording the patch series for 1819-cloud-merge.<br>
* Zooko is still code-reviewing the patches.<br>
* Check for the transition experience — what happens the first<br>
time you upgrade, for example.<br>
* There is at least one incomplete detail about transition:<br>
starter leases don't get added (there isn't a ticket for this — we<br>
should open one).<br>
* Zooko and David-Sarah want to implement #1834 and related<br>
tickets — not necessarily before we land it on trunk, but before we<br>
release 1.11. Or we could do it on the branch before we land it on<br>
trunk.<br>
<br>
* Tahoe-LAFS v1.10<br>
* Let's package up what we have currently on trunk (plus, Zooko<br>
added to these notes after the meeting, possibly a few other good<br>
patches that are basically already done, are very non-disruptive —<br>
such as documentation-only patches — and/or have forward-compatibility<br>
implications, such as #1240, #1802, #1789, #1477, #901, #1539, #1643,<br>
#1842, and #1679).<br>
* Everyone review pending tickets!<br>
<a href="https://tahoe-lafs.org/trac/tahoe-lafs/milestone/1.10.0" target="_blank">https://tahoe-lafs.org/trac/tahoe-lafs/milestone/1.10.0</a><br>
* The next Weekly Dev Hangout will be about Tahoe-LAFS v1.10<br>
* goal: get trunk to meet our desires for Tahoe-LAFS v1.10, release<br>
from trunk<br>
* Brian wants to fix #1767, which has forward-compatibility implications.<br>
<br>
* tarcieri's new HTML<br>
* not for 1.10<br>
* It changes only the front page and so the other pages are<br>
inconsistent with the new front page.<br>
* But commit it to a branch ASAP and demonstrate to tarcieri that<br>
we're serious about merging it to trunk as soon as it is complete.<br>
<br>
* Proof-of-Retrievability<br>
* Zooko has written a rough draft of a tahoe-dev post/science<br>
paper, arguing that real "Strong" Proof-of-Retrievability is possible,<br>
that the current exemplars in the crypto literature fail to provide<br>
Strong Proofs-of-Retrievability, and that Tahoe-LAFS combined with Tor<br>
would make a nice basis on which to build a Strong<br>
Proof-of-Retrievability, and that if it did, it would be a practical<br>
censorship-resistance tool.<br>
* Brian posed some good challenges in practical terms about the<br>
performance and bandwidth costs.<br>
* The key difference that makes this new concept of<br>
Proof-of-Retrievability different and better than previous attempts is<br>
that it uses multiple storage servers (which are hopefully not<br>
colluding with one another), and erasure-coding in order to keep total<br>
upload and storage costs fixed even while scaling a single file,<br>
horizontally, to a large number of storage servers.<br>
* That's also the key to answering Brian's challenge — that sort of<br>
spreading across storage servers alllows one to gain verification<br>
assurance — *even* against Adaptive Malicious Storage Servers — at a<br>
fraction of the aggregate bandwidth cost of a full download. If there<br>
were only a single storage server then Juels-2009 and<br>
Brian-in-this-meeting would be right that no efficient Strong PoR is<br>
possible.<br>
* Next steps: Zooko needs to rewrite the second half of the current<br>
document to emphasize these insights gained from this meeting and to<br>
streamline it. Several experts have volunteered to review it already.<br>
Then: post it to tahoe-dev?<br>
* David-Sarah has some idea that Brian and Zooko don't quite get<br>
about improving the quantitative advantage to the defender by<br>
increasing erasure coding parameters and storing multiple shares per<br>
server.<br>
* Let's get drunk and argue about whether God can see into the future.<br>
_______________________________________________<br>
tahoe-dev mailing list<br>
<a href="mailto:tahoe-dev@tahoe-lafs.org" target="_blank">tahoe-dev@tahoe-lafs.org</a><br>
<a href="https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev" target="_blank">https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev</a><br>
</blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
tahoe-dev mailing list<br>
<a href="mailto:tahoe-dev@tahoe-lafs.org">tahoe-dev@tahoe-lafs.org</a><br>
<a href="https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev" target="_blank">https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Shawn<br>
</div>