show sizes in unambiguous way that doesn't get mistaken for different units

[USSJoin]

When setting up a storage node, it took me a long time to figure out why the storage wasn't respecting my set 15GB of reserved space on the drive. I finally realized that *like* hard drive manufacturers but *unlike* the rest of the planet, Tahoe is counting size in base-10, not base-2-- so kilobytes are 1000 bytes, not 1024, and so on. This leads to reports from Tahoe being *dramatically* different than, say, df -h, and thus creates confusion.

[zooko]

A good way out of this mess is to spell it out explicitly -- write "2³⁰" or "10⁹" or "one billion".

See also: ​http://en.wikipedia.org/wiki/Binary_prefix

[zooko]

Hitherto I believe we've been using "GiB" to mean 2³⁰ (per ​http://en.wikipedia.org/wiki/Binary_prefix ) and we may have sometimes been using "GB" to mean 10⁹. That latter usage, while technically correct, and accurate and meaningful to the vast majority of users (who do base-10 arithmetic in their heads but not base-2 arithmetic), is confusing to hackers like USSJoin. So we should avoid it, possibly by replacing uses of "GB" with "10⁹".

[USSJoin]

Alternately, you could simply use GiB consistently everywhere (for instance, on the storage information page) and do the units *as* GiB, not 10^{9. That way people don't have to go "what's a 10}9?" when looking at information pages.

[zooko]

We generally prefer base-10 arithmetic, because it is easier for users to use. For example, if you ask my mom how many 20-byte things she can store in a 2-terabyte bucket, she'll probably ask "What's a terabyte?", and if you tell her it is a trillion bytes, she'll say "Then I can store 100 billion of them.". If instead you tell her that it is 2⁴⁰ bytes, then she'll either have to get out a calculator or she'll just give up. In fact, I strongly suspect that a similar problem applies to computer hackers as well as to moms. Quick, how many 20-byte things can you store in a bucket of size 2⁴¹ bytes (two ​TiB)? I think it will take you longer to answer that question that it would take my mom to answer the base-10 variant of the question. If you answered "about a hundred billion of them" then your answer was 10% off! Back when our buckets were on the order of thousands of elements in size, the approximation of 2¹⁰≅10³ was only 2.5% off. The approximation of 2²⁰≅10⁶ is 5% off, 2³⁰≅10⁹ is 7.5% off, and 2⁴⁰≅10¹² is 10% off. By the way, Apple products now report file sizes and filesystem spaces in base-10: ​http://support.apple.com/kb/TS2419

[davidsarah]

We're with USSJoin on this. Resist the hard disk manufacturers' conspiracy.

Replying to zooko:

We generally prefer base-10 arithmetic, because it is easier for users to use. For example, if you ask my mom how many 20-byte things she can store in a 2-terabyte bucket, she'll probably ask "What's a terabyte?", and if you tell her it is a trillion bytes, she'll say "Then I can store 100 billion of them."

This is not a convincing argument, since you're more likely to need to know how many 1 MiB files, say, can be stored in a terabyte. (BTW, your mom's estimate would be wildly wrong for 20-byte files due to overhead.)

What mattered was that there was a consistent convention. Since most uses of "GB" (for example) still mean 2^{30 bytes, Tahoe is going in the wrong direction to reduce confusion.}

[kmarkley86]

People have strong reasons for strong preferences on both sides. How about making this configurable, so then we can fight about the default instead of forcing one style on everyone? (Also, base-2 sizes should be the default.)

[warner]

/me runs into the room waving his hands madly like a muppet. nooo!

Please don't contribute to the confusion by printing "x GB" and silently using it to mean 2³⁰. The entire non-computer world, the SI, and every dictionary on the planet knows that the metric G suffix means giga means 10⁹. And while I find "GiB" pretty funny-looking, it is an unambiguous, learnable, and eventually-straightforward term that clearly means 2³⁰. Let's not conflate the two. Sure, this helps the hard-drive manufacturers, but it's a terminology bugfix, not a conspiracy :).

I'm -1 on having a config option for pretending GB=2³⁰: someone looking at the web page (and not at the config file) would be unable to learn the truth.

In places where we have evidence that people want both sorts of values, we should give them both sorts of values. For example, on the "Storage Server Status" page, we currently show abbreviated GB (10⁹) and unabbreviated number-of-bytes:

Total disk space:       319.73 GB       (319728959488)
Disk space used:        - 311.44 GB     (311444250624)

My hope was that the "319728959488" would look enough like the "319.73" to cue the reader into remembering that GB means 10⁹, but the original poster's experience suggests this failed. Some other options for that display:

Total disk space:       319.73 GB       (319728959488) (297.77 GiB)

Total disk space:       319.73 GB(10^9) (319728959488)

Total disk space:       319.73 GB(10^9) (319728959488) 297.77 GiB(2^30)

Since this is a web page, we could also have a popup over the "319.73 GB" line that displays a number of other formats, not unlike we recently added a popup to Foolscap's log-web-viewer display to show timestamps in alternate formats (UTC/local/short/long):

319728959488
319.73 GB (10^9)
297.77 GiB (2^30)

I'm -0 on having a config option that makes these pages display GiB instead of GB, as long as it never ever tries to pretend that GB is 2³⁰, and that there continues to be a full-number-of-bytes display so that someone looking at the abbreviation has a chance to figure out what it means and become confident in our consistent use of terms.

This is not a convincing argument, since you're more likely to need to know how many 1 MiB files, say, can be stored in a terabyte. (BTW, your mom's estimate would be wildly wrong for 20-byte files due to overhead.)

Huh? Except for programmer-driven test cases, I don't think there's any particular quantization on filesizes. I'm not counting 1 MB or 1 MiB files, I'm counting how many digital pictures I can stuff onto a disk, and they're all sorts of random sizes. The only real quantization I can think of would be the chapters on a ripped DVD image (according to wikipedia these are usually 1 GiB in size), but I really don't think "how many non-terminal DVD VOB files can I fit on this disk" is a common question.

Hitherto I believe we've been using "GiB" to mean 2³⁰ (per ​http://en.wikipedia.org/wiki/Binary_prefix ) and we may have sometimes been using "GB" to mean 10⁹.

We're always using GB to mean 10⁹ and GiB to mean 2³⁰. I fix the code if I discover it doing otherwise.

[zooko]

This is what people call ​a bike shed. The theory goes that few people are willing to contribute their opinions about designing nuclear power plants, because that is very complex and requires high expertise, but many people are willing to contribute their opinions about designing a bike shed, because it is simple enough that they can see how they would like it to be.

(Aside: I don't really like that metaphor of a "bike shed" because it belittles the concerns of the contributors. I actually agree with USSJoin, davidsarah, and kmarkley86 that user interface issues are important, including this one. Don't forget that the original post by USSJoin explained how he actually lost some of his time due to confusion. Wasting user time is not okay! Also, a design being simple and easy to understand doesn't mean that it doesn't matter how it is done!)

However, this issue has now distracted both David-Sarah and Brian from ​building nuclear power plants. Let's put a stop to the discussion. Our policy will be to express numbers in units that are as unambiguous as possible so that a user who assumes that "GB" means 2²⁰ and a user who assume that "GB" means 10⁹ will both have a minimal chance of wasting their time with confusion. Specifically, the suggestions that Brian made in comment:7 about redundantly listing the same value in different units would probably help.

That's the main idea -- to make the user interface sufficiently clear (even at the cost of redundancy) that nobody wastes their time mistaking the units. I believe this policy will ​satisfice.

We will continue to use KiB to mean 10³, MiB to mean 10⁶, GiB to mean 10⁹, TiB to mean 10¹² etc. as per ​http://en.wikipedia.org/wiki/Binary_prefix , and never use KB to mean 2¹⁰ etc.. However, as per the main idea, above, we will probably try to reduce the use of KB at all in favor of less ambiguous designations.

[zooko]

Replying to warner:

In places where we have evidence that people want both sorts of values, we should give them both sorts of values. For example, on the "Storage Server Status" page, we currently show abbreviated GB (10⁹) and unabbreviated number-of-bytes:

Total disk space:       319.73 GB       (319728959488)
Disk space used:        - 311.44 GB     (311444250624)

My hope was that the "319728959488" would look enough like the "319.73" to cue the reader into remembering that GB means 10⁹, but the original poster's experience suggests this failed.

You know what? This might have worked if the bytes display had included commas, like this:

Total disk space:       319.73 GB       (319,728,959,488)
Disk space used:        - 311.44 GB     (311,444,250,624)

I don't know about USSJoin, but for me, my eyeballs just slide right off of "319728959488" after the first couple of digits. Proposed action items to make this ticket closable:

• change the Summary to "show sizes in unambiguous way that doesn't get mistaken for different units"
• add commas to outputs which are in terms of bytes
• ask people for feedback on whether this is now sufficiently clear to them
• close the ticket as fixed

[warner]

I like the commas idea: my eyeballs slide off long numbers too. My hesitation is that every once in a rare while, I cut-and-paste a number like that into a calculator or python repl, and the commas would mess that up. But I think readability trumps cut-and-pasteability. So +1 on the commas.

You might also add units: (319,728,959,488 bytes). But maybe not.

Oh, you know, there might possibly be a CSS styling thing that lets you tell the system that this is a number, and that it ought to add comma-like things according to the current locale (since they'd be periods in europe). I have a hazy memory that suggests doing this would also retain cut-and-pasteability, because the commas/periods would be purely visual: cut/paste would still get the original non-comma-ified number. Does this ring any bells for anyone, or am I completely imagining it?

[ScottD]

The only way I know to pull off locale-formatted numbers is to use a span with a CSS-class and use javascript to read those elements using parseInt()/parseFloat() and replacing them with toLocaleString(). The locale will get picked up from whatever browser is being used and degrades nicely to plain numbers if javascript is unavailable.

[davidsarah]

Replying to ScottD:

The only way I know to pull off locale-formatted numbers is to use a span with a CSS-class and use javascript to read those elements using parseInt()/parseFloat() and replacing them with toLocaleString(). The locale will get picked up from whatever browser is being used and degrades nicely to plain numbers if javascript is unavailable.

That's not worth the complexity IMHO. Separating the digit groups with spaces, e.g.

Total disk space:       319.73 GB       (319 728 959 488 bytes)
Disk space used:        - 311.44 GB     (311 444 250 624 bytes)

is understood internationally, and personally I think it's more readable. In HTML,   (NARROW NO-BREAK SPACE) might be better.

[Zancas]

Replying to zooko:

This is what people call ​a bike shed. The theory goes that few people are willing to contribute their opinions about designing nuclear power plants, because that is very complex and requires high expertise, but many people are willing to contribute their opinions about designing a bike shed, because it is simple enough that they can see how they would like it to be.

(Aside: I don't really like that metaphor of a "bike shed" because it belittles the concerns of the contributors. I actually agree with USSJoin, davidsarah, and kmarkley86 that user interface issues are important, including this one. Don't forget that the original post by USSJoin explained how he actually lost some of his time due to confusion. Wasting user time is not okay! Also, a design being simple and easy to understand doesn't mean that it doesn't matter how it is done!)

However, this issue has now distracted both David-Sarah and Brian from ​building nuclear power plants. Let's put a stop to the discussion. Our policy will be to express numbers in units that are as unambiguous as possible so that a user who assumes that "GB" means 2²⁰ and a user who assume that "GB" means 10⁹ will both have a minimal chance of wasting their time with confusion. Specifically, the suggestions that Brian made in comment:7 about redundantly listing the same value in different units would probably help.

That's the main idea -- to make the user interface sufficiently clear (even at the cost of redundancy) that nobody wastes their time mistaking the units. I believe this policy will ​satisfice.

We will continue to use KiB to mean 10³, MiB to mean 10⁶, GiB to mean 10⁹, TiB to mean 10¹² etc. as per ​http://en.wikipedia.org/wiki/Binary_prefix , and never use KB to mean 2¹⁰ etc.. However, as per the main idea, above, we will probably try to reduce the use of KB at all in favor of less ambiguous designations.

Oops.... isn't it KiB means 2¹⁰... et cetera? If I Understand Correctly the infixed "i" means base 2, and exponent increments of 10.

[zooko]

Replying to Zancas:

We will continue to use KiB to mean 10³, MiB to mean 10⁶, GiB to mean 10⁹, TiB to mean 10¹² etc. as per ​http://en.wikipedia.org/wiki/Binary_prefix , and never use KB to mean 2¹⁰ etc.. However, as per the main idea, above, we will probably try to reduce the use of KB at all in favor of less ambiguous designations.

Oops.... isn't it KiB means 2¹⁰... et cetera? If I Understand Correctly the infixed "i" means base 2, and exponent increments of 10.

Argh! How did I screw that up‽ Thanks, Zancas, for noticing. Everyone reading this: disregard what I wrote and just believe that we're going to do what ​http://en.wikipedia.org/wiki/Binary_prefix says about how to spell the base-2 things. Also, as previously mentioned on this ticket, spelling out numbers with commas in place is unambiguous and is the standard format for integers in Internet English writing.