Questions tagged [floating-point]
82 questions
32
votes
12 answers
Why do you need float/double?
I was watching http://www.joelonsoftware.com/items/2011/06/27.html and laughed at Jon Skeet joke about 0.3 not being 0.3. I personally never had problems with floats/decimals/doubles but then I remember I learned 6502 very early and never needed…
user2528
11
votes
2 answers
How to calculate min/max values of floating point numbers?
I'm trying to calculate the min/max, or the lowest to highest value range of a 48 bit floating point type MIL-STD-1750A (PDF) (WIKI).
Ex: How a double range is 1.7E +/- 308
I've looked around for equations, and am unsure if what I have found will…
CodeMonkey
- 245
7
votes
3 answers
When do rounding problems become a real problem? Is the least significant digit being one off really a big deal?
If I do .1 + .1 + .1 in Python, I get 0.30000000000000004. (I am not asking about Python in particular, and do not want Python specific answers.)
The only problem I can see with this is 0.30000000000000004 != 0.3, so I need to take care in how I…
Buttons840
- 1,866
4
votes
0 answers
Who first said, "If you think floating point is the answer, you don't understand the problem."
I had a C (programming language) teacher at the Lowell Institute at MIT who said this in the Spring of 1996. It's an overstatement, but still seems like a classic quote to me. I've had no luck searching for attribution over the years.
I don't want…
GlenPeterson
- 14,920
4
votes
0 answers
Rationale of IEEE754 direct rounding behavior near infinity
A preface: it's well known that IEEE754 defines five rounding modes (in 2008 edition terms, with my abbreviations):
rounding to nearest, ties to even (RNE) - the default mode for binary arithmetic;
rounding to nearest, ties away (RNA) - required…
Netch
- 1,520
3
votes
2 answers
Why does IEEE 754 Single Float have 8 bits for exponent but only allow 38 base ten Max Value
Why is it that the IEEE 754 standard provide 8 bits for the exponent but puts a range maximum of 38 on the largest acceptable exponent?
Seems kind of ridiculous to have the data storage there but not allow it to actually be used.
I wanted to pack…
JamesHoux
- 254
2
votes
5 answers
Will two equal floating point compare equal after the exact same set or operations?
Say if i have two floating point numbers
double a = 0.1
double b = 0.1
Then a, and b, after going through the EXACT SAME set of complicated arithmetic operations, on the same computer, then compare. will a==b always return true in this case?
Yituo
- 123
1
vote
1 answer
what *should* you get when you add minus-zero to plus-zero?
this Wikipedia article did not address this contingency and i am having trouble finding a definitive IEEE-754 document that addresses this.
robert bristow-johnson
- 1,190
0
votes
6 answers
How to handle precision problems with the correct terminology
I'm writing a math library and I've encountered trouble with floating point imprecision. I want to know if there are any standard workarounds that are used often. I have doubts about my current method and my lack of knowledge and terminology when it…
Scene
- 109
-1
votes
1 answer
Is this statement correct for floating point numbers?
The decimal point can "float" to accommodate larger numbers while
staying in 32-bits which is why float is considered "in-accurate"
Is this an accurate statement? I want to know if I understand floating point numbers.
Rhys
- 29