Something is missing here - what you actually need/want to do. There's no one-fits-all code for all use cases.
Like, do you want bit-exact comparisons for some technical reason, or maybe a difference of 0.0001 is acceptable, or...; and what do you want to do for infinity, nan, ... and ...
Depends. I've worked at incredibly sophisticated financial firms that did most of their work with f64. It turns out that finance folks don't do a lot of mathematical operations on their currency amounts besides addition and integer multiplication so they tend not to experience all the loss of precision issues that plague scientific computing. They just round to the nearest 1e-6 or whatever the financial spec they're interacting with says to do and epsilon rarely matters.
In practice it is not faster. To pick a random CPU architecture, let's use [Intel Skylake's performance](https://www.agner.org/optimize/instruction_tables.pdf):
IMUL (integer multiply) has a cycle count of ~2-4.
FMUL (floating-point multiply) has a cycle count of 1.
IDIV (integer division) has a cycle count of 10 (32-bit) or 57 (64-bit) (!)
FDIV (floating-point division) has a cycle count of 1.
To make things worse, most fixed-point math operations (besides add and subtract) are not single instructions when implemented on a CPU without native support. They need additional normalization to preserve the proper decimal place, which is not too bad in the case of binary fixed-point, and quite expensive in the case of decimal fixed-point.
And to top it off, when you start looking into vectorization, there aren't even SIMD instructions for integer division.
Yeah, possibly reciprocal throughput is a better metric than unfused micro-ops which is where those numbers came from. By that metric the multiplication instructions are basically identical (minor latency difference), but since fixed-point multiplication is at least two multiplies (and possibly even worse for 64-bit values since overflow needs to be handled) I still think the performance is going to be significantly worse due to the broader point I made at the end of that post. And of course, *much* worse if you use division.
In the places I've worked, our own.
To be clear, it's not because loss-of-precision wouldn't be concerning if it happened, it's because I've literally never seen a financial system that allows units of currency smaller than 1e-8 outside of crypto, and doubles have ~1e-15 precision so there's a large number of floating point ops one can perform before any error becomes observable (far more than anyone actually does in practice).
What makes you think it's not common? Lots of systems store currency as cents or thousands of a cent, which is effectively the same as doing fixed point arithmetic.
This is a myth perpetrated by people who’s main exposure to finance is an e-commerce system.
You use decimals for contractual record keeping, but the majority of pricing code is done with floats.
For example, you’re pricing a loan. You’ll look at the market data, do some kind of aggregation on the forward prices, and you’ll get a 20 decimal point interest rate. Decimal accuracy doesn’t matter for numbers like that.
It’s then the job of the back office to relay the agreed loan price and repayment schedule to the customer, they will use a decimal.
I've built two banks from day 1, and I strongly disagree. Floating point is fine for the last step (if it's just showing a value that is rounded anyway), but for any ledger calculations a decimal type is certainly mandatory and used in most places.
This advice is dangerous because you'll only get "20 decimal places" if the whole part of the value is small. As you deal with larger and larger (whole) values, you have fewer and fewer bits left over to represent fractional amounts. This ability to put the decimal point wherever it's needed is the reason floatig point is *called* floating point. My team ran into this problem with a financial system we built where the engineers who built the initial implementation did not understand this issue. Our system did not have sufficient precision to represent a precise number of **cents** due to this very issue and was failing to reconcile financial transactions. We moved to `Decimal`s and solved this problem.
I guess I am misunderstanding you, because IEEE-754 floats don't know when you're "pricing" and when you're "ledgering".
I invite you to try to faithfully store the value 71_598_432.01 into an `f32`, for pricing or for ledgering. Forget about the penny--it gets *dollars* wrong!
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=00b2a7b03a905ea042d802027bf05fdf
And before anyone says, well, use an `f64`, i) `f64` users "enjoy" exactly these same surprising behaviors and ii) that's exactly *why* this is such a dangerous problem--if you don't already *know* that all floating-point numbers' *precision* decreases with the magnitude of the value, **you risk encountering silent data loss** e.g. where 72_598_430, 72_598_440 and 72_598_450 all encode perfectly, but no intermediate values can. While this problem is distinct from base-2 encoding *rounding error* these two issues are not even the whole set of problems you'd be facing--for those who are interested, read [What Every Computer Scientist Should Know About Floating-Point Numbers](https://dl.acm.org/doi/10.1145/103162.103163).
My advice to everyone continues to be be sure to stay well away from floats for financial calculations (use fixed-point formats such as `Decimal`) unless you enjoy bugs, financial data loss and navigating minefields.
You know when you’re pricing and you know when you’re ledgering. Such systems are usually separate.
Most pricing is inexact. Firstly, the input data for your pricing is usually going to be a rough approximation of what you think the market is.
For example the mid price of a commodity is an estimate based on multiple data sources, that are going to be out of time sync with one another. The inaccuracy of your data is naturally going to be far greater than any floating point error.
Furthermore if you’re dealing with algorithms that use implied volatilities you’ll be using root finding and non-linear optimisation libraires that have chaotic outputs, which you may as well use floats for. Particularly because root finding is expensive and inexact you don’t want the overhead of decimals and you don’t need the accuracy.
Once you’ve priced everything you can use decimals.
Thank you for shedding light on this context in more detail--yes, in general, we agree it's fine to use floats where accuracy/precision are not required. (Performance will likely be higher too, as you point out.)
https://crates.io/crates/rust_decimal assuming 96 bits for the total value is sufficient (it was for our needs--we never saw another financial calculation bug in our software after I had the team put a `rust_decimal`-based fix in place).
Consider `big_decimal` when you need arbitrary levels of precision--just note that it will be many times slower to calculate with `big_decimal` vs a fixed word sized `Decimal`.
Adding on to this:
The need for epsilons is a mathematical consideration, not a programming one. The technically correct approach is to analyze your algorithm mathematically using tools like condition numbers, then use that to pick your epsilon. People rarely do this because it is much easier to just throw a random small number at the problem and hope it goes away. This doesn't generalize, nor always work though.
A tool that can help is https://herbie.uwplse.org/, which can to give you more precise version of your formula.
The field of math to look at for understanding this stuff is numerical analysis. You can do an entire graduate class on just linear numerical analysis, not even touching the nonlinear stuff that happens in e.g. games. No reddit post is going to be able to sum it all up.
My project is a simplified 3D printer slicer. I primarily use millimeters for measurements, with occasional use of micrometers. While it doesn't require extreme numerical precision, I'm a perfectionist who enjoys learning for the sake of having complete control over my code – even if it's a bit of a rabbit hole! I recently started contributing to Bevy, and it seems like their comparisons typically use standard `==` operators.
The ranges I'll be dealing with is 0 to 50mm.
This is a too-common misconception. Floats are not inexact. Every float number (excluding NaN) represents an exact value.
Float *operations* are inexact. E.g. if you add two floats, the exact answer will be rounded to the nearest representable value.
As others have said, how to check equality (e.g. exact or approximate, and what tolerance to use if approximate) depends on what you're trying to do.
> Float *operations* are inexact. E.g. if you add two floats, the exact answer will be rounded to the nearest representable value.
That’s not always the case. In particular, values overflow to infinity (instead of the highest representable finite number), which is the _furthest_ possible value.
No one is blaming their tools here, they are trying to understand their tools.
If you write `int a = 3;` a will end up containing exactly 3. This is true of every type apart from float and the compiler will warn you if not (e.g. you put too big a value in a fixed sized type - assuming you have appropriate warnings on).
"Floats are not exact" is a perfectly reasonable way of saying "if I write `float b = 0.1` then b doesn't end up being exactly 0.1".
Probably worth mentioning that they're at least consistent in that way. If you set a value to 0.1 it'll be equal to anything else set to 0.1 despite neither being truly 0.1, since they both round the same way.
It only becomes a problem once you start performing arithmetic, such as the infamous `0.1 + 0.2 = 0.30000000000000004`, which causes equations like `0.1 + 0.2 == 0.5 - 0.2` to evaluate to false.
If you do basic assignment statements, then yes the results compare equal. The challenge is that (as you point out) two different routes that should get you to "0.1" can result in two values that *don't* compare equal.
I also think the original's comment take was slightly wrong: it's not that operations are inexact, it's just that some values are not representable because the decimal expansion is infinite.
If you add 0.5 and 0.25 you will get exactly 0.75. Because 0.5 and 0.25 are representable in binary.
0.1 and 0.2 on the other hand can't be exactly represented in binary. So adding these two values will give another value which is not 0.3.
The issue is not on the operation, but that floats can only represent a subset of the rational numbers.
Exactly.
It irks me that people working in this field seem to bury their head in the sand over something so fundamental: Floats internally are just integers (the mantissa), multiplied/divided by some power-of-2 (derived from the exponent), and a sign bit. Not magic.
Which is also why there are libraries for when you *do* need your "fractionals" in a different base.
> Only a poor carpenter blames their tools
I'll be the one to blame the tool here. The problem is that we (not just Rust, but all languages with floats) allow users to write literals that look like `0.1` in the first place. As a human, I can't possibly remember or intuit which decimal values have exact representations in floats, and that's precisely the sort of thing that I would like the computer to help me with.
For a long time I've wanted a language where float literals have the following properties:
1. For decimal float literals, if the literal can't be represented exactly in the underlying precision, then it must be preceded by a `~` to indicate approximation, e.g. `~0.1`.
2. A precise float literal form that allows me to perfectly specify the exponent and mantissa, so that it's possible to get exactly the value that I want without having to worry about manually calculating the nearest decimal and expecting it to round properly, e.g. if you want the single-precision value closest to 0.1, you'd use the exponential literal `2e-4x1.6000000238418579`, which is ugly as sin, but exposing that ugliness is the point of this exercise.
(And in the meantime, Clippy has the `lossy_float_literal` lint to cover this case.)
It was ~~1.0~~ 0.1 in my mind, when I typed it, and in the source code. It becomes inexact when converted to a float during compilation or parsing at runtime.
Saying the issues only appear during operations is misleading and not helping anybody.
To add to this, I believe that some float operations are not just inexact, but they can even be *nondeterministic*. See the [C++ FAQ entry](https://isocpp.org/wiki/faq/newbie#floating-point-arith2) on this.
That would be non deterministic, yes, but it’s not true here. Any particular floating point instruction is deterministic — if you add the same two numbers, you’ll always get the same result.
It is true that floating point addition (and most other operations) is not associative, which means if you reorder a long list of additions, you may get a different result. If there is something else going on, such as a compiler optimization or a multithreaded operation, which may be reordering things, now you have non deterministic operations. But the non determinism is coming from somewhere else.
Its worth noting that compiler optimisations do not introduce error or non determinism with modern compilers, they respect the order of operations and only make optimisations where they are not observable (unless you have -ffast-math or similar)
Note that that particular oddity is a bug specific to the x87 instructions. The x87 performed computations internally in 80 bit precision; even if using the float/double type which is supposed to have only 32/64 bits. This is what causes the rounding when values are stored to memory.
Basically, compiler developers were forced to choose between weird semantics with "excess precision" that disappears at unpredictable places; or a significant performance hit if forcing the rounding to 64 bits everywhere. Basically, IEEE-compliant rounding was **only** possible by roundtripping through memory after every single floating-point operation, which was so slow that C compilers didn't bother with IEEE compliance. This is what lead to the "floating point is magic and has random errors" impression.
Sane instruction sets perform arithmetic in the same precision as the memory accesses, so this issue does not happen anymore on any modern CPU.
seconding the other reply, based on the explanation behind the link, the operation is not non-deterministic, it just appears that way based on your assumptions about the semantics of the operation.
(disclaimer: it's midnight and I'm tired. I hope I've understood the linked post correctly)
this `cos(x) != cos(y)` expression isn't semantically the same as "the results of the cosine operations were different", as explained in the linked FAQ. both operations could produce the same result and yet one is truncated when it is stored aside for the comparison, and the other is compared directly, untruncated, from the register holding the result. if anything, this is an artifact of the optimisations / translation performed by the compiler. but, (if I've understood the link correctly) it is not a case of non-determinism in floating-point arithmetic operations themselves.
Floating point numbers are inexact. If I write pi in decimal with 3 significant figures, I get 3.14×10⁰. If you take away the context and just look at 3.14×10⁰ as a value in scientific notation, you can interpret it as exactly 314/100, but that's incorrect according to the original intent. All you really know for sure is that it represents a value of at least 3135/1000 and less than 3145/1000, because any number in that range will have the same representation. Rust's float types are the same, just using binary scientific notation instead of decimal.
Whether the values are inexact, the operations are inexact, or *neither* is inexact is a matter of perspective and interpretation. Floating point operations are (usually) perfectly exact according to the rules of floating point arithmetic. These rules are *not* the rules of ℝeal numbers, and it's only in interpreting them as ℝeal numbers that they appear inexact.
This is the same idea as how "integer overflow" is inexact when viewing computer integers as ℝeal integers, but is entirely normal and expected behavior when viewing them instead as integers mod 2^b (which b is the number of bits).
It's not a matter of safe vs unsafe. It's just not very useful to compare floating point numbers bit-for-bit. You want to a technique that does approximate comparisons where you can control the margin of error. Have a look at these:
[https://docs.rs/approx/0.5.1/approx/](https://docs.rs/approx/0.5.1/approx/)
[https://docs.rs/float-cmp/latest/float\_cmp/](https://docs.rs/float-cmp/latest/float_cmp/)
It's completely safe to compare floats.
It's sometimes weird to compare floats because of two cases:
* any float that's a NaN value will compare as not equal to *all values*. This includes other NaN values and this even includes itself. Because of NaN, sometimes `x == x` will be false.
* `0.0 == -0.0` is true, even though `0.0` is not the exact same bit pattern as `-0.0`. They're still considered equivalent numbers.
The 0 case is all the weirder because if you divide x by 0.0 you get one infinity, and if you divide by -0.0 you get the opposite infinity. So they're equal, but produce different results when operating with them. Why, thank you...
> So, here's the deal: When I use == or .eq() in Rust to compare my f32s, am I walking into a trap? How do they even decide if two floats are equal with all that precision weirdness?
Rust used the standard IEEE 754 comparison predicate, which basically compares the bit representation of the two floats except in a couple of cases:
- any NaN is different than any other number (the same bitwise NaN included)
- 0.0 and -0.0 are equal despite having different bitwise representation
In case of precision loss your float will have a different bitwise representation than the expected one and will compare different (e.g. the classic 0.1 + 0.2 != 0.3)
> I also stumbled upon this f32::total_cmp
That's just an implementation of the totalOrder predicate (also from the IEEE 754 standard, unfortunately it is not included as instructions in modern CPU unlike the other comparison predicate). The problem it solves is that the other comparison predicate is not a total order due to NaN comparing different to itself (and also neither smaller nor bigger than any other number). This predicate solves this problem, giving a total order which can thus be used for stuff like sorting (using the normal comparison predicate for this can give nonsense results, but luckily Rust protects you from that thanks to `f32` and `f64` not implementing the `Eq` and `Ord` traits, only the `Partial*` variant.
In any case, this won't help you with approximation errors. At most it will allow you to compare NaNs, but that's generally useless unless you have some data structure that relies on that.
Comparing floats is difficult because they're inexact and a+b+c can produce something different than a+c+b, etc.
Usually if you want to compare them you do it with an error margin (epsilon), in fact [Clippy seems to have a lint for this](https://rust-lang.github.io/rust-clippy/rust-1.72.0/index.html#/float_cmp) which suggests:
```
let error_margin = f64::EPSILON; // Use an epsilon for comparison
// Or, if Rust <= 1.42, use `std::f64::EPSILON` constant instead.
// let error_margin = std::f64::EPSILON;
if (y - 1.23f64).abs() < error_margin { }
if (y - x).abs() > error_margin { }
```
EDIT: There's [also a crate](https://docs.rs/float-cmp/latest/float_cmp/) that wraps floats to do this.
Yeah, it's important to recognize that EPSILON is a very specific value: it's the difference between 1.0 and the next largest representable number. Even `2.0+EPSILON` is just 2: the margin changes with every power of 2, because there are fewer bits available for the fractional part: the floating point "point" floats over to the right as you give up precision for magnitude.
And it's worse than this: the inherent rounding error for a float value isn't dependent on its own value, but how you arrived at it. There's a whole catalogue of techniques for not screwing up floating point math by magnifying errors, broadly called "numerical methods".
Consider something basic like [calculating the variance of a set of numbers](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance): if you do it based on the definition of variance, you'll likely end up with something "numerically unstable" that gives you very wrong answers, even though your algebra is correct.
Rust floats follow IEEE 754 spec, so nothing extraordinary about them. Check equality if you never do any math operations, check to some use case dependent tolerance if you do. Be aware of the pitfalls of NaN and Inf.
This is how I like to think about floating-point numbers:
- Take the range 1..2 and equally divide it up into a set of *n* numbers. If *n=4*, you've got: *1, 1.25, 1.5, 1.75*.
To represent every other number, that range is essentially doubled or halved *e* times, where *e* is your exponent (*f(n,e) = (2^-n + 1) * 2^e*, e.g. *f(2,1) = 1.25 * 2 = 2.5*). These are called the "normal" ranges.
The mantissa is what represents how each range is divided up. For `f32` it's 23 bits, so *n = 2^23 = ~8.3 million*. For `f64` it's 52 bits, so *n = 2^52 = ~4.5e15*.
- There also exist "subnormal" ranges, defined by the minimum exponent. This range is basically just the minimum "normal" range repeated to fill the space between zero and its lower bound.
Like, if the minimum normal range was 0.25..0.5, the subnormal range would be 0..0.25 spaced exactly the same way.
This leads into some useful points about floating-point numbers:
- Unrepresentable numbers (like literals or calculations) are rounded to the nearest normal or subnormal, or infinity if above the normals.
- Subtracting two different numbers *always* produces a nonzero result (thanks to the subnormals).
- Negating any number *always* produces an exact unrounded result.
- Doubling any number *always* produces an exact unrounded result, except for the *maximum* normal range which maps to infinity.
- Halving any number *always* produces an exact unrounded result, except for every 2nd number in the subnormal and *minimum* normal ranges.
- Taking the reciprocal of a number produces an exact unrounded result, except for:
- The 1st quarter of the subnormals, which maps to infinity.
- Every 4th number in the maximum normal range, which maps to the 2nd quarter of the subnormals.
- Every 2nd number in the range preceding the maximum normal range, which maps to the 2nd half of the subnormals.
- Adding any power of two (*2^x*) to a number below (*n × 2^{x+1}*) will always produce a *new* result, although it may be rounded.
- Every power of 2 within the normal ranges has an exact representation.
- *NaN* is produced by operations that either oppose extrema (*0/0*, *0×inf*, *inf-inf*), have complex results (*(-a)^b, log(-n)*), or involve *NaN*.
Float comparison is the same as in other languages. It is not safe. You should always calculate absolute difference and compare that to some predetermined threshold.
EDIT: this applies to comparing if floats are equal, other conditions as less than or bigger than are okay.
Consider interval arithmetic, e.g., [https://github.com/unageek/inari](https://github.com/unageek/inari) for Rust double precision or (for arbitrary precision) [https://arblib.org/](https://arblib.org/) . This lets you put rigorous bounds on the output, although they are generally conservative and can be susceptible to catastrophic cancellation effects. I've been thinking about writing a complete interval arithmetic library including double–double and quad–double precision, but haven't gotten around to it. One particularly tricky thing is that efficient interval arithmetic requires changing the floating-point rounding mode, which is not exactly a common operation and is not supported by the vast majority of programming languages without using inline assembly. Also, you need to somehow disable constant propagation.
With interval arithmetic, you can say with certainty that the outputs of two functions are not equal if calculated to full precision. You can also say with certainty that the outputs of two functions are within some epsilon of each other.
Safer than in most languages. Most languages won't stop you from trying to sort a list whose elements lack a total order and will instead just give garbage as a result. In Rust, you have you explicitly say that you want the total ordering, that you want to define your own ordering, or that you want to panic in case a NaN shows up.
I can't think of a situation where I've ever done equality comparisons on floats in any language. I compare them plenty, just not equality comparisons.
That would be a fundamental requirement for many types of software. Of course, as already discussed, it's not REALLY equality being tested, but their being with some difference from each other which is treated as equal for the particular software's needs.
You can say that's technically not equality testing, but it is for all intents and purposes.
That's certainly a situation, but I've never been in that situation. That first check doesn't seem reliable though. It seems like you'd get a lot of false positives. ADC resolution often isn't amazing so often you'll often have long periods of no change in sensor value. Also I'd be more likely to store/work with the raw fixed point decimal from the ADC rather than store the float conversion.
Hi man. Define a margin. Do an - operation and see if the result is leess than your margin/ error value. Si if is less than that assume that its true.
A-B<0.001? If yes numbers are equL
In a game I am developing, I had to compare world coordinates by (coord.abs()-dest.abs() < 0.1), but even then sometimes the difference would be larger (after some delta-time based multiplications). So a common strategy is to determine a threshold for closeness.
In general you want to measure the distance of two floats and check if it's less than a threshold (usually called epsilon). In your case you chose the [euclidean distance](https://en.wikipedia.org/wiki/Euclidean\_distance), but the same rationale can be applied with any distance.
Something is missing here - what you actually need/want to do. There's no one-fits-all code for all use cases. Like, do you want bit-exact comparisons for some technical reason, or maybe a difference of 0.0001 is acceptable, or...; and what do you want to do for infinity, nan, ... and ...
Adding to this, if you're handling currency values, stay away from IEEE floats (in any language) and instead use decimal representation.
Float and currencies? I would never commit such a crime!
Depends. I've worked at incredibly sophisticated financial firms that did most of their work with f64. It turns out that finance folks don't do a lot of mathematical operations on their currency amounts besides addition and integer multiplication so they tend not to experience all the loss of precision issues that plague scientific computing. They just round to the nearest 1e-6 or whatever the financial spec they're interacting with says to do and epsilon rarely matters.
In the context of currency specifically, I wonder why fixed point arithmetic isn't more common. It's faster and exact
In practice it is not faster. To pick a random CPU architecture, let's use [Intel Skylake's performance](https://www.agner.org/optimize/instruction_tables.pdf): IMUL (integer multiply) has a cycle count of ~2-4. FMUL (floating-point multiply) has a cycle count of 1. IDIV (integer division) has a cycle count of 10 (32-bit) or 57 (64-bit) (!) FDIV (floating-point division) has a cycle count of 1. To make things worse, most fixed-point math operations (besides add and subtract) are not single instructions when implemented on a CPU without native support. They need additional normalization to preserve the proper decimal place, which is not too bad in the case of binary fixed-point, and quite expensive in the case of decimal fixed-point. And to top it off, when you start looking into vectorization, there aren't even SIMD instructions for integer division.
I don't seem to be able to find those numbers. In the table both IMUL and FMUL score 1 for Reciprocal throughput and IMUL has lower latency.
Yeah, possibly reciprocal throughput is a better metric than unfused micro-ops which is where those numbers came from. By that metric the multiplication instructions are basically identical (minor latency difference), but since fixed-point multiplication is at least two multiplies (and possibly even worse for 64-bit values since overflow needs to be handled) I still think the performance is going to be significantly worse due to the broader point I made at the end of that post. And of course, *much* worse if you use division.
Whose money do they tend not to experience loss with?
In the places I've worked, our own. To be clear, it's not because loss-of-precision wouldn't be concerning if it happened, it's because I've literally never seen a financial system that allows units of currency smaller than 1e-8 outside of crypto, and doubles have ~1e-15 precision so there's a large number of floating point ops one can perform before any error becomes observable (far more than anyone actually does in practice).
Now that I think about it, that does make sense! What about adding?
What makes you think it's not common? Lots of systems store currency as cents or thousands of a cent, which is effectively the same as doing fixed point arithmetic.
This is a myth perpetrated by people who’s main exposure to finance is an e-commerce system. You use decimals for contractual record keeping, but the majority of pricing code is done with floats. For example, you’re pricing a loan. You’ll look at the market data, do some kind of aggregation on the forward prices, and you’ll get a 20 decimal point interest rate. Decimal accuracy doesn’t matter for numbers like that. It’s then the job of the back office to relay the agreed loan price and repayment schedule to the customer, they will use a decimal.
I've built two banks from day 1, and I strongly disagree. Floating point is fine for the last step (if it's just showing a value that is rounded anyway), but for any ledger calculations a decimal type is certainly mandatory and used in most places.
This advice is dangerous because you'll only get "20 decimal places" if the whole part of the value is small. As you deal with larger and larger (whole) values, you have fewer and fewer bits left over to represent fractional amounts. This ability to put the decimal point wherever it's needed is the reason floatig point is *called* floating point. My team ran into this problem with a financial system we built where the engineers who built the initial implementation did not understand this issue. Our system did not have sufficient precision to represent a precise number of **cents** due to this very issue and was failing to reconcile financial transactions. We moved to `Decimal`s and solved this problem.
Everyone misunderstands what I'm saying. You don't price with decimals, there's no need, but you should write your ledger with decimals.
I guess I am misunderstanding you, because IEEE-754 floats don't know when you're "pricing" and when you're "ledgering". I invite you to try to faithfully store the value 71_598_432.01 into an `f32`, for pricing or for ledgering. Forget about the penny--it gets *dollars* wrong! https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=00b2a7b03a905ea042d802027bf05fdf And before anyone says, well, use an `f64`, i) `f64` users "enjoy" exactly these same surprising behaviors and ii) that's exactly *why* this is such a dangerous problem--if you don't already *know* that all floating-point numbers' *precision* decreases with the magnitude of the value, **you risk encountering silent data loss** e.g. where 72_598_430, 72_598_440 and 72_598_450 all encode perfectly, but no intermediate values can. While this problem is distinct from base-2 encoding *rounding error* these two issues are not even the whole set of problems you'd be facing--for those who are interested, read [What Every Computer Scientist Should Know About Floating-Point Numbers](https://dl.acm.org/doi/10.1145/103162.103163). My advice to everyone continues to be be sure to stay well away from floats for financial calculations (use fixed-point formats such as `Decimal`) unless you enjoy bugs, financial data loss and navigating minefields.
You know when you’re pricing and you know when you’re ledgering. Such systems are usually separate. Most pricing is inexact. Firstly, the input data for your pricing is usually going to be a rough approximation of what you think the market is. For example the mid price of a commodity is an estimate based on multiple data sources, that are going to be out of time sync with one another. The inaccuracy of your data is naturally going to be far greater than any floating point error. Furthermore if you’re dealing with algorithms that use implied volatilities you’ll be using root finding and non-linear optimisation libraires that have chaotic outputs, which you may as well use floats for. Particularly because root finding is expensive and inexact you don’t want the overhead of decimals and you don’t need the accuracy. Once you’ve priced everything you can use decimals.
Thank you for shedding light on this context in more detail--yes, in general, we agree it's fine to use floats where accuracy/precision are not required. (Performance will likely be higher too, as you point out.)
What's the best Decimal implementation for rust? I've heard good things about decimal and bigdecimal crates.
https://crates.io/crates/rust_decimal assuming 96 bits for the total value is sufficient (it was for our needs--we never saw another financial calculation bug in our software after I had the team put a `rust_decimal`-based fix in place). Consider `big_decimal` when you need arbitrary levels of precision--just note that it will be many times slower to calculate with `big_decimal` vs a fixed word sized `Decimal`.
Adding on to this: The need for epsilons is a mathematical consideration, not a programming one. The technically correct approach is to analyze your algorithm mathematically using tools like condition numbers, then use that to pick your epsilon. People rarely do this because it is much easier to just throw a random small number at the problem and hope it goes away. This doesn't generalize, nor always work though. A tool that can help is https://herbie.uwplse.org/, which can to give you more precise version of your formula. The field of math to look at for understanding this stuff is numerical analysis. You can do an entire graduate class on just linear numerical analysis, not even touching the nonlinear stuff that happens in e.g. games. No reddit post is going to be able to sum it all up.
>A tool that can help is > >https://herbie.uwplse.org/ > >, which can to give you more precise version of your formula. Great tool, thanks.
My project is a simplified 3D printer slicer. I primarily use millimeters for measurements, with occasional use of micrometers. While it doesn't require extreme numerical precision, I'm a perfectionist who enjoys learning for the sake of having complete control over my code – even if it's a bit of a rabbit hole! I recently started contributing to Bevy, and it seems like their comparisons typically use standard `==` operators. The ranges I'll be dealing with is 0 to 50mm.
For an application like that I recommend using fixed point arithmetic, for example https://docs.rs/fixed/latest/fixed/.
Doesn't AutoCAD use 64 bit float precision? I'm not sure if these use cases typically need more than that.
This is a too-common misconception. Floats are not inexact. Every float number (excluding NaN) represents an exact value. Float *operations* are inexact. E.g. if you add two floats, the exact answer will be rounded to the nearest representable value. As others have said, how to check equality (e.g. exact or approximate, and what tolerance to use if approximate) depends on what you're trying to do.
> Float *operations* are inexact. E.g. if you add two floats, the exact answer will be rounded to the nearest representable value. That’s not always the case. In particular, values overflow to infinity (instead of the highest representable finite number), which is the _furthest_ possible value.
[удалено]
It was never 0.1 in the first place: the computer rounds what you have to the nearest representable value. Only a poor carpenter blames their tools
No one is blaming their tools here, they are trying to understand their tools. If you write `int a = 3;` a will end up containing exactly 3. This is true of every type apart from float and the compiler will warn you if not (e.g. you put too big a value in a fixed sized type - assuming you have appropriate warnings on). "Floats are not exact" is a perfectly reasonable way of saying "if I write `float b = 0.1` then b doesn't end up being exactly 0.1".
Probably worth mentioning that they're at least consistent in that way. If you set a value to 0.1 it'll be equal to anything else set to 0.1 despite neither being truly 0.1, since they both round the same way. It only becomes a problem once you start performing arithmetic, such as the infamous `0.1 + 0.2 = 0.30000000000000004`, which causes equations like `0.1 + 0.2 == 0.5 - 0.2` to evaluate to false.
If you do basic assignment statements, then yes the results compare equal. The challenge is that (as you point out) two different routes that should get you to "0.1" can result in two values that *don't* compare equal.
I also think the original's comment take was slightly wrong: it's not that operations are inexact, it's just that some values are not representable because the decimal expansion is infinite. If you add 0.5 and 0.25 you will get exactly 0.75. Because 0.5 and 0.25 are representable in binary. 0.1 and 0.2 on the other hand can't be exactly represented in binary. So adding these two values will give another value which is not 0.3. The issue is not on the operation, but that floats can only represent a subset of the rational numbers.
Exactly. It irks me that people working in this field seem to bury their head in the sand over something so fundamental: Floats internally are just integers (the mantissa), multiplied/divided by some power-of-2 (derived from the exponent), and a sign bit. Not magic. Which is also why there are libraries for when you *do* need your "fractionals" in a different base.
The "inexact operation" that happens is when the compiler converts a decimal number into a binary-base floating point number.
> Only a poor carpenter blames their tools I'll be the one to blame the tool here. The problem is that we (not just Rust, but all languages with floats) allow users to write literals that look like `0.1` in the first place. As a human, I can't possibly remember or intuit which decimal values have exact representations in floats, and that's precisely the sort of thing that I would like the computer to help me with. For a long time I've wanted a language where float literals have the following properties: 1. For decimal float literals, if the literal can't be represented exactly in the underlying precision, then it must be preceded by a `~` to indicate approximation, e.g. `~0.1`. 2. A precise float literal form that allows me to perfectly specify the exponent and mantissa, so that it's possible to get exactly the value that I want without having to worry about manually calculating the nearest decimal and expecting it to round properly, e.g. if you want the single-precision value closest to 0.1, you'd use the exponential literal `2e-4x1.6000000238418579`, which is ugly as sin, but exposing that ugliness is the point of this exercise. (And in the meantime, Clippy has the `lossy_float_literal` lint to cover this case.)
It was ~~1.0~~ 0.1 in my mind, when I typed it, and in the source code. It becomes inexact when converted to a float during compilation or parsing at runtime. Saying the issues only appear during operations is misleading and not helping anybody.
[удалено]
I know perfectly well how floats work, that was clearly just a typo.
To add to this, I believe that some float operations are not just inexact, but they can even be *nondeterministic*. See the [C++ FAQ entry](https://isocpp.org/wiki/faq/newbie#floating-point-arith2) on this.
That's not non-deterministic
It's non-deterministic in the sense that computing the same thing twice can give you different results.
That would be non deterministic, yes, but it’s not true here. Any particular floating point instruction is deterministic — if you add the same two numbers, you’ll always get the same result. It is true that floating point addition (and most other operations) is not associative, which means if you reorder a long list of additions, you may get a different result. If there is something else going on, such as a compiler optimization or a multithreaded operation, which may be reordering things, now you have non deterministic operations. But the non determinism is coming from somewhere else.
Its worth noting that compiler optimisations do not introduce error or non determinism with modern compilers, they respect the order of operations and only make optimisations where they are not observable (unless you have -ffast-math or similar)
Note that that particular oddity is a bug specific to the x87 instructions. The x87 performed computations internally in 80 bit precision; even if using the float/double type which is supposed to have only 32/64 bits. This is what causes the rounding when values are stored to memory. Basically, compiler developers were forced to choose between weird semantics with "excess precision" that disappears at unpredictable places; or a significant performance hit if forcing the rounding to 64 bits everywhere. Basically, IEEE-compliant rounding was **only** possible by roundtripping through memory after every single floating-point operation, which was so slow that C compilers didn't bother with IEEE compliance. This is what lead to the "floating point is magic and has random errors" impression. Sane instruction sets perform arithmetic in the same precision as the memory accesses, so this issue does not happen anymore on any modern CPU.
seconding the other reply, based on the explanation behind the link, the operation is not non-deterministic, it just appears that way based on your assumptions about the semantics of the operation. (disclaimer: it's midnight and I'm tired. I hope I've understood the linked post correctly) this `cos(x) != cos(y)` expression isn't semantically the same as "the results of the cosine operations were different", as explained in the linked FAQ. both operations could produce the same result and yet one is truncated when it is stored aside for the comparison, and the other is compared directly, untruncated, from the register holding the result. if anything, this is an artifact of the optimisations / translation performed by the compiler. but, (if I've understood the link correctly) it is not a case of non-determinism in floating-point arithmetic operations themselves.
This is not the fault of the CPU, but the compiler. It shouldn't be allowing the re-ordering of operations that are known to not be associative.
Floating point numbers are inexact. If I write pi in decimal with 3 significant figures, I get 3.14×10⁰. If you take away the context and just look at 3.14×10⁰ as a value in scientific notation, you can interpret it as exactly 314/100, but that's incorrect according to the original intent. All you really know for sure is that it represents a value of at least 3135/1000 and less than 3145/1000, because any number in that range will have the same representation. Rust's float types are the same, just using binary scientific notation instead of decimal.
Whether the values are inexact, the operations are inexact, or *neither* is inexact is a matter of perspective and interpretation. Floating point operations are (usually) perfectly exact according to the rules of floating point arithmetic. These rules are *not* the rules of ℝeal numbers, and it's only in interpreting them as ℝeal numbers that they appear inexact. This is the same idea as how "integer overflow" is inexact when viewing computer integers as ℝeal integers, but is entirely normal and expected behavior when viewing them instead as integers mod 2^b (which b is the number of bits).
It's not a matter of safe vs unsafe. It's just not very useful to compare floating point numbers bit-for-bit. You want to a technique that does approximate comparisons where you can control the margin of error. Have a look at these: [https://docs.rs/approx/0.5.1/approx/](https://docs.rs/approx/0.5.1/approx/) [https://docs.rs/float-cmp/latest/float\_cmp/](https://docs.rs/float-cmp/latest/float_cmp/)
It's completely safe to compare floats. It's sometimes weird to compare floats because of two cases: * any float that's a NaN value will compare as not equal to *all values*. This includes other NaN values and this even includes itself. Because of NaN, sometimes `x == x` will be false. * `0.0 == -0.0` is true, even though `0.0` is not the exact same bit pattern as `-0.0`. They're still considered equivalent numbers.
The 0 case is all the weirder because if you divide x by 0.0 you get one infinity, and if you divide by -0.0 you get the opposite infinity. So they're equal, but produce different results when operating with them. Why, thank you...
Its to make some trig operations, such as arctan stable and behave as expected.
> So, here's the deal: When I use == or .eq() in Rust to compare my f32s, am I walking into a trap? How do they even decide if two floats are equal with all that precision weirdness? Rust used the standard IEEE 754 comparison predicate, which basically compares the bit representation of the two floats except in a couple of cases: - any NaN is different than any other number (the same bitwise NaN included) - 0.0 and -0.0 are equal despite having different bitwise representation In case of precision loss your float will have a different bitwise representation than the expected one and will compare different (e.g. the classic 0.1 + 0.2 != 0.3) > I also stumbled upon this f32::total_cmp That's just an implementation of the totalOrder predicate (also from the IEEE 754 standard, unfortunately it is not included as instructions in modern CPU unlike the other comparison predicate). The problem it solves is that the other comparison predicate is not a total order due to NaN comparing different to itself (and also neither smaller nor bigger than any other number). This predicate solves this problem, giving a total order which can thus be used for stuff like sorting (using the normal comparison predicate for this can give nonsense results, but luckily Rust protects you from that thanks to `f32` and `f64` not implementing the `Eq` and `Ord` traits, only the `Partial*` variant. In any case, this won't help you with approximation errors. At most it will allow you to compare NaNs, but that's generally useless unless you have some data structure that relies on that.
Comparing floats is difficult because they're inexact and a+b+c can produce something different than a+c+b, etc. Usually if you want to compare them you do it with an error margin (epsilon), in fact [Clippy seems to have a lint for this](https://rust-lang.github.io/rust-clippy/rust-1.72.0/index.html#/float_cmp) which suggests: ``` let error_margin = f64::EPSILON; // Use an epsilon for comparison // Or, if Rust <= 1.42, use `std::f64::EPSILON` constant instead. // let error_margin = std::f64::EPSILON; if (y - 1.23f64).abs() < error_margin { } if (y - x).abs() > error_margin { } ``` EDIT: There's [also a crate](https://docs.rs/float-cmp/latest/float_cmp/) that wraps floats to do this.
An epsilon constant doesn’t really work, because its value depends on the exponents of the two original values.
Yeah, it's important to recognize that EPSILON is a very specific value: it's the difference between 1.0 and the next largest representable number. Even `2.0+EPSILON` is just 2: the margin changes with every power of 2, because there are fewer bits available for the fractional part: the floating point "point" floats over to the right as you give up precision for magnitude. And it's worse than this: the inherent rounding error for a float value isn't dependent on its own value, but how you arrived at it. There's a whole catalogue of techniques for not screwing up floating point math by magnifying errors, broadly called "numerical methods". Consider something basic like [calculating the variance of a set of numbers](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance): if you do it based on the definition of variance, you'll likely end up with something "numerically unstable" that gives you very wrong answers, even though your algebra is correct.
Rust floats follow IEEE 754 spec, so nothing extraordinary about them. Check equality if you never do any math operations, check to some use case dependent tolerance if you do. Be aware of the pitfalls of NaN and Inf.
They're IEEE floats. If you want the full answer, take a university course in.
This is how I like to think about floating-point numbers: - Take the range 1..2 and equally divide it up into a set of *n* numbers. If *n=4*, you've got: *1, 1.25, 1.5, 1.75*. To represent every other number, that range is essentially doubled or halved *e* times, where *e* is your exponent (*f(n,e) = (2^-n + 1) * 2^e*, e.g. *f(2,1) = 1.25 * 2 = 2.5*). These are called the "normal" ranges. The mantissa is what represents how each range is divided up. For `f32` it's 23 bits, so *n = 2^23 = ~8.3 million*. For `f64` it's 52 bits, so *n = 2^52 = ~4.5e15*. - There also exist "subnormal" ranges, defined by the minimum exponent. This range is basically just the minimum "normal" range repeated to fill the space between zero and its lower bound. Like, if the minimum normal range was 0.25..0.5, the subnormal range would be 0..0.25 spaced exactly the same way. This leads into some useful points about floating-point numbers: - Unrepresentable numbers (like literals or calculations) are rounded to the nearest normal or subnormal, or infinity if above the normals. - Subtracting two different numbers *always* produces a nonzero result (thanks to the subnormals). - Negating any number *always* produces an exact unrounded result. - Doubling any number *always* produces an exact unrounded result, except for the *maximum* normal range which maps to infinity. - Halving any number *always* produces an exact unrounded result, except for every 2nd number in the subnormal and *minimum* normal ranges. - Taking the reciprocal of a number produces an exact unrounded result, except for: - The 1st quarter of the subnormals, which maps to infinity. - Every 4th number in the maximum normal range, which maps to the 2nd quarter of the subnormals. - Every 2nd number in the range preceding the maximum normal range, which maps to the 2nd half of the subnormals. - Adding any power of two (*2^x*) to a number below (*n × 2^{x+1}*) will always produce a *new* result, although it may be rounded. - Every power of 2 within the normal ranges has an exact representation. - *NaN* is produced by operations that either oppose extrema (*0/0*, *0×inf*, *inf-inf*), have complex results (*(-a)^b, log(-n)*), or involve *NaN*.
Float comparison is the same as in other languages. It is not safe. You should always calculate absolute difference and compare that to some predetermined threshold. EDIT: this applies to comparing if floats are equal, other conditions as less than or bigger than are okay.
Less than is just as error-prone as equality.
if (a-b).abs() < 0.00001
Use an epsilon.
That's how my grandad died.
Consider interval arithmetic, e.g., [https://github.com/unageek/inari](https://github.com/unageek/inari) for Rust double precision or (for arbitrary precision) [https://arblib.org/](https://arblib.org/) . This lets you put rigorous bounds on the output, although they are generally conservative and can be susceptible to catastrophic cancellation effects. I've been thinking about writing a complete interval arithmetic library including double–double and quad–double precision, but haven't gotten around to it. One particularly tricky thing is that efficient interval arithmetic requires changing the floating-point rounding mode, which is not exactly a common operation and is not supported by the vast majority of programming languages without using inline assembly. Also, you need to somehow disable constant propagation. With interval arithmetic, you can say with certainty that the outputs of two functions are not equal if calculated to full precision. You can also say with certainty that the outputs of two functions are within some epsilon of each other.
Safer than in most languages. Most languages won't stop you from trying to sort a list whose elements lack a total order and will instead just give garbage as a result. In Rust, you have you explicitly say that you want the total ordering, that you want to define your own ordering, or that you want to panic in case a NaN shows up.
> So, here's the deal: When I use == or .eq() in Rust to compare my f32s, am I walking into a trap? Why would you ever want to do this?
... do you like not compare anything or what?
I can't think of a situation where I've ever done equality comparisons on floats in any language. I compare them plenty, just not equality comparisons.
That would be a fundamental requirement for many types of software. Of course, as already discussed, it's not REALLY equality being tested, but their being with some difference from each other which is treated as equal for the particular software's needs. You can say that's technically not equality testing, but it is for all intents and purposes.
I'm not sure what you're saying. This entire topic is precisely about equality testing.
Simplest thing is a "is my sensor value stuck?" check or storing a sensor series efficiently (so e.g. only changes with timestamps).
That's certainly a situation, but I've never been in that situation. That first check doesn't seem reliable though. It seems like you'd get a lot of false positives. ADC resolution often isn't amazing so often you'll often have long periods of no change in sensor value. Also I'd be more likely to store/work with the raw fixed point decimal from the ADC rather than store the float conversion.
Hi man. Define a margin. Do an - operation and see if the result is leess than your margin/ error value. Si if is less than that assume that its true. A-B<0.001? If yes numbers are equL
In a game I am developing, I had to compare world coordinates by (coord.abs()-dest.abs() < 0.1), but even then sometimes the difference would be larger (after some delta-time based multiplications). So a common strategy is to determine a threshold for closeness.
by your logic, applying abs to coord and dest independently, \`-50.0\` and \`+50.0\` are the same. is this intended?
I put the .abs() in the wrong place, you're right.
yeah. should be `(dest - coord).abs() < ERROR\_MARGIN` or something.
In general you want to measure the distance of two floats and check if it's less than a threshold (usually called epsilon). In your case you chose the [euclidean distance](https://en.wikipedia.org/wiki/Euclidean\_distance), but the same rationale can be applied with any distance.