Severe Decimal error in Foundation on Linux needs workaround

monagle · July 10, 2018, 6:27am

There is a serious bug in Decimal which causes incorrect results when adding and subtracting. The bug can be seen here

It's been fixed and merged into the 4.2 branch, but that leaves many potential people running production code on 4.1.2 who may not be aware that this problem exists.

I wanted to post this here to generate a little awareness around this problem and encourage people to have a good look at their Decimal usage. One of Decimal's biggest use cases is currency - obviously not an ideal place to have a bug of this nature.

In the absence of a patch release for 4.1, it will be at least September before the release of 4.2 - a long time to have a bug of this magnitude in production. Today I started working through the process of creating a patched version of Swift in Docker for us to use in lieu of the offical 4.1.2 branch, but I've already hit several problems getting the build process to work and I think it's going to take a fair bit of my time to get it working.

It has occurred to me that others may have better ideas on how to get a running container up with a version of Foundation that contains the patch. So, if anybody has any input, it would be hugely appreciated!

tkrajacic · July 10, 2018, 8:44am

Now that's a pretty bug…

Nicole_Jacque · July 27, 2018, 9:04pm

I agree, this is a pretty serious bug. We have cherry-picked the fix to the 4.1 branch, and we're going to do a Swift 4.1.3 release with snapshots so that folks can pick this up. We are working on generating the snapshots today, but they may not be complete until Monday.

monagle · July 30, 2018, 12:30am

Thank you so much for getting this done Nicole! We are big advocates for Swift and use it extensively in production so this has been a bit of a headache for us. We will be building based on the new snapshot today and hopefully the days of incorrect invoices will be behind us.

Nicole_Jacque · July 30, 2018, 5:40pm

Ok, snapshot is posted!

dIV · November 8, 2018, 9:40pm

Thank you for this post and the fix in 4.1.3. There seems to be a related problem with NumberFormatter(). When converting the result of a Decimal calculation that results in 0.0. NumberFormatter().string(from:Decimal) returns the last element of the calc rather than 0.00. This happens in 4.1.3 and 4.2 Linux, but works fine in Mac OS. I'm trying to generate order acknowledgements and its important for calculation results to be displayed accurately and properly formatted. It seems that a zero calculated result is the only problem (but not sure and concerned). A workaround is to test for a zero result and then fix the string output. Please let me know if I'm doing something wrong. Thanks!

Here is a simple test script that can be run on Mac OS and Linux with different results:

#!/usr/bin/swift

import Foundation

print("Simple Test\nDecimal\t\tFormatted")

let nf = NumberFormatter()
nf.locale = Locale(identifier: "en_US")
nf.numberStyle = .decimal
nf.minimumFractionDigits = 2
nf.maximumFractionDigits = 2

let x = Decimal(10.5)
let y = Decimal(9.0)
let z = Decimal(1.5)

let a = x - y - z
let b = x - z - y

print("x=Decimal(10.5):Formatted \(nf.string(from: NSDecimalNumber(decimal:x)) ?? "")")
print("y=Decimal(9.0):\tFormatted \(nf.string(from: NSDecimalNumber(decimal:y)) ?? "")")
print("z=Decimal(1.5):\tFormatted \(nf.string(from: NSDecimalNumber(decimal:z)) ?? "")")
print("x - y - z = \(a):\tFormatted \(nf.string(from: NSDecimalNumber(decimal:a)) ?? "")")
print("x - z - y = \(b):\tFormatted \(nf.string(from: NSDecimalNumber(decimal:b)) ?? "")")

print("Possible Workaround?")
print("\\(b==Decimal(0.0) ? (nf.string(from: NSDecimalNumber(decimal:Decimal(0.0))) ?? \"\"):(nf.string(from: NSDecimalNumber(decimal:b)) ?? \"\"))")
print("Result \(b==Decimal(0.0) ? (nf.string(from: NSDecimalNumber(decimal:Decimal(0.0))) ?? ""):(nf.string(from: NSDecimalNumber(decimal:b)) ?? ""))")

Results - Docker Official - Swift 4.2 ubuntu:16.04

Simple Test
Decimal		Formatted
x=Decimal(10.5):Formatted 10.50
y=Decimal(9.0):	Formatted 9.00
z=Decimal(1.5):	Formatted 1.50
x - y - z = 0:	Formatted 1.50
x - z - y = 0:	Formatted 9.00
Possible Workaround?
\(b==Decimal(0.0) ? (nf.string(from: NSDecimalNumber(decimal:Decimal(0.0))) ?? ""):(nf.string(from: NSDecimalNumber(decimal:b)) ?? ""))
Result 0.00

Results Swift 4.1.2 MacOS 10.13.6

Simple Test
Decimal		Formatted
x=Decimal(10.5):Formatted 10.50
y=Decimal(9.0):	Formatted 9.00
z=Decimal(1.5):	Formatted 1.50
x - y - z = 0:	Formatted 0.00
x - z - y = 0:	Formatted 0.00
Possible Workaround?
\(b==Decimal(0.0) ? (nf.string(from: NSDecimalNumber(decimal:Decimal(0.0))) ?? ""):(nf.string(from: NSDecimalNumber(decimal:b)) ?? ""))
Result 0.00

spevans · November 9, 2018, 4:03pm

It looks to be a bug with the .doubleValue property in Decimal not generating the correct output whereas .description seems to be correct. Would you mind opening a JIRA for this? Thanks

mattjgalloway · June 15, 2020, 7:45am

I seem to have found another pretty severe problem with Decimal on Linux:

github.com/apple/swift-corelibs-foundation

[SR-13015] Decimal calculation incorrect on Linux

opened 08:30PM - 14 Jun 20 UTC

closed 09:47AM - 23 Jun 20 UTC

swift-ci

bug Foundation

| | | |------------------|-----------------|… |Previous ID | SR-13015 | |Radar | rdar://problem/64517015 | |Original Reporter | mattjgalloway (JIRA User) | |Type | Bug | |Status | Resolved | |Resolution | Done | <details> <summary>Environment</summary> macOS (Catalina 10.15.5): ``` java $ swift --version Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53) Target: x86_64-apple-darwin19.5.0 ``` Linux (Debian stretch, using Ubuntu 16.04 binaries): ``` java $ swift --version Swift version 5.2.4 (swift-5.2.4-RELEASE) Target: x86_64-unknown-linux-gnu ``` en_GB locale </details> <details> <summary>Additional Detail from JIRA</summary> | | | |------------------|-----------------| |Votes | 4 | |Component/s | Foundation | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: 5c0852cd4db2eecc9ea3b60ca7a075dc </details> **Issue Description:** I've noticed through a rather strange turn of events that there appears to be a problem in Foundation's `Decimal` class on Linux. Take for example this code: ``` java let a = Decimal(string: "119.993")! let b = Decimal(string: "4.1565")! let c = Decimal(string: "18.209")! let d = Decimal(string: "258.469")! let result = (a * b) + (c * (a / d)) ``` On macOS you'll get the following result: ``` java result: Decimal = 507.204346 ``` On Linux you'll get the following result: ``` java result: Decimal = 166.921979 ``` My calculator agrees with macOS. If I change pretty much any of the values for `a`, `b`, `c` and `d` then the result will end up being the same on both platforms. For example, if you change `a` to `119.99` - so simply removing the final `3` - then the result is exactly the same on both platforms. Apologies for the convoluted calculation - I've been trying to get a minimal test case, but whenever I change anything the problem goes away.

Posting here for visibility in case anyone here has thoughts!

johannesweiss · June 16, 2020, 1:59pm

CC @millenomi

casperes1996 · September 29, 2020, 1:12am

Wait, you say use with currency, but isn't the Decimal type still a fixed precision floating point structure, even if it avoids some of the normal issues of float? Wouldn't using an integer type for currency be much better?