Fix Decimal for Swift 5.2

tkrajacic · October 17, 2019, 10:52am

Since Swift 5.2 seems to be considered a "…significant quality and performance enhancements" release, can we PLEASE fix Decimals in Swift?

github.com/apple/swift

[SR-920] Re-design builtin compiler protocols for literal convertible types

opened 02:22AM - 11 Mar 16 UTC

gribozavr

improvement compiler feature standard library affects ABI swift evolution proposal needed

| | | |------------------|-----------------|… |Previous ID | SR-920 | |Radar | rdar://problem/25100871 | |Original Reporter | @gribozavr | |Type | Improvement | <details> <summary>Additional Detail from JIRA</summary> | | | |------------------|-----------------| |Votes | 17 | |Component/s | Standard Library | |Labels | Improvement, AffectsABI | |Assignee | None | |Priority | Medium | md5: 9c9daaa0076abc9937d022ed16def26e </details> **is duplicated by**: * [SR-3317](https://bugs.swift.org/browse/SR-3317) Literal protocol for decimal literals should support precise decimal accuracy * [SR-3970](https://bugs.swift.org/browse/SR-3970) Rework ExpressibleBy(Integer|Float)Literal protocols **relates to**: * [SR-7124](https://bugs.swift.org/browse/SR-7124) Double-rounding in floating-point literal conversion **Issue Description:** Swift aims to be an extensible language through the built-in compiler protocols, which allow user-defined types to adopt literal syntax. If we want to commit to that design as ABI, we need to improve the protocols so that they are actually useful outside of the standard library. 1\. There should be a type in the standard library that can represent the literal in a lossless way. For example, currently integer and floating point literals are truncated to the maximum precision supported by the library (Int64 and Float80) before they can be handled by user code. For example, the library could provide an IntegerLiteral type, which wraps a 2048-bit integer that user types adopting the literal protocols could use. Same for floating-point literals (but the number should be represented exactly to allow decimal foating point literals). 2\. Protocols related to string literals should have reasonable default implementations. Currently, to implement a string literal convertible type, you need to write many functions that just forward to each other to conform to the protocol. 3\. When we get special transport types, the initializer labels can become simpler (e.g. drop the ‘literal’ from the label, since that would repeat type information).

and also the blatant data corruption when using Codable:

github.com/apple/swift-corelibs-foundation

[SR-7054] JSONDecoder Decimal precision error

opened 08:26AM - 22 Feb 18 UTC

swift-ci

bug Foundation

| | | |------------------|-----------------|… |Previous ID | SR-7054 | |Radar | rdar://problem/33491336 | |Original Reporter | tiborbodecs (JIRA User) | |Type | Bug | <details> <summary>Environment</summary> Xcode 9.2 (9C40b) Apple Swift version 4.0.3 (swiftlang-900.0.74.1 clang-900.0.39.2) Same output with 4.1 swift-DEVELOPMENT-SNAPSHOT-2018-02-20-a-ubuntu16.04. </details> <details> <summary>Additional Detail from JIRA</summary> | | | |------------------|-----------------| |Votes | 38 | |Component/s | Foundation | |Labels | Bug | |Assignee | bendjones (JIRA) | |Priority | Medium | md5: 4d586dd9920250512d8b01ea1b3597c9 </details> **Issue Description:** Decoding decimals from a JSON structure returns incorrect numbers. See the following example: ``` none #!/usr/bin/env swift import Foundation struct DoubleItem : Codable \{ var name: String var price: Double } struct DecimalItem : Codable \{ var name: String var price: Decimal } let jsonString = """ \{ "name": "Gum ball", "price": 46.984765 } """ let jsonData = jsonString.data(using: .utf8)! let decoder = JSONDecoder() do \{ let doubleItem = try decoder.decode(DoubleItem.self, from: jsonData) print(doubleItem) let decimalItem = try decoder.decode(DecimalItem.self, from: jsonData) print(decimalItem) } catch \{ print(error) } ``` Output: ``` none DoubleItem(name: "Gum ball", price: 46.984765000000003) DecimalItem(name: "Gum ball", price: 46.98476500000001024) ``` Expected result for the DecimalItem should be: 46.984765 I know that floating point values can not be represented precisely, but decimals should keep their original values, am I wrong? Is this a Swift foundation bug or an intended behavior? Note that encoding the same value with the JSONEncoder will provide the correct value in the JSON. My actual problem is that the JSONEncoder and JSONDecoder classes are working inconsistently. If the encoder will output the value as a number, I'd expect that the decoder can decode the exact same value from that if I use the decimal type in my model. My other idea is that the encoder should transform and save the value into a string type inside the JSON instead of the current representation, this could be supported with a decimal coding strategy. ``` none let encoder = JSONEncoder() encoder.decimalEncodingStrategy = .string //outputs as string: "46.984765" encoder.decimalEncodingStrategy = .precise //outputs as number: 46.984765 encoder.decimalEncodingStrategy = .lossy //current output like: 46.984765 let encoder = JSONDecoder() encoder.decimalDecodingStrategy = .string //decoded value from json string: 46.984765 encoder.decimalDecodingStrategy = .precise //decoded value from json number: 46.984765 encoder.decimalDecodingStrategy = .lossy //current value from number: 46.98476500000001024 ``` Please share your thoughts about this idea.

Sry for whining here but it really boggles my mind how this can go unsolved for so long.

cukr · October 17, 2019, 11:28am

Decimal isn't part of swift. It's a part of Foundation, a closed-source apple framework that is older than the swift project, so "us fixing Decimals in Swift" isn't possible, apple has to fix it themselves.

There's also swift-corelibs-foundation that's open source, but just tries to maintain feature parity with apple foundation

hisekaldma · October 17, 2019, 11:35am

But the problem that Decimal can't be initialized exactly from what looks like a decimal literal, but is actually a floating point literal, is definitely a part of Swift. Apple couldn’t fix this in Foundation without also introducing proper decimal literals to Swift.

xwu · October 17, 2019, 1:38pm

“We” can certainly fix this; the way to do so is outlined in the bug. I’d like to see it fixed as well but I don’t have time to do it.

I’m not sure what we’re discussing here. Are you volunteering to implement the redesign?

tkrajacic · October 17, 2019, 2:41pm

Sadly my skills aren't nearly sufficient in that regard, so we're discussing "Please can someone fix this" rather

nuclearace · October 17, 2019, 3:02pm

It's unfortunate that it's been this way for so long, but I don't think these forums are really the place for "this is broke and I really want it fixed now" topics.

There are many things in the Swift toolchain and compiler that are frustrating to work with, either because they don't exist (generic math functions, a standard library BigNum/BigDecimal, first-class concurrency, local lazy/property wrappers, etc), or they don't work quite right (this being a good example, or most of the linux foundation, SPM still missing key features)

I think the best thing to do in these cases is to comment on the bug tracking this issue and to try and get it prioritized.

The forums would be a good place to ask for help in the implementation of the changes, or if there are still outstanding design questions, we can discuss those. But just saying, "hey, this should be fixed in Swift 5.2", isn't going to be very productive. As I think we all can think of several things we would like to see fixed in Swift ASAP.

scanon · October 17, 2019, 3:25pm

Adding to what @nuclearace said: "this is important to me" is a useful signal to have in the bug tracker. "This is important to me because I need to do ..." is even better, and has the additional benefit that in many cases someone will be able to provide you with a workaround or alternative solution in the meantime.

tkrajacic · October 17, 2019, 3:42pm

I apologise if I came across too demanding or complaining, and while I can sympathise with your points of view, I think that it does not quite apply.

This is not only something that "is important to me" but something that is fundamentally broken in the language for quite some time imho.

I know that I am in no position to argue about that, if only it was for my inferior knowledge of numerics, but if a common user can't initialise a decimal variable without immediately getting a corrupt value, that's seriously broken, and not a matter of "just for me, or some users".

I have filed many bugs, and happily continue to do so, but this one doesn't get the attention it deserves I think, and commenting on the bug reports has done nothing it seems to me.

nuclearace · October 17, 2019, 4:00pm

Of course, this is a nasty gotcha that has been sitting around in the language since forever. And maybe it was a mistake to have Decimal conform to the float literal protocol without this "bug" fixed, but we can't really change that now. Just today I was toying around with creating a BigDecimal type built on AttaSwift's BigInt and I remembered this exact pitfall WRT float literals. Which is why I'm not making my BigDecimal type conform to that literal protocol.

Perhaps a more interesting discussion is to look at why it hasn't been fixed in all these years. Is the solution truly a difficult one to implement? Is the suggested solution even still viable? If it isn't viable, what are the alternatives? We could maybe even start looking for some pointers on where someone should start playing around with trying to tackle the issue.