Pure Cocoa NSNumbers and AnyHashable


(Joe Groff) #1

There's a hole in our AnyHashable implementation when it comes to what I'll call "pure" NSNumbers coming from Cocoa, which were instantiated using -[NSNumber numberWith*:] factories or @(n) syntax in Objective-C. While we maintain type specificity when Swift number types are bridged through NSNumber, NSNumbers constructed in ObjC do not necessarily remember the type they were constructed with or expect to be strictly used as only that type, so we resign to being "fuzzy" and let them bridge back to any Swift type. We however fail to bring this fuzziness to AnyHashable. When we construct an AnyHashable, we'll bring bridged NSNumbers back to their original Swift types, but we leave a pure NSNumber as an NSNumber, so it doesn't hash or equate with numeric values in Swift:

// ObjC
@import Foundation;

NSDictionary *foo() {
  return @{@(1): @"one"};
}

// Swift
let theFoo /*: [AnyHashable: Any]*/ = foo()
theFoo[1] // returns nil, ought to find the value "one"

One way to address this would be to make Swift's number types use the same hashing as NSNumber does. We could go so far as to switch the "custom AnyHashable" polarity around and coerce the Swift number types into NSNumbers when we put them inside AnyHashable, which would give us consistent hashing and fuzzy equality, but would come at a performance cost when converting a number to AnyHashable. We would also lose type specificity in equality for Swift values, since NSNumber's -isEqual: only compares numeric value, unless we special-cased NSNumber in AnyHashable's implementation.

If we didn't want to adopt NSNumber's hashing for Swift's types, but we were willing to say that all of Swift's number types produce the same hashValue for the same numeric value (so 12.hashValue == 12.0.hashValue == (12 as UInt8).hashValue, etc.), we could also go the other direction, and customize a pure NSNumber's AnyHashable implementation to use Swift's number hashing. We would still need special handling for equality of a pure NSNumber with Swift numbers, but maybe that's inevitable.

-Joe


(Joe Groff) #2

A quick ping. I'd like some feedback about how to address this problem.

-Joe

···

On Nov 2, 2016, at 10:26 AM, Joe Groff via swift-dev <swift-dev@swift.org> wrote:

There's a hole in our AnyHashable implementation when it comes to what I'll call "pure" NSNumbers coming from Cocoa, which were instantiated using -[NSNumber numberWith*:] factories or @(n) syntax in Objective-C. While we maintain type specificity when Swift number types are bridged through NSNumber, NSNumbers constructed in ObjC do not necessarily remember the type they were constructed with or expect to be strictly used as only that type, so we resign to being "fuzzy" and let them bridge back to any Swift type. We however fail to bring this fuzziness to AnyHashable. When we construct an AnyHashable, we'll bring bridged NSNumbers back to their original Swift types, but we leave a pure NSNumber as an NSNumber, so it doesn't hash or equate with numeric values in Swift:

// ObjC
@import Foundation;

NSDictionary *foo() {
  return @{@(1): @"one"};
}

// Swift
let theFoo /*: [AnyHashable: Any]*/ = foo()
theFoo[1] // returns nil, ought to find the value "one"

One way to address this would be to make Swift's number types use the same hashing as NSNumber does. We could go so far as to switch the "custom AnyHashable" polarity around and coerce the Swift number types into NSNumbers when we put them inside AnyHashable, which would give us consistent hashing and fuzzy equality, but would come at a performance cost when converting a number to AnyHashable. We would also lose type specificity in equality for Swift values, since NSNumber's -isEqual: only compares numeric value, unless we special-cased NSNumber in AnyHashable's implementation.

If we didn't want to adopt NSNumber's hashing for Swift's types, but we were willing to say that all of Swift's number types produce the same hashValue for the same numeric value (so 12.hashValue == 12.0.hashValue == (12 as UInt8).hashValue, etc.), we could also go the other direction, and customize a pure NSNumber's AnyHashable implementation to use Swift's number hashing. We would still need special handling for equality of a pure NSNumber with Swift numbers, but maybe that's inevitable.

-Joe
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Philippe Hausler) #3

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?

Also the other side of things is that NSNumber is a different concept than Int or Float or Double etc. if it was written in Swift would roughly look like this:

enum Numeric : Hashable {
    case bool(Bool)
    case char(Int8)
    case unsignedChar(UInt8)
    case short(Int16)
    case unsignedShort(UInt16)
    case int(Int32)
    case unsignedInt(UInt32)
    case long(Int)
    case unsignedLong(UInt)
    case float(Float)
    case double(Double)
    case longLong(Int64)
    case unsignedLongLong(UInt64)
    
    var longValue: Int {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return Int(value)
        case .unsignedChar(let value):
            return Int(value)
        case .short(let value):
            return Int(value)
        case .unsignedShort(let value):
            return Int(value)
        case .int(let value):
            return Int(value)
        case .unsignedInt(let value):
            return Int(value)
        case .long(let value):
            return value
        case .unsignedLong(let value):
            return Int(value)
        case .float(let value):
            return Int(value)
        case .double(let value):
            return Int(value)
        case .longLong(let value):
            return Int(value)
        case .unsignedLongLong(let value):
            return Int(value)
        }
    }
    
    var unsignedLongValue: UInt {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return UInt(value)
        case .unsignedChar(let value):
            return UInt(value)
        case .short(let value):
            return UInt(value)
        case .unsignedShort(let value):
            return UInt(value)
        case .int(let value):
            return UInt(value)
        case .unsignedInt(let value):
            return UInt(value)
        case .long(let value):
            return UInt(value)
        case .unsignedLong(let value):
            return value
        case .float(let value):
            return UInt(value)
        case .double(let value):
            return UInt(value)
        case .longLong(let value):
            return UInt(value)
        case .unsignedLongLong(let value):
            return UInt(value)
        }
    }
    
    var integerValue: Int {
        return longValue
    }
    
    var unsignedIntegerValue: UInt {
        return unsignedLongValue
    }
    
    var doubleValue: Double {
        switch self {
        case .bool(let value):
            return value ? 1.0 : 0.0
        case .char(let value):
            return Double(value)
        case .unsignedChar(let value):
            return Double(value)
        case .short(let value):
            return Double(value)
        case .unsignedShort(let value):
            return Double(value)
        case .int(let value):
            return Double(value)
        case .unsignedInt(let value):
            return Double(value)
        case .long(let value):
            return Double(value)
        case .unsignedLong(let value):
            return Double(value)
        case .float(let value):
            return Double(value)
        case .double(let value):
            return value
        case .longLong(let value):
            return Double(value)
        case .unsignedLongLong(let value):
            return Double(value)
        }
    }
    
    var longLongValue: Int64 {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return Int64(value)
        case .unsignedChar(let value):
            return Int64(value)
        case .short(let value):
            return Int64(value)
        case .unsignedShort(let value):
            return Int64(value)
        case .int(let value):
            return Int64(value)
        case .unsignedInt(let value):
            return Int64(value)
        case .long(let value):
            return Int64(value)
        case .unsignedLong(let value):
            return Int64(value)
        case .float(let value):
            return Int64(value)
        case .double(let value):
            return Int64(value)
        case .longLong(let value):
            return Int64(value)
        case .unsignedLongLong(let value):
            return Int64(value)
        }
    }
    
    private static let HASHFACTOR = 2654435761
    
    private static func hashInt(_ i: Int) -> Int {
        return (i > 0 ? (i) : (-i)) * HASHFACTOR
    }
    
    private static func hashDouble(_ d_: Double) -> Int {
        var d = d_
        if d < 0 { d = -d }
        let dInt = floor(d + 0.5)
        let integralHash = HASHFACTOR * Int(fmod(dInt, Double(UInt.max)))
        return (integralHash + Int(((d - dInt) * Double(UInt.max))))
    }
    
    var hashValue: Int {
        switch self {
        case .long:
            fallthrough
        case .int:
            fallthrough
        case .char:
            fallthrough
        case .bool:
            let i = integerValue
            return Numeric.hashInt(i)
        case .unsignedLong:
            fallthrough
        case .unsignedInt:
            fallthrough
        case .unsignedChar:
            let i = unsignedIntegerValue
            return i > UInt(Int.max) ? Numeric.hashDouble(Double(i)) : Numeric.hashInt(Int(i))
        case .longLong(let value):
            return Numeric.hashDouble(Double(value))
        case .unsignedLongLong(let value):
            return Numeric.hashDouble(Double(value))
        default:
            return Numeric.hashDouble(doubleValue)
        }
    }
    
    private static func compareDoubles(_ d1: Double, _ d2: Double) -> ComparisonResult {
        if d1.isNaN || d2.isNaN {
            if d1.isNaN {
                if d2.isNaN {
                    return .orderedSame
                }
                return copysign(1.0, d1) < 0.0 ? .orderedDescending : .orderedAscending
            }
            return copysign(1.0, d1) < 0 ? .orderedAscending : .orderedDescending
        }
        if d1 < d2 { return .orderedAscending }
        if d2 < d1 { return .orderedDescending }
        return .orderedSame
    }
    
    static func ==(_ lhs: Numeric, _ rhs: Numeric) -> Bool {
        switch (lhs, rhs) {
        case (.double, .double):
            fallthrough
        case (.double, .float):
            fallthrough
        case (.float, .double):
            fallthrough
        case (.float, .float):
            return compareDoubles(lhs.doubleValue, rhs.doubleValue) == .orderedSame
        case (.unsignedLongLong(let lhsValue), .unsignedLongLong(let rhsValue)):
            return lhsValue == rhsValue
        case (.unsignedLongLong(let lhsValue), _):
            if lhsValue >= UInt64(Int64.max) { return false }
            break
        case (_, .unsignedLongLong(let rhsValue)):
            if rhsValue >= UInt64(Int64.max) { return false }
            break
        default:
            break
        }
        return lhs.longLongValue == rhs.longLongValue
    }
}

···

On Nov 10, 2016, at 9:48 AM, Joe Groff <jgroff@apple.com> wrote:

A quick ping. I'd like some feedback about how to address this problem.

-Joe

On Nov 2, 2016, at 10:26 AM, Joe Groff via swift-dev <swift-dev@swift.org> wrote:

There's a hole in our AnyHashable implementation when it comes to what I'll call "pure" NSNumbers coming from Cocoa, which were instantiated using -[NSNumber numberWith*:] factories or @(n) syntax in Objective-C. While we maintain type specificity when Swift number types are bridged through NSNumber, NSNumbers constructed in ObjC do not necessarily remember the type they were constructed with or expect to be strictly used as only that type, so we resign to being "fuzzy" and let them bridge back to any Swift type. We however fail to bring this fuzziness to AnyHashable. When we construct an AnyHashable, we'll bring bridged NSNumbers back to their original Swift types, but we leave a pure NSNumber as an NSNumber, so it doesn't hash or equate with numeric values in Swift:

// ObjC
@import Foundation;

NSDictionary *foo() {
return @{@(1): @"one"};
}

// Swift
let theFoo /*: [AnyHashable: Any]*/ = foo()
theFoo[1] // returns nil, ought to find the value "one"

One way to address this would be to make Swift's number types use the same hashing as NSNumber does. We could go so far as to switch the "custom AnyHashable" polarity around and coerce the Swift number types into NSNumbers when we put them inside AnyHashable, which would give us consistent hashing and fuzzy equality, but would come at a performance cost when converting a number to AnyHashable. We would also lose type specificity in equality for Swift values, since NSNumber's -isEqual: only compares numeric value, unless we special-cased NSNumber in AnyHashable's implementation.

If we didn't want to adopt NSNumber's hashing for Swift's types, but we were willing to say that all of Swift's number types produce the same hashValue for the same numeric value (so 12.hashValue == 12.0.hashValue == (12 as UInt8).hashValue, etc.), we could also go the other direction, and customize a pure NSNumber's AnyHashable implementation to use Swift's number hashing. We would still need special handling for equality of a pure NSNumber with Swift numbers, but maybe that's inevitable.

-Joe
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Philippe Hausler) #4

Just realized some of that code won't work correctly for certain hashes - it needs to all be UInt bit patterns since Cocoa uses unsigned values for hashing and not signed Int.

···

Sent from my iPhone

On Nov 10, 2016, at 10:30 AM, Philippe Hausler via swift-dev <swift-dev@swift.org> wrote:

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?

Also the other side of things is that NSNumber is a different concept than Int or Float or Double etc. if it was written in Swift would roughly look like this:

enum Numeric : Hashable {
    case bool(Bool)
    case char(Int8)
    case unsignedChar(UInt8)
    case short(Int16)
    case unsignedShort(UInt16)
    case int(Int32)
    case unsignedInt(UInt32)
    case long(Int)
    case unsignedLong(UInt)
    case float(Float)
    case double(Double)
    case longLong(Int64)
    case unsignedLongLong(UInt64)
    
    var longValue: Int {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return Int(value)
        case .unsignedChar(let value):
            return Int(value)
        case .short(let value):
            return Int(value)
        case .unsignedShort(let value):
            return Int(value)
        case .int(let value):
            return Int(value)
        case .unsignedInt(let value):
            return Int(value)
        case .long(let value):
            return value
        case .unsignedLong(let value):
            return Int(value)
        case .float(let value):
            return Int(value)
        case .double(let value):
            return Int(value)
        case .longLong(let value):
            return Int(value)
        case .unsignedLongLong(let value):
            return Int(value)
        }
    }
    
    var unsignedLongValue: UInt {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return UInt(value)
        case .unsignedChar(let value):
            return UInt(value)
        case .short(let value):
            return UInt(value)
        case .unsignedShort(let value):
            return UInt(value)
        case .int(let value):
            return UInt(value)
        case .unsignedInt(let value):
            return UInt(value)
        case .long(let value):
            return UInt(value)
        case .unsignedLong(let value):
            return value
        case .float(let value):
            return UInt(value)
        case .double(let value):
            return UInt(value)
        case .longLong(let value):
            return UInt(value)
        case .unsignedLongLong(let value):
            return UInt(value)
        }
    }
    
    var integerValue: Int {
        return longValue
    }
    
    var unsignedIntegerValue: UInt {
        return unsignedLongValue
    }
    
    var doubleValue: Double {
        switch self {
        case .bool(let value):
            return value ? 1.0 : 0.0
        case .char(let value):
            return Double(value)
        case .unsignedChar(let value):
            return Double(value)
        case .short(let value):
            return Double(value)
        case .unsignedShort(let value):
            return Double(value)
        case .int(let value):
            return Double(value)
        case .unsignedInt(let value):
            return Double(value)
        case .long(let value):
            return Double(value)
        case .unsignedLong(let value):
            return Double(value)
        case .float(let value):
            return Double(value)
        case .double(let value):
            return value
        case .longLong(let value):
            return Double(value)
        case .unsignedLongLong(let value):
            return Double(value)
        }
    }
    
    var longLongValue: Int64 {
        switch self {
        case .bool(let value):
            return value ? 1 : 0
        case .char(let value):
            return Int64(value)
        case .unsignedChar(let value):
            return Int64(value)
        case .short(let value):
            return Int64(value)
        case .unsignedShort(let value):
            return Int64(value)
        case .int(let value):
            return Int64(value)
        case .unsignedInt(let value):
            return Int64(value)
        case .long(let value):
            return Int64(value)
        case .unsignedLong(let value):
            return Int64(value)
        case .float(let value):
            return Int64(value)
        case .double(let value):
            return Int64(value)
        case .longLong(let value):
            return Int64(value)
        case .unsignedLongLong(let value):
            return Int64(value)
        }
    }
    
    private static let HASHFACTOR = 2654435761
    
    private static func hashInt(_ i: Int) -> Int {
        return (i > 0 ? (i) : (-i)) * HASHFACTOR
    }
    
    private static func hashDouble(_ d_: Double) -> Int {
        var d = d_
        if d < 0 { d = -d }
        let dInt = floor(d + 0.5)
        let integralHash = HASHFACTOR * Int(fmod(dInt, Double(UInt.max)))
        return (integralHash + Int(((d - dInt) * Double(UInt.max))))
    }
    
    var hashValue: Int {
        switch self {
        case .long:
            fallthrough
        case .int:
            fallthrough
        case .char:
            fallthrough
        case .bool:
            let i = integerValue
            return Numeric.hashInt(i)
        case .unsignedLong:
            fallthrough
        case .unsignedInt:
            fallthrough
        case .unsignedChar:
            let i = unsignedIntegerValue
            return i > UInt(Int.max) ? Numeric.hashDouble(Double(i)) : Numeric.hashInt(Int(i))
        case .longLong(let value):
            return Numeric.hashDouble(Double(value))
        case .unsignedLongLong(let value):
            return Numeric.hashDouble(Double(value))
        default:
            return Numeric.hashDouble(doubleValue)
        }
    }
    
    private static func compareDoubles(_ d1: Double, _ d2: Double) -> ComparisonResult {
        if d1.isNaN || d2.isNaN {
            if d1.isNaN {
                if d2.isNaN {
                    return .orderedSame
                }
                return copysign(1.0, d1) < 0.0 ? .orderedDescending : .orderedAscending
            }
            return copysign(1.0, d1) < 0 ? .orderedAscending : .orderedDescending
        }
        if d1 < d2 { return .orderedAscending }
        if d2 < d1 { return .orderedDescending }
        return .orderedSame
    }
    
    static func ==(_ lhs: Numeric, _ rhs: Numeric) -> Bool {
        switch (lhs, rhs) {
        case (.double, .double):
            fallthrough
        case (.double, .float):
            fallthrough
        case (.float, .double):
            fallthrough
        case (.float, .float):
            return compareDoubles(lhs.doubleValue, rhs.doubleValue) == .orderedSame
        case (.unsignedLongLong(let lhsValue), .unsignedLongLong(let rhsValue)):
            return lhsValue == rhsValue
        case (.unsignedLongLong(let lhsValue), _):
            if lhsValue >= UInt64(Int64.max) { return false }
            break
        case (_, .unsignedLongLong(let rhsValue)):
            if rhsValue >= UInt64(Int64.max) { return false }
            break
        default:
            break
        }
        return lhs.longLongValue == rhs.longLongValue
    }
}

On Nov 10, 2016, at 9:48 AM, Joe Groff <jgroff@apple.com> wrote:

A quick ping. I'd like some feedback about how to address this problem.

-Joe

On Nov 2, 2016, at 10:26 AM, Joe Groff via swift-dev <swift-dev@swift.org> wrote:

There's a hole in our AnyHashable implementation when it comes to what I'll call "pure" NSNumbers coming from Cocoa, which were instantiated using -[NSNumber numberWith*:] factories or @(n) syntax in Objective-C. While we maintain type specificity when Swift number types are bridged through NSNumber, NSNumbers constructed in ObjC do not necessarily remember the type they were constructed with or expect to be strictly used as only that type, so we resign to being "fuzzy" and let them bridge back to any Swift type. We however fail to bring this fuzziness to AnyHashable. When we construct an AnyHashable, we'll bring bridged NSNumbers back to their original Swift types, but we leave a pure NSNumber as an NSNumber, so it doesn't hash or equate with numeric values in Swift:

// ObjC
@import Foundation;

NSDictionary *foo() {
return @{@(1): @"one"};
}

// Swift
let theFoo /*: [AnyHashable: Any]*/ = foo()
theFoo[1] // returns nil, ought to find the value "one"

One way to address this would be to make Swift's number types use the same hashing as NSNumber does. We could go so far as to switch the "custom AnyHashable" polarity around and coerce the Swift number types into NSNumbers when we put them inside AnyHashable, which would give us consistent hashing and fuzzy equality, but would come at a performance cost when converting a number to AnyHashable. We would also lose type specificity in equality for Swift values, since NSNumber's -isEqual: only compares numeric value, unless we special-cased NSNumber in AnyHashable's implementation.

If we didn't want to adopt NSNumber's hashing for Swift's types, but we were willing to say that all of Swift's number types produce the same hashValue for the same numeric value (so 12.hashValue == 12.0.hashValue == (12 as UInt8).hashValue, etc.), we could also go the other direction, and customize a pure NSNumber's AnyHashable implementation to use Swift's number hashing. We would still need special handling for equality of a pure NSNumber with Swift numbers, but maybe that's inevitable.

-Joe
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Joe Groff) #5

The type-preserving subclasses remember the exact Swift type that a value was bridged from, to preserve type specificity of casts so that e.g. `0 as Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> AnyObject round-trip involves ObjC, thereby providing somewhat more consistent behavior between Darwin and Corelibs-based Swift. If we were willing to give that up, and say that NSNumbers just flat-out lose type info and can cast back to any Swift number type, then it seems to me we could use the pure Cocoa subclasses. Since those are almost always tagged pointers, or at least pooled and cached on platforms that don't support tagged pointers, the overhead of then having AnyHashable push numbers into NSNumbers would be less.

-Joe

···

On Nov 10, 2016, at 10:30 AM, Philippe Hausler <phausler@apple.com> wrote:

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?


(Matthew Johnson) #6

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?

The type-preserving subclasses remember the exact Swift type that a value was bridged from, to preserve type specificity of casts so that e.g. `0 as Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> AnyObject round-trip involves ObjC, thereby providing somewhat more consistent behavior between Darwin and Corelibs-based Swift. If we were willing to give that up, and say that NSNumbers just flat-out lose type info and can cast back to any Swift number type, then it seems to me we could use the pure Cocoa subclasses.

Would these only be value-preserving casts and return nil if information loss would occur? I think that might be preferable anyway. Maybe I’m just not thinking hard enough, but when would the original type information be important as long as information isn’t lost? When I interact with these casts and NSNumber (JSON parsing, etc) I generally *do not* want an attempted cast that would be value-preserving to ever fail.

I'm inclined to agree that the cast should be value-preserving rather than type-preserving. There was concern about the behavior being different on Darwin and Linux, which is why we try to be type-preserving so that pure Swift code that uses number values with Any or other polymorphic interfaces behaves consistently with Cocoa Foundation code that has to traffic in NSNumber for the same effect.

Are you saying that Swift on Darwin can’t have value-preserving behavior? It seems like I’m missing something here. If value-preserving is the desirable behavior can you elaborate on the specific challenges getting in the way of having that behavior everywhere?

···

On Nov 10, 2016, at 1:44 PM, Joe Groff <jgroff@apple.com> wrote:

On Nov 10, 2016, at 11:42 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev <swift-dev@swift.org> wrote:

On Nov 10, 2016, at 10:30 AM, Philippe Hausler <phausler@apple.com> wrote:

-Joe


(Joe Groff) #7

It would require a change to the type relationships between the number value types on Swift. They are currently plain, unrelated struct types, so you can't do something like:

  let x: Int = 1
  x as? Float // produces nil

We could conceivably special-case the number types in Swift so that they do value-preserving casts, and maybe that's even a good idea, but we don't today.

-Joe

···

On Nov 10, 2016, at 11:51 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Nov 10, 2016, at 1:44 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Nov 10, 2016, at 11:42 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

On Nov 10, 2016, at 10:30 AM, Philippe Hausler <phausler@apple.com <mailto:phausler@apple.com>> wrote:

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?

The type-preserving subclasses remember the exact Swift type that a value was bridged from, to preserve type specificity of casts so that e.g. `0 as Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> AnyObject round-trip involves ObjC, thereby providing somewhat more consistent behavior between Darwin and Corelibs-based Swift. If we were willing to give that up, and say that NSNumbers just flat-out lose type info and can cast back to any Swift number type, then it seems to me we could use the pure Cocoa subclasses.

Would these only be value-preserving casts and return nil if information loss would occur? I think that might be preferable anyway. Maybe I’m just not thinking hard enough, but when would the original type information be important as long as information isn’t lost? When I interact with these casts and NSNumber (JSON parsing, etc) I generally *do not* want an attempted cast that would be value-preserving to ever fail.

I'm inclined to agree that the cast should be value-preserving rather than type-preserving. There was concern about the behavior being different on Darwin and Linux, which is why we try to be type-preserving so that pure Swift code that uses number values with Any or other polymorphic interfaces behaves consistently with Cocoa Foundation code that has to traffic in NSNumber for the same effect.

Are you saying that Swift on Darwin can’t have value-preserving behavior? It seems like I’m missing something here. If value-preserving is the desirable behavior can you elaborate on the specific challenges getting in the way of having that behavior everywhere?


(Matthew Johnson) #8

What bothers me about the current behavior is that when you have a numeric value of type `Any` its casting behavior depends on how it was constructed. This makes it easy to write code that works with some numeric values of type Any and be badly broken for others. One can argue this is ok because the values have different types underlying types, but I think it turns out to be pretty confusing and problematic for numeric types in practice.

This problem is compounded by the fact that *most* of the time when we work with opaque numeric values we’re working with actually working with values that do cast to all of the standard library numeric types, but aren’t necessarily value-preserving. They can truncate or overflow.

To clarify what bothers me about the current behavior I’ll give an example:

let json = "{ \"one\": 1, \"onePointZero\": 1.0, \"onePointOne\": 1.1, \"onePointNine\": 1.1, \"largerThanInt8Max\": 270 }"
let data = json.data(using: .utf8)!

// casts always succeed, but might truncate or overflow
//let dict = try! JSONSerialization.jsonObject(with: data, options: []) as! [String: Any]

// casts always succeed, but might truncate or overflow
//let dict: [String: Any] = ["one": 1 as NSNumber, "onePointZero": 1.0 as NSNumber, "onePointOne": 1.1 as NSNumber, "onePointNine": 1.9 as NSNumber, "largerThanInt8Max": 270 as NSNumber]

// casts always fail unless the target type matches the original type
let dict: [String: Any] = ["one": 1, "onePointZero": 1.0, "onePointOne": 1.1, "onePointNine": 1.9, "largerThanInt8Max": 270]

let oneAsDouble = dict["one"] as? Double // 1 or nil
let onePointZeroAsInt = dict["onePointZero"] as? Int // 1 or nil
let onePointOneAsInt = dict["onePointOne"] as? Int // 1 (truncated) or nil
let onePointNineAsInt = dict["onePointNine"] as? Int // 1 (truncated) or nil
let int8Overflow = dict["largerThanInt8Max"] as? Int8 // 14 (overflow) or nil

The net result of this is that it’s pretty hard to write correct code for numeric values of type Any.

The truncation and overflow are particularly troublesome because this behavior is exhibited when dealing with JSON values from external sources whose behavior could change on us leading to garbage values rather than more immediate nil values.

I would strongly prefer to see a single behavior that does not ever produce garbage values. It seems like value-preserving behavior is the only way to do that. That said, I have an open mind if there are other options. But I think we should try to do something better than we currently do.

The current behavior is very subtle, taking a nontrivial amount of effort to understand despite the fact that it’s not at all obvious that there is anything that one should pay attention to at all (I would guess that most programmers will just expect value-preserving casts until they run into a bug). IMO this is not a good thing for a use case that applies to just about every app out there.

Note: I’m only concerned with code that deals with numeric values of type Any here. If that requires a change in behavior to direct numeric type casts like your example I wouldn’t object, but I am not specifically asking for that.

Matthew

···

On Nov 10, 2016, at 2:41 PM, Joe Groff <jgroff@apple.com> wrote:

On Nov 10, 2016, at 11:51 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Nov 10, 2016, at 1:44 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Nov 10, 2016, at 11:42 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

On Nov 10, 2016, at 10:30 AM, Philippe Hausler <phausler@apple.com <mailto:phausler@apple.com>> wrote:

So I think there are a few rough edges here not just the hashing or equality. I think the issue comes down to the subclass of NSNumber that is being used - it is defeating not only hashing but also performance and allocation optimizations in Foundation.

So what would we have to do to get rid of the “type preserving” NSNumber subclass?

The type-preserving subclasses remember the exact Swift type that a value was bridged from, to preserve type specificity of casts so that e.g. `0 as Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> AnyObject round-trip involves ObjC, thereby providing somewhat more consistent behavior between Darwin and Corelibs-based Swift. If we were willing to give that up, and say that NSNumbers just flat-out lose type info and can cast back to any Swift number type, then it seems to me we could use the pure Cocoa subclasses.

Would these only be value-preserving casts and return nil if information loss would occur? I think that might be preferable anyway. Maybe I’m just not thinking hard enough, but when would the original type information be important as long as information isn’t lost? When I interact with these casts and NSNumber (JSON parsing, etc) I generally *do not* want an attempted cast that would be value-preserving to ever fail.

I'm inclined to agree that the cast should be value-preserving rather than type-preserving. There was concern about the behavior being different on Darwin and Linux, which is why we try to be type-preserving so that pure Swift code that uses number values with Any or other polymorphic interfaces behaves consistently with Cocoa Foundation code that has to traffic in NSNumber for the same effect.

Are you saying that Swift on Darwin can’t have value-preserving behavior? It seems like I’m missing something here. If value-preserving is the desirable behavior can you elaborate on the specific challenges getting in the way of having that behavior everywhere?

It would require a change to the type relationships between the number value types on Swift. They are currently plain, unrelated struct types, so you can't do something like:

  let x: Int = 1
  x as? Float // produces nil

We could conceivably special-case the number types in Swift so that they do value-preserving casts, and maybe that's even a good idea, but we don't today.