Remove AnyObject Constraint for Objective-C Lightweight Generics

Hi Swift-Evolution,

Back when SE-0057 (https://github.com/apple/swift-evolution/blob/master/proposals/0057-importing-objc-generics.md) was proposed, it included the following passage:

The generic type parameters in Swift will always be class-bound, i.e., the generic class will have the requirement T : AnyObject.
This made sense at the time, since Swift <-> Objective-C interoperability was only possible with class types (AnyObject). However, several months after SE-0057 was accepted, SE-0116 (https://github.com/apple/swift-evolution/blob/master/proposals/0116-id-as-any.md) was accepted, which allowed for bridging any type to Objective-C, not just class types.

This greatly improved interoperability between Swift and Objective-C code, but the AnyObject restriction on Objective-C generics remained. This issue is especially apparent when using lesser-known Objective-C collection types such as NSCache, where it may make sense to store value types or use value types as the keys, but the compiler does not allow it.

I propose that this restriction is lifted, and that generic Objective-C parameters are no longer restricted to conforming to AnyObject. I’m assuming this is not as straightforward as it might seem at first to implement, but I think the benefits would make the effort worth it, since this seems like an overlooked case and not intentionally kept this way.

Thoughts?

In principle it makes sense, but there are implementation challenges we didn't have time to consider. It would be nice to make this happen when we have the time to make it work.

-Joe

···

On Nov 8, 2017, at 11:49 AM, Riley Testut via swift-evolution <swift-evolution@swift.org> wrote:

Hi Swift-Evolution,

Back when SE-0057 (https://github.com/apple/swift-evolution/blob/master/proposals/0057-importing-objc-generics.md) was proposed, it included the following passage:

  • The generic type parameters in Swift will always be class-bound, i.e., the generic class will have the requirement T : AnyObject.
This made sense at the time, since Swift <-> Objective-C interoperability was only possible with class types (AnyObject). However, several months after SE-0057 was accepted, SE-0116 (https://github.com/apple/swift-evolution/blob/master/proposals/0116-id-as-any.md) was accepted, which allowed for bridging any type to Objective-C, not just class types.

This greatly improved interoperability between Swift and Objective-C code, but the AnyObject restriction on Objective-C generics remained. This issue is especially apparent when using lesser-known Objective-C collection types such as NSCache, where it may make sense to store value types or use value types as the keys, but the compiler does not allow it.

I propose that this restriction is lifted, and that generic Objective-C parameters are no longer restricted to conforming to AnyObject. I’m assuming this is not as straightforward as it might seem at first to implement, but I think the benefits would make the effort worth it, since this seems like an overlooked case and not intentionally kept this way.

Thoughts?

I find more limiting the ability not to declare @objc a property of a class that doesn't inherit from NSObject:

class Foo {
  var x: Int = 0
}

class Bar {
  var foo = Foo() // Why shouldn't we allow this @objc?
}

or

protocol BarProtocol: AnyObject {
  func fooDidBar(_ foo: Foo)
}

class BarContoller: NSViewController {
  @IBOutlet weak var delegate: BarProtocol? // Error - not expressible in ObjC.
}

While Foo doesn't inherit from NSObject, it still can have @objc members used for binding. I have quite few classes that need to inherit from NSObject just for that purpose.

I fully understand the logic behind it that it needs to be exported to ObjC headers and runtime metadata, but I'd prefer these being exported as (id) into ObjC rather than not allowing this at all as these are all objects that can be passed into the ObjC world...

···

On Nov 8, 2017, at 8:49 PM, Riley Testut via swift-evolution <swift-evolution@swift.org> wrote:

Hi Swift-Evolution,

Back when SE-0057 (https://github.com/apple/swift-evolution/blob/master/proposals/0057-importing-objc-generics.md) was proposed, it included the following passage:

The generic type parameters in Swift will always be class-bound, i.e., the generic class will have the requirement T : AnyObject.
This made sense at the time, since Swift <-> Objective-C interoperability was only possible with class types (AnyObject). However, several months after SE-0057 was accepted, SE-0116 (https://github.com/apple/swift-evolution/blob/master/proposals/0116-id-as-any.md) was accepted, which allowed for bridging any type to Objective-C, not just class types.

This greatly improved interoperability between Swift and Objective-C code, but the AnyObject restriction on Objective-C generics remained. This issue is especially apparent when using lesser-known Objective-C collection types such as NSCache, where it may make sense to store value types or use value types as the keys, but the compiler does not allow it.

I propose that this restriction is lifted, and that generic Objective-C parameters are no longer restricted to conforming to AnyObject. I’m assuming this is not as straightforward as it might seem at first to implement, but I think the benefits would make the effort worth it, since this seems like an overlooked case and not intentionally kept this way.

Thoughts?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I have personally filed a few bugs on this; and I definitely consider it a bug that we cannot store Any in generics for objc. There are however some problem areas that might be worth considering while fixing this bug.

1) We need to ensure this does not cause source churn - I would expect swift 4 to be source compatible with swift 5.

2) There are a few cases that might be a bit cagey - you claim NSCache, but would it be surprising that the boxed object having no refs gets purged? How bout NSPointerArray?

3) Since Foundation is likely the most impact here I think it would be useful to audit the results of this before pushing it out; specifically the Foundation internal builds so that we can make sure the things we are working on function correctly.

Do you have implementations in the works yet? I really think this is important for us to get in (especially before the ABI gets locked down cause it could have impact there…)

···

On Nov 8, 2017, at 11:49 AM, Riley Testut via swift-evolution <swift-evolution@swift.org> wrote:

Hi Swift-Evolution,

Back when SE-0057 (https://github.com/apple/swift-evolution/blob/master/proposals/0057-importing-objc-generics.md) was proposed, it included the following passage:

The generic type parameters in Swift will always be class-bound, i.e., the generic class will have the requirement T : AnyObject.
This made sense at the time, since Swift <-> Objective-C interoperability was only possible with class types (AnyObject). However, several months after SE-0057 was accepted, SE-0116 (https://github.com/apple/swift-evolution/blob/master/proposals/0116-id-as-any.md) was accepted, which allowed for bridging any type to Objective-C, not just class types.

This greatly improved interoperability between Swift and Objective-C code, but the AnyObject restriction on Objective-C generics remained. This issue is especially apparent when using lesser-known Objective-C collection types such as NSCache, where it may make sense to store value types or use value types as the keys, but the compiler does not allow it.

I propose that this restriction is lifted, and that generic Objective-C parameters are no longer restricted to conforming to AnyObject. I’m assuming this is not as straightforward as it might seem at first to implement, but I think the benefits would make the effort worth it, since this seems like an overlooked case and not intentionally kept this way.

Thoughts?
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

+1000. In addition to the issues raised above, this limitation wreaks havoc with my attempts to properly support KVO in Swift projects. You either have to create a second Any-typed property, bloating the interface, or you have to go with the old string-based key path API, which also locks you out of using the Swift 4 closure-based observers.

Charles

···

On Nov 8, 2017, at 10:51 PM, Charlie Monroe via swift-evolution <swift-evolution@swift.org> wrote:

I find more limiting the ability not to declare @objc a property of a class that doesn't inherit from NSObject:

class Foo {
  var x: Int = 0
}

class Bar {
  var foo = Foo() // Why shouldn't we allow this @objc?
}

or

protocol BarProtocol: AnyObject {
  func fooDidBar(_ foo: Foo)
}

class BarContoller: NSViewController {
  @IBOutlet weak var delegate: BarProtocol? // Error - not expressible in ObjC.
}

While Foo doesn't inherit from NSObject, it still can have @objc members used for binding. I have quite few classes that need to inherit from NSObject just for that purpose.

I fully understand the logic behind it that it needs to be exported to ObjC headers and runtime metadata, but I'd prefer these being exported as (id) into ObjC rather than not allowing this at all as these are all objects that can be passed into the ObjC world...

Just out of curiosity, what were some of those implementation challenges? And have any of these changed with the evolution of the standard library since then?

···

On Nov 8, 2017, at 11:51 AM, Joe Groff <jgroff@apple.com> wrote:

In principle it makes sense, but there are implementation challenges we didn't have time to consider. It would be nice to make this happen when we have the time to make it work.

I have personally filed a few bugs on this; and I definitely consider it a bug that we cannot store Any in generics for objc. There are however some problem areas that might be worth considering while fixing this bug.

1) We need to ensure this does not cause source churn - I would expect swift 4 to be source compatible with swift 5.

Agreed. I'd be surprised if this would cause churn though, since this is effectively just loosening a restriction, and all existing use cases would still be allowed.

2) There are a few cases that might be a bit cagey - you claim NSCache, but would it be surprising that the boxed object having no refs gets purged? How bout NSPointerArray?

I agree there are certain cases where true reference semantics are important, and off the top of my head I can think of two (relatively easy) ways we could accommodate this:

1) The Objective-C class declaration explicitly specifies an upper-bound of NSObject (or NSObjectProtocol).
2) Add a new keyword (similar to existing __covariant and __contravariant keywords) such as __reference (where the final name would of course be bike-shedded)

I’m leaning towards an approach similar to 2) since 1) might be confusing to newcomers due to it seemingly have no purpose considering the NSObject constraint would implicitly there where using the generic class from Objective-C code.

I’m not familiar with NSCache’s internals, so I wasn’t aware references play a role in whether or not NSCache purges an object. That being said, I don’t think it would be surprising if NSCache purged a large Data value under memory pressure, as long as it didn’t affect any “copies” I had retrieved and was currently using.

As for NSPointerArray, we’d still need to get Objective-C generics for it first 😉 Though assuming that is added, the generic parameter would need to explicitly say it requires a reference.

3) Since Foundation is likely the most impact here I think it would be useful to audit the results of this before pushing it out; specifically the Foundation internal builds so that we can make sure the things we are working on function correctly.

Do you have implementations in the works yet? I really think this is important for us to get in (especially before the ABI gets locked down cause it could have impact there…)

No I don’t, but would be open to digging into it and seeing what I could do as a proof-of-concept (I just don’t know where I’d start looking to accomplish this).

···

On Nov 9, 2017, at 9:01 AM, Philippe Hausler <phausler@apple.com> wrote:

I have personally filed a few bugs on this; and I definitely consider it a bug that we cannot store Any in generics for objc. There are however some problem areas that might be worth considering while fixing this bug.

1) We need to ensure this does not cause source churn - I would expect swift 4 to be source compatible with swift 5.

Agreed. I'd be surprised if this would cause churn though, since this is effectively just loosening a restriction, and all existing use cases would still be allowed.

2) There are a few cases that might be a bit cagey - you claim NSCache, but would it be surprising that the boxed object having no refs gets purged? How bout NSPointerArray?

I agree there are certain cases where true reference semantics are important, and off the top of my head I can think of two (relatively easy) ways we could accommodate this:

1) The Objective-C class declaration explicitly specifies an upper-bound of NSObject (or NSObjectProtocol).
2) Add a new keyword (similar to existing __covariant and __contravariant keywords) such as __reference (where the final name would of course be bike-shedded)

I’m leaning towards an approach similar to 2) since 1) might be confusing to newcomers due to it seemingly have no purpose considering the NSObject constraint would implicitly there where using the generic class from Objective-C code.

Option 2 may take a lot of effort just as a heads up since that will mean that we would need to audit the entire macOS, iOS, tvOS, and watchOS SDKs and find any edge cases (my guess is very very few and perhaps only Foundation)

I’m not familiar with NSCache’s internals, so I wasn’t aware references play a role in whether or not NSCache purges an object. That being said, I don’t think it would be surprising if NSCache purged a large Data value under memory pressure, as long as it didn’t affect any “copies” I had retrieved and was currently using.

As for NSPointerArray, we’d still need to get Objective-C generics for it first :wink: Though assuming that is added, the generic parameter would need to explicitly say it requires a reference.

NSMapTable or NSHashTable might be better examples; if the key or value is weakly stored the translated reference will drop off if the held structure is mutated.
e.g.

var m = NSMapTable<NSString, NSData>(keyOptions: [.copyIn, .objectPersonality], valueOptions: [.weakMemory, .objectPersonality])

var key = "hello" as NSString
var value = Data(bytes: [0, 1, 2, 3]) as NSData

m.setObject(value, forKey: key)

assert(m.object(forKey: key) != nil)

That works as expected - so lets change it to a structure

var m = NSMapTable<String, Data>(keyOptions: [.copyIn, .objectPersonality], valueOptions: [.weakMemory, .objectPersonality])

var key = "hello"
var value = Data(bytes: [0, 1, 2, 3])

m.setObject(value, forKey: key) // after here there are no more references to value so it is destroyed

assert(m.object(forKey: key) != nil) // this now fails

In that second example even if you had a usage of value past the setObject method call that was a mutation it would also fail the assert because the backing reference would have changed.

I guess what I am saying is there are edge cases that we have to be careful with.

3) Since Foundation is likely the most impact here I think it would be useful to audit the results of this before pushing it out; specifically the Foundation internal builds so that we can make sure the things we are working on function correctly.

Do you have implementations in the works yet? I really think this is important for us to get in (especially before the ABI gets locked down cause it could have impact there…)

No I don’t, but would be open to digging into it and seeing what I could do as a proof-of-concept (I just don’t know where I’d start looking to accomplish this).

Dont get me wrong; I think this is a great idea and vastly improves the state of affairs – imho it is very well worth doing and getting this done before we cannot change it.

···

On Nov 29, 2017, at 2:07 PM, Riley Testut <rileytestut@gmail.com> wrote:

On Nov 9, 2017, at 9:01 AM, Philippe Hausler <phausler@apple.com <mailto:phausler@apple.com>> wrote:

One issue is that it would require a different boxing strategy for unbridged types. Right now, we have a single ObjC class in the Swift runtime for this purpose (you may have seen objects of class _SwiftValue coming from Swift value types in ObjC). If Swift value types can conform to ObjC protocols, then we may need to instantiate a boxing ObjC class per conforming value type on which to hang those conformances. We wouldn't want to emit an ObjC class for every type declaration in Swift, because that'd have significant metadata size and startup time costs, but since modules can extend types that don't belong to them to conform to additional protocols, we can't know with confidence whether a type needs an objc class for this purpose, and thanks to dlopen, that may even change at runtime. Also, since ObjC generics are based on type erasure, but bridging in some situations relies on reified runtime type information, we need to think through whether it's possible in all situations you'd call an ObjC generic from Swift to keep the type information around on the Swift side to bridge outputs back.

-Joe

···

On Nov 29, 2017, at 2:06 PM, Riley Testut <rileytestut@gmail.com> wrote:

On Nov 8, 2017, at 11:51 AM, Joe Groff <jgroff@apple.com> wrote:

In principle it makes sense, but there are implementation challenges we didn't have time to consider. It would be nice to make this happen when we have the time to make it work.

Just out of curiosity, what were some of those implementation challenges? And have any of these changed with the evolution of the standard library since then?