Casting shadow protocols

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types defined in foundation: NSString, NSArray, NSDictionary, etc. But core is imported by Foundation, so it can’t (circular references between modules). Thankfully, everything in ObjC is pretty opaquely defined and uniform, and Swift knows how to hook into that uniform layout. So the core library defines Shadow Protocols which provide whatever subset of that type’s API is considered interesting. These protocols are then used in place of the ObjC types. There’s also some magic compiler hooks so core lib types can subclass those foundation types.

However there’s sometimes two Shadow Protocols: one that defines the APIs the stdlib should provide (_NSFooCore), and one that extends that with extra APIs the stdlib wants to consume (_NSFoo). This leads to an awkward situation: as far as the runtime is concerned, the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do because it’s all just a big lie to hook into ObjC message passing with a bit of type safety, but the runtime doesn’t. So if you try to do a safe type cast, it will fail. This leads to a situation where we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive refactoring hazard.

For instance, there was a struct-containing-a-class that was being cast to _NSFoo in HashedCollections. This happened to work (but was probably still a violation of strict aliasing?) because the struct’s only field was the class. However the struct was later changed to a class, which silently made the cast completely incorrect, banishing the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where we also cast an AnyObject into an _NSFoo, but there’s some chance this is all legacy junk that can be updated to at least use _NSFooCore, if not _NSFoo itself.

If the casts are always in one direction, can you make one protocol refine another?

Also note that @objc protocols are self-conforming as long as they don’t contain initializers or static methods, but I’m not sure if that helps.

···

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types defined in foundation: NSString, NSArray, NSDictionary, etc. But core is imported by Foundation, so it can’t (circular references between modules). Thankfully, everything in ObjC is pretty opaquely defined and uniform, and Swift knows how to hook into that uniform layout. So the core library defines Shadow Protocols which provide whatever subset of that type’s API is considered interesting. These protocols are then used in place of the ObjC types. There’s also some magic compiler hooks so core lib types can subclass those foundation types.

However there’s sometimes two Shadow Protocols: one that defines the APIs the stdlib should provide (_NSFooCore), and one that extends that with extra APIs the stdlib wants to consume (_NSFoo). This leads to an awkward situation: as far as the runtime is concerned, the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do because it’s all just a big lie to hook into ObjC message passing with a bit of type safety, but the runtime doesn’t. So if you try to do a safe type cast, it will fail. This leads to a situation where we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive refactoring hazard.

For instance, there was a struct-containing-a-class that was being cast to _NSFoo in HashedCollections. This happened to work (but was probably still a violation of strict aliasing?) because the struct’s only field was the class. However the struct was later changed to a class, which silently made the cast completely incorrect, banishing the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where we also cast an AnyObject into an _NSFoo, but there’s some chance this is all legacy junk that can be updated to at least use _NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

···

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org> wrote:

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

The casting is largely just to get the compiler to let you call ObjC methods on an object reference, right? We could sidestep the undefined-ness of unsafeBitCast perhaps by just giving the standard library a `Builtin.objc_msgSend` that takes a selector and a bag of arguments, and performs the method call with ObjC method semantics.

-Joe

···

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types defined in foundation: NSString, NSArray, NSDictionary, etc. But core is imported by Foundation, so it can’t (circular references between modules). Thankfully, everything in ObjC is pretty opaquely defined and uniform, and Swift knows how to hook into that uniform layout. So the core library defines Shadow Protocols which provide whatever subset of that type’s API is considered interesting. These protocols are then used in place of the ObjC types. There’s also some magic compiler hooks so core lib types can subclass those foundation types.

However there’s sometimes two Shadow Protocols: one that defines the APIs the stdlib should provide (_NSFooCore), and one that extends that with extra APIs the stdlib wants to consume (_NSFoo). This leads to an awkward situation: as far as the runtime is concerned, the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do because it’s all just a big lie to hook into ObjC message passing with a bit of type safety, but the runtime doesn’t. So if you try to do a safe type cast, it will fail. This leads to a situation where we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive refactoring hazard.

For instance, there was a struct-containing-a-class that was being cast to _NSFoo in HashedCollections. This happened to work (but was probably still a violation of strict aliasing?) because the struct’s only field was the class. However the struct was later changed to a class, which silently made the cast completely incorrect, banishing the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where we also cast an AnyObject into an _NSFoo, but there’s some chance this is all legacy junk that can be updated to at least use _NSFooCore, if not _NSFoo itself.

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

···

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:
on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

···

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:
on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

Does _unsafeReferenceCast at least verify that the types in question could theoretically be cast into each other? That is, one is derived from the other? If so, that would probably be an acceptable improvement. (the best we could ever hope to do?)

···

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

I am confused. Then why can't you just do an unconditional checked down cast to _NSFoo from _NSFooCore? That can definitely be expressed at the SIL level?

···

On Nov 7, 2016, at 11:23 AM, Alexis via swift-dev <swift-dev@swift.org> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Does _unsafeReferenceCast at least verify that the types in question
could theoretically be cast into each other? That is, one is derived
from the other?

Not IIRC

If so, that would probably be an acceptable improvement. (the best we
could ever hope to do?)

It might be, if that's the only way it's being used.

···

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

--
-Dave

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

I am confused. Then why can't you just do an unconditional checked down cast to _NSFoo from
_NSFooCore? That can definitely be expressed at the SIL level?

The check will fail. There is no actual relationship between the type
and the protocol; we are just using the protocol to call some ObjC
methods without being able to see the actual type's declaration.

···

on Tue Nov 08 2016, Michael Gottesman <mgottesman-AT-apple.com> wrote:

On Nov 7, 2016, at 11:23 AM, Alexis via swift-dev <swift-dev@swift.org> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org > <mailto:swift-dev@swift.org>> wrote:
on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

Does _unsafeReferenceCast at least verify that the types in question could theoretically be cast into each other? That is, one is derived from the other? If so, that would probably be an acceptable improvement. (the best we could ever hope to do?)

This is what I call lying about types. _unsafeReferenceCast does not and should not support that. It *should* actually have sanity checks to catch such incorrect usage. The checks aren’t implemented yet. Instead you may end up with a crash in the runtime.

unsafeBitCast is the way to lie about a type. It only works when
- dealing with a @objc class protocol
- we guarantee that the mistyped reference will never be dynamically cast or accessed in any way other than msgSend

Those conditions make it immune from struct aliasing based on a technicality. It does encourage bad practice, and our memory model documentation now needs to make exceptions for this special, confusing case.

I like JoeG’s idea of a msgSend builtin. I’m just not sure that would completely obsolete shadow protocols. Are we also using them to satisfy protocol requirements?

-Andy

···

On Nov 7, 2016, at 12:15 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Does _unsafeReferenceCast at least verify that the types in question could theoretically be cast into each other? That is, one is derived from the other? If so, that would probably be an acceptable improvement. (the best we could ever hope to do?)

This is what I call lying about types. _unsafeReferenceCast does not and should not support that. It *should* actually have sanity checks to catch such incorrect usage. The checks aren’t implemented yet. Instead you may end up with a crash in the runtime.

unsafeBitCast is the way to lie about a type. It only works when
- dealing with a @objc class protocol
- we guarantee that the mistyped reference will never be dynamically cast or accessed in any way other than msgSend

Those conditions make it immune from struct aliasing based on a technicality. It does encourage bad practice, and our memory model documentation now needs to make exceptions for this special, confusing case.

I like JoeG’s idea of a msgSend builtin. I’m just not sure that would completely obsolete shadow protocols. Are we also using them to satisfy protocol requirements?

Once “inside” the realm of shadow protocols, they do provide some decent static type safety guarantees. They provide an assertion that the NSFooCore API is satisfied by all the types we expect. It also provides a nice single place for us to declare the APIs we’re interested in. What does messing up a msgSend imply? Is it type-safe? Do I just get a “no such message” crash at runtime?

···

On Nov 8, 2016, at 6:22 PM, Andrew Trick <atrick@apple.com> wrote:

On Nov 7, 2016, at 12:15 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

-Andy

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

The msg send built in is worse than exploiting AnyObject magic. I'd rather fall back to that

···

Sent from my moss-covered three-handled family gradunza

On Nov 8, 2016, at 4:34 PM, Alexis <abeingessner@apple.com> wrote:

On Nov 8, 2016, at 6:22 PM, Andrew Trick <atrick@apple.com> wrote:

On Nov 7, 2016, at 12:15 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

Does _unsafeReferenceCast at least verify that the types in question could theoretically be cast into each other? That is, one is derived from the other? If so, that would probably be an acceptable improvement. (the best we could ever hope to do?)

This is what I call lying about types. _unsafeReferenceCast does not and should not support that. It *should* actually have sanity checks to catch such incorrect usage. The checks aren’t implemented yet. Instead you may end up with a crash in the runtime.

unsafeBitCast is the way to lie about a type. It only works when
- dealing with a @objc class protocol
- we guarantee that the mistyped reference will never be dynamically cast or accessed in any way other than msgSend

Those conditions make it immune from struct aliasing based on a technicality. It does encourage bad practice, and our memory model documentation now needs to make exceptions for this special, confusing case.

I like JoeG’s idea of a msgSend builtin. I’m just not sure that would completely obsolete shadow protocols. Are we also using them to satisfy protocol requirements?

Once “inside” the realm of shadow protocols, they do provide some decent static type safety guarantees. They provide an assertion that the NSFooCore API is satisfied by all the types we expect. It also provides a nice single place for us to declare the APIs we’re interested in. What does messing up a msgSend imply? Is it type-safe? Do I just get a “no such message” crash at runtime?

-Andy

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Does _unsafeReferenceCast at least verify that the types in
question could theoretically be cast into each other? That is, one
is derived from the other? If so, that would probably be an
acceptable improvement. (the best we could ever hope to do?)

This is what I call lying about types. _unsafeReferenceCast does not
and should not support that. It *should* actually have sanity checks
to catch such incorrect usage. The checks aren’t implemented
yet. Instead you may end up with a crash in the runtime.

unsafeBitCast is the way to lie about a type. It only works when
- dealing with a @objc class protocol
- we guarantee that the mistyped reference will never be dynamically cast or accessed in any way

other than msgSend

Those conditions make it immune from struct aliasing based on a
technicality. It does encourage bad practice, and our memory model
documentation now needs to make exceptions for this special,
confusing case.

I like JoeG’s idea of a msgSend builtin. I’m just not sure that
would completely obsolete shadow protocols. Are we also using them
to satisfy protocol requirements?

Once “inside” the realm of shadow protocols, they do provide some
decent static type safety guarantees. They provide an assertion that
the NSFooCore API is satisfied by all the types we expect. It also
provides a nice single place for us to declare the APIs we’re
interested in. What does messing up a msgSend imply? Is it type-safe?

No, and if you send the same message from multiple places you have to
make sure you get it right in all those places. That's why I'd *much*
rather just declare the shadow protocols, then cast our things to
AnyObject and use the magic .anyDeclaredObjCThing accessor from
AnyObject to send the right messages. We could declare the Swift names
of these methods to be quite distinct (e.g. start with "_swift_") and
give them the right objC selectors with an explicit @objc(selectorname)
declaration, to help ensure that we're not picking up the wrong API.

···

on Tue Nov 08 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 8, 2016, at 6:22 PM, Andrew Trick <atrick@apple.com> wrote:

On Nov 7, 2016, at 12:15 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

Do I just get a “no such message” crash at runtime?

-Andy

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
-Dave

The msg send built in is worse than exploiting AnyObject magic. I'd rather fall back to that

As a builtin, we could probably constrain it to be type safe. The AnyObject magic implementation right now implies a bunch of optional meddling that I don't think we optimize away well, though it does have the benefit of already being readily available. Doug also still wants to kill it some day…

-Joe

···

On Nov 8, 2016, at 4:41 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

Sent from my moss-covered three-handled family gradunza

On Nov 8, 2016, at 4:34 PM, Alexis <abeingessner@apple.com> wrote:

On Nov 8, 2016, at 6:22 PM, Andrew Trick <atrick@apple.com> wrote:

On Nov 7, 2016, at 12:15 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

Does _unsafeReferenceCast at least verify that the types in question could theoretically be cast into each other? That is, one is derived from the other? If so, that would probably be an acceptable improvement. (the best we could ever hope to do?)

This is what I call lying about types. _unsafeReferenceCast does not and should not support that. It *should* actually have sanity checks to catch such incorrect usage. The checks aren’t implemented yet. Instead you may end up with a crash in the runtime.

unsafeBitCast is the way to lie about a type. It only works when
- dealing with a @objc class protocol
- we guarantee that the mistyped reference will never be dynamically cast or accessed in any way other than msgSend

Those conditions make it immune from struct aliasing based on a technicality. It does encourage bad practice, and our memory model documentation now needs to make exceptions for this special, confusing case.

I like JoeG’s idea of a msgSend builtin. I’m just not sure that would completely obsolete shadow protocols. Are we also using them to satisfy protocol requirements?

Once “inside” the realm of shadow protocols, they do provide some decent static type safety guarantees. They provide an assertion that the NSFooCore API is satisfied by all the types we expect. It also provides a nice single place for us to declare the APIs we’re interested in. What does messing up a msgSend imply? Is it type-safe? Do I just get a “no such message” crash at runtime?

-Andy

On Nov 7, 2016, at 2:30 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Nov 07 2016, Alexis <abeingessner-AT-apple.com> wrote:

On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev <swift-dev@swift.org> wrote:

on Fri Nov 04 2016, Slava Pestov <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/&gt;&gt; wrote:

If the casts are always in one direction, can you make one protocol
refine another?

Yeah, I am shocked if they don't do that already.

They do; _NSFoo: _NSFooCore

But the problem is that we have _NSFooCores and want _NSFoos.

Right. Then you need type punning without any dynamic checks,
a.k.a. _unsafeReferenceCast

Also note that @objc protocols are self-conforming as long as they
don’t contain initializers or static methods, but I’m not sure if that
helps.

Doesn't; we're not using these in a generic context; they're just
existentials.

On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev <swift-dev@swift.org> wrote:

The swift standard library has this nasty little pattern/problem in it:

The types in the core library want to know about several types
defined in foundation: NSString, NSArray, NSDictionary, etc. But
core is imported by Foundation, so it can’t (circular references
between modules). Thankfully, everything in ObjC is pretty opaquely
defined and uniform, and Swift knows how to hook into that uniform
layout. So the core library defines Shadow Protocols which provide
whatever subset of that type’s API is considered interesting. These
protocols are then used in place of the ObjC types. There’s also
some magic compiler hooks so core lib types can subclass those
foundation types.

However there’s sometimes two Shadow Protocols: one that defines the
APIs the stdlib should provide (_NSFooCore), and one that extends
that with extra APIs the stdlib wants to consume (_NSFoo). This
leads to an awkward situation: as far as the runtime is concerned,
the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
because it’s all just a big lie to hook into ObjC message passing
with a bit of type safety, but the runtime doesn’t. So if you try to
do a safe type cast, it will fail. This leads to a situation where
we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
refactoring hazard.

For instance, there was a struct-containing-a-class that was being
cast to _NSFoo in HashedCollections. This happened to work (but was
probably still a violation of strict aliasing?) because the struct’s
only field was the class. However the struct was later changed to a
class, which silently made the cast completely incorrect, banishing
the real _NSFoo to the shadow (protocol) realm.

Can we do anything better here? Note that there’s a few places where
we also cast an AnyObject into an _NSFoo, but there’s some chance
this is all legacy junk that can be updated to at least use
_NSFooCore, if not _NSFoo itself.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

<https://lists.swift.org/mailman/listinfo/swift-dev&gt;

--
-Dave

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev