[Pitch] Reflection

[aheze (Andrew Zheng)]

+1, looks really cool. Not sure if this proposal covers it, but will this let us see the case labels of system enums? For example, this code prints out ContentMode is: UIViewContentMode.

let imageView = UIImageView()
let contentMode = imageView.contentMode
print("ContentMode is: \(contentMode)")

To see what contentMode actually is, I'd have to do a huge switch.

switch contentMode {
case .scaleToFill:
    print("mode is scaleToFill")
case .scaleAspectFit:
    print("mode is scaleAspectFit")
case .scaleAspectFill:
    print("mode is scaleAspectFill")
case .redraw:
    print("mode is redraw")
case .center:
    print("mode is center")
case .top:
    print("mode is top")
case .bottom:
    print("mode is bottom")
case .left:
    print("mode is left")
case .right:
    print("mode is right")
case .topLeft:
    print("mode is topLeft")
case .topRight:
    print("mode is topRight")
case .bottomLeft:
    print("mode is bottomLeft")
case .bottomRight:
    print("mode is bottomRight")
@unknown default:
    print("default")
}

Will Reflection help with this, or is it an entirely different problem? Again, pitch sounds great. Nice work!

(originally asked on Stack Overflow: ios - Printing description of system enum value only prints the enum name - Stack Overflow)

[Joakim_Hassila1 (Joakim Hassila)]

Very happy to see this capability improved!

Just had a quick read-through, will try to provide more careful feedback later.

Just one short question:

@Alejandro

extension Field {
  /// If this field is a struct or class stored property, this will return true
  /// if the field in question was syntactically marked 'var' and false if it
  /// marked 'let'. For a tuple element, this will always be false.
  @inlinable
  public var isVar: Bool { get }

Wouldn't it be natural to have this as an enum instead?

At the point of usage, it may be that one would like to check for e.g. let constants and then the negative check is not perhaps natural.

E.g.

enum StoredPropertyType {
  case `var`
  case `let`
  case tuple // ? 
}

?

[dnadoba (David Nadoba)]

@Alejandro

The order of generic arguments start from the outermost type and works its way
inwards. Consider the following scenario:

struct ABC<A, B: Equatable> {
  struct XYZ<C: Hashable, D> {}
}

Effectively XYZ has 4 generic arguments needed to fully realize it. Given
the following list of types: [Int, String, Double, Array<Float>], we start
with the outermost type, ABC, and assign the following generic arguments:
<A = Int, B = String> and lookup the Equatable conformance for String. Now
for XYZ, we finish the generic assignment with the following:
<C = Double, D = Array<Float>> and lookup Double's Hashable.

I'm wondering how this will play with variadic generics.
For example:

struct ABC<A..., B: Equatable> {
  struct XYZ<C: Hashable, D...> {}
}

let partial = Type(ABC<Int, Int, Int>.XYZ<Int, Int>.self).partial 

Does create(with:) support different shapes of A and D or just the shape specified during the creation of the partial type e.g. 2 for A and 1 for D in the example above?

[mickeyl (Dr. Mickey Lauer)]

What an amazing pitch! Thanks for bringing reflection in Swift forward.

[saagarjha]

As someone who has done some pretty horrible things with the Swift runtime because these APIs were missing, I am very pleased to see their addition to the language. Two very short comments from skimming the proposal:

@Alejandro

Returns a collection wrapper over 'Field'

I think it would also be nice to have APIs that could do lookup by name, and (perhaps) by type. In particular these would be slightly faster than iterating the entire list of children for large types, which I have found to be a performance pitfall in several apps.

@Alejandro

We could allow returning mutable keypaths to these, but there's no way to
opt out of that with this current API for your types which is a potential
security concern.

I take mild umbrage at this being called a "security concern" but do want to bring up that (to my knowledge) there is no way to fundamentally prevent the creation of a WwritableKeyPath to a mutable field in the language. I would be all for a future API that helped add hints for which fields were advertised as mutable and perhaps also vend suggestions as to which mutable ones should not be touched, though :)

@Alejandro

Having a generic argument represent the type that we're reflecting over has some
benefits over the erased version.

I think you've identified that it would be useful to have both of these, perhaps we should include both and make it possible to translate between the two?

[Alejandro (Alejandro Alonso)]

@saagarjha

there is no way to fundamentally prevent the creation of a WwritableKeyPath to a mutable field in the language

Yeah, I agree with you here. I think the thing we considered was that we don't want the easy API to be able to easily vend these mutable keypaths without any way of opting in/out.

@saagarjha

I think it would also be nice to have APIs that could do lookup by name, and (perhaps) by type. In particular these would be slightly faster than iterating the entire list of children for large types, which I have found to be a performance pitfall in several apps.

We could make an API to get a field from name, however the implementation would still need to iterate the list of fields to find the specific field (just like how one would do it now). However, unlike Mirror, iterating the collection of Fields is significantly faster because of some of the performance benefits I mentioned when comparing Mirror (and all of the collection wrappers are RandomAccessCollection). All of the APIs on Field are lazily retrieved unlike the children of Mirror. @Karoy_Lorentey mentioned an API synopsis for things like these collections wrappers that I would be more than happy to add.

@dnadoba

I'm wondering how this will play with variadic generics.
For example:

struct ABC<A..., B: Equatable> {
  struct XYZ<C: Hashable, D...> {}
}

let partial = Type(ABC<Int, Int, Int>.XYZ<Int, Int>.self).partial 

Does create(with:) support different shapes of A and D or just the shape specified during the creation of the partial type e.g. 2 for A and 1 for D in the example above?

A lot about variadic generics is still unknown at the runtime layer besides what @John_McCall posted here: ABI for variadic generics, so I'm unsure how it'll look and interact with the language at runtime for reflection purposes. That being said, I plan to support variadic generics however much I can with these new APIs (or propose new APIs if it makes sense).

[Alejandro (Alejandro Alonso)]

@aheze

+1, looks really cool. Not sure if this proposal covers it, but will this let us see the case labels of system enums? For example, this code prints out ContentMode is: UIViewContentMode.

let imageView = UIImageView()
let contentMode = imageView.contentMode
print("ContentMode is: \(contentMode)")

Unfortunately, Objective-C enums do not get their case names emitted for reflection purposes, so interacting with these enums will still result in empty names.

[Alejandro (Alejandro Alonso)]

@rnantes

However, I am wondering if you would be able to differentiate between static and instance variables of a class/struct such as tableName in my example.

At the moment, we'd only be able to inspect instance variables because static ones are not emitted for reflective purposes. It could be something we add later on as mentioned in future directions.

[Slava_Pestov]

@Alejandro

PartialType represents a somewhat new concept to Swift developers. Imagine you
want to compare two types who are generic over <T>, but you don't care if both
types share the same type for T, and only care if the enclosing type is the
same. That's what a PartialType helps you achieve. It represents Array, not
Array<Int> for example.

This pitch looks great, I only have a minor aesthetic comment. I think "partial type" is a non-standard term for this, can we call it a "type constructor" or "generic type" or something instead?

Eventually it would be great to see APIs to read other kinds of reflection information we already emit:

• protocol requirements
• requirement signature of a protocol (which includes inherited protocols)
• class methods
• superclass of a class
• generic requirements of a generic type
• protocol conformance records
• conditional requirements of a conformance

[Max_Desiatov]

@Slava_Pestov

can we call it a "type constructor" or "generic type" or something instead?

+1 for using "type constructor" as a term of art instead of "partial type"

[Philippe_Hausler]

This is most definitely an ergonomic improvement and a functionality improvement that will allow some really great stuff to be written. +1

Is there space for consideration for reflection into key paths? For example recently I have found the need to split up a key path into its constituent components and iterate down them. I think there are probably a few nifty tools we could use in that range of thing. Or is that out of scope of this?

[AlexanderM (Alexander Momchilov)]

There are some existing (fairly popular, if GitHub stars are anything to go by) reflection libraries that were made by the community.

Could we do a comparison of design choices, names, etc. and see if there's anything we can learn from their first-mover positions?

• 600 Zewo/Reflection (Now deprecated, but was popular nonetheless)
• 234 Azoy/Echo
• ... any others?

If anybody has used any of these (or any others), it would be valuable to hear what worked well, and what didn't!

[AlexanderM (Alexander Momchilov)]

And am I the only one who liked the "Mirror" naming? Get it? A mirror reflects things? I thought it was so clever!

Not just clever though, IMO it adds clarity. Rather than saying:

I have an object of the Class class that represents Foo

You can say:

I have a ClassMirror reflecting Foo

It helps communicate the difference between Type and "type" without depending on stylized text for the distinction.

[ksluder (Kyle Sluder)]

@AlexanderM

And am I the only one who liked the "Mirror" naming? Get it? A mirror reflects things? I thought it was so clever!

“Mirror” comes from the same too-cute-by-half tradition as “lens”, which Swift calls “keypaths”.

[curt (Curt Clifton)]

This paper by Bracha and Ungar has a bit of history on the coining of the term “mirror” in Self, see section 5.1.1:

Named originally both as a pun on “reflection” and also to suggest “smoke and mirrors”, …

The paper also has a solid overview of mirror-based reflection, though from a strongly object-oriented perspective.

[NSExceptional (Tanner Bennett)]

Could you remove the manual line breaks? I read These on my phone

[Jumhyn (Frederick Kellison-Linn)]

OTOH reading a post with manual 80-column wrapping on mobile does give it a bit of a poetic feeling

[benrimmington]

Reflection.Type can be initialized from either a metatype or an instance, but will the latter ignore a CustomReflectable conformance? Should the initializers be failable, in case the -disable-reflection-metadata option has been used? Or could they have any Reflectable.Type and any Reflectable parameters, using the recently-pitched marker protocol?

Like the MemoryLayout.offset(of:) method, should the Field.offset property fail by returning nil rather than zero? Should the other MemoryLayout APIs (size, stride, alignment) be added to Reflection.Type?

Would [AnyHashable: Any] or [Never: Never] be more appropriate than [Int: Int] in the PartialType examples? Should the textual representation also include placeholders (e.g. "Dictionary<_, _>")? Instead of the name properties, could there be description and debugDescription properties, for unqualified and qualified names?

To avoid confusion between metatypes and metadata, I suggest renaming:

• Type to Metadata
• Type.swiftType to Metadata.type
• Field.type to Field.metadata
• Case.payloadType to Case.metadata

The @usableFromInline internal APIs should probably be removed from the detailed design, but the index and tag might be useful as public APIs. The @frozen and @inlinable attributes could also be removed for readability.

[bbrk24]

@benrimmington

Would [AnyHashable: Any] or [Never: Never] be more appropriate than [Int: Int] in the PartialType examples?

I was thinking of [Void: Void], but I guess there's not always going to be a good equivalent -- what about AsyncLineSequence for example? Would I have to say something like AsyncLineSequence<TaskGroup<UInt8>>? There is no AnyAsyncSequence<T> type, and the AsyncSequence protocol doesn't currently support the generic syntax.

[filip-sakel]

I think Reflection APIs really need to be more transparent to the compiler. As @benrimmington mentioned, one way is to explicitly mark types with the recently pitched Reflectable protocol. But even in that case, there is a variety of metadata that is emitted and that needlessly adds to binary size. Ideally, reflection APIs should also be known to the compiler so that they could be potentially constant-folded when the reflected types are known, e.g. Type(InternalType).fields. This is obviously quite an advanced feature, but it could conceivably make reflection really performant without complicating code or slowing down compilation with macros.

I’d like to see this “transparent reflection” direction addressed in the pitch, but I don’t know what would be the best way to implement it. LTO is certainly one way of eliminating unused metadata. To constant-fold, though, the compiler would need to be aware of reflective code in earlier stages of compilation. I’m not a compiler engineer, so I don’t know if that would be possible.

I’m really glad to see this feature being pitched and am excited about making reflection more powerful! But I think we should start to consider how reflection could be made efficient enough to be more accessible to low-resource environments (such as Wasm Swift apps).