[Pitch] Reflection

Evolution Pitches
#41

Excited for this! Getting KeyPaths for fields is important for our codebase, and we're currently relying on _forEachFieldWithKeyPath via @_spi(Reflection) import Swift, which is a little unfortunate. Further in the past, we did it using Mirror, which was at least equally unfortunate.
It would be great to have a proper API for this functionality!

We do need the KeyPaths to be Writable however, which isn't included in the original pitch.

But from the following I gather it would still be possible to make the KeyPaths from Field writable? Perhaps with a cast? Am I reading this correctly?

2 Likes
#42

Without distracting too much from the current pitch, iterating through the constituents of a KeyPath is very interesting functionality to us as well.
Seems like it may also contribute towards enabling keypath codability?

1 Like
#43

This is very possible! Do you think isClass, isStruct, etc. accessors are the way to go, or would you prefer something like type.kind == .class?

As of right now, I don't have plans to make an API that vends a mutable key path to a field just yet. I think I still need some more designing there to come up with a good solution that doesn't allow folks to ask for mutable key paths to properties they can't access.

#44

Great! I think, an enum is the cleaner solution. It could also get properties such as isValueType so that you can ask type.kind.isValueType.

#45

Probably the latter, because it's more ammendable to switch statements and exhaustiveness checking, as opposed to if/else if/else ladders checking every kind of isX property

2 Likes
#46

That could be a bit deceiving. I think people will unknowingly call that when what they're looking for is hasValueSemantics, which is something that is actually pretty tough to nail down, certainly in any automated capacity.

(Instances of classes that have CoW can have value semantics, and instances of structs that contain references to objects can have reference semantics, so it's really not all that straight forward.)

#47

Maybe you're right.

#48

Just finally getting to catching up on this. Like others, I'm very excited about the possibilities, and this looks like a great start!

I do hope we'll have an opportunity to iterate on the API design aspect a bit. At first blush I was concerned that the initial design of Type doesn't use a label to distinguish between initializing from a value or from a type, which is something that we took care to do with other APIs (such as MemoryLayout.stride, etc.). I do worry about whether confusion could arise when the value is a type.

Supposing we were to (say) distinguish Type(Foo.self) from Type(of: fooInstance), a question arises which actually is present even with the API as pitched: users might rightly be confused about the distinction between type(of:) and Type(...)—and more fundamentally, as you mention later, the distinction between a type and a Type.

While I empathize with not just sticking the API directly on Any.Type and possibly even all concrete types, I don't think that's the only alternative to inventing a top-level Type that is distinct from the actual type:

Have you considered a design where, for example, we have a reflected property on types as an "umbrella" for reflection (in the same vein that we have lazy in the language for lazy sequences)? That is to say:

for field in Dog.reflected.fields { ... }

// ...as opposed to:
for field in Type(Dog).fields { ... }
// `Type(Dog)` isn't `Dog.Type` or `Dog`...
// That's a lot of different "types" to understand!

// ...or:
for field in Dog.self.fields { ... }
// Now we avoid creating yet another `Type`, but either we make
// `self` even more "magical" or we pollute code completion for
// the concrete type.

This would moreover provide a nice symmetry where Dog.self gives you the type and Dog.reflected gives you the "reflected type." From a value, one would then write type(of: dog).reflected.

Other than that design question I would just echo previous commenters regarding the "partial type" terminology.

I'd also point out that while Field is nice and short, I believe the user-facing terminology (in TSPL, say; and indeed in the just-pitched declaration macros document) has called it a "stored property," UI code would use "field" to mean something very different, and I think there is value in trying to maintain consistency here throughout our documentation and APis.

Anyway, bravo on the overall pitch—very exciting that it exposes new possibilities for the language in an exciting way.

12 Likes
#49

It sounds like your concern is not necessarily with the name Type, but rather the method in how one accesses a value of this type? X.reflected is an interesting approach, but would require compiler support (which is not a deal breaker). Is there a library approach we could take to solve this a different way? Could there be a global function in Reflection that could achieve this, similar to type(of:)? E.g:

for field in reflect(Dog.self).fields {}

for field in reflect(type(of: dog)).fields {}

Does X.reflected return a value of type Type? or is the following disallowed:

let x = Dog.reflected

Yeah, it seems lots of people agree this is not the best name for this thing. A few suggested GenericType which is really close in my opinion, but doesn't make a whole lot of sense for types like Int or String who have GenericTypes but aren't generic.

Would you prefer this to be called Property?

for prop in reflect(Dog.self).properties {}

This makes sense to me, but might be confusing as it doesn't return computed properties right now. Maybe in the future if we add reflective information for computed properties we just spell that out explicitly?

for cprop in reflect(Dog.self).computedProperties {}
#50

These are duals of the same problem.

As Ben Cohen pointed out in a different design discussion, Swift has (a large number of) technically public type names that are not in practice user-facing. For example, most users will never write PrefixUpTo, UnfoldFirstSequence, etc., but will instead get instances of such types by using much more common affordances such as the range operators, sequence(first:next:), etc.

With the API design that you propose, the name Type has big shoes to fill because it is very user-facing. So on the one hand, it would be very unwieldy to name that type something like ReflectedType or TypeReflection, because every use site would then be littered with extraneous information, but on the other hand, to keep the name succinct we’re left with something that is very much “overloaded” with meaning and therefore (pun intended) generic and in certain contexts inscrutable.

This is why I point to lazy and other accessors as a model here: the underlying lazy sequence types themselves have pretty specific names and aren’t all crammed into a type called Lazy. Adopting that pattern would allow you to have a contextually succinct user-facing spelling (e.g.: Dog.reflected) without constraining what you actually name the type (for example, you could rename Type—not that you should—to SwiftNextGenerationAwesomeSuperUsefulReflectionOfType without affecting user ergonomics in most uses).

I don’t see a reason to disallow it, but as I touch on above, it probably shouldn’t be named Type, which is confusable with the metatype, etc. Because the appearance of the type name will be significantly demoted and for many people appear only in diagnostics or debugging info, it can be given a more descriptive name without impacting the user experience.

I believe in the macro pitch it’s called storedProperties: I’d love for whatever terminology we choose to be preserved verbatim where possible between macros and reflection APIs.

7 Likes
#51

Hi! This is looking great! Our use case might look like the following:

public protocol NewKeyPathIterable{}
public extension NewKeyPathIterable // Just so `Self` is concrete.
{
    var keyPaths: [PartialKeyPath<Self>]
    {
        var out = [PartialKeyPath<Self>]()
        guard #available(macOS 9999, *) else
        {
            print(" `guard #available(macOS 9999, *)` condition did not succeed.")
            return out
        }
        out.append(contentsOf: Type(Self.self).fields.compactMap{$0.keyPath as? PartialKeyPath<Self>})
        return out
    }
}

struct Restaurant: NewKeyPathIterable
{
    var burgerCost: Float
    var pieCost: Float
    var sodaCost: Float
}

let r = Restaurant(burgerCost: 4.99, pieCost: 1.99, sodaCost: 0.99)
for keypath in r.keyPaths // Optionally, get member names and iterate over them as well.
{
    print("Cost:",r[keyPath: keypath]) // Prints the cost of each item in the instance.
}

At the moment it doesn't look like we can get the KeyPath for a struct within a struct. For example, if we place an inner struct within Restaurant:

struct Sodas: NewKeyPathIterable
{
    var colaCost: Float
    var orangeCost: Float
    var lemonLimeCost: Float
}
struct Restaurant: NewKeyPathIterable
{
    var burgerCost: Float
    var pieCost: Float
    var sodas: Sodas
}

let r = Restaurant(burgerCost: 4.99, pieCost: 1.99, sodas: Sodas(colaCost: 0.99, orangeCost: 0.79, lemonLimeCost: 0.89))

then we get an EXC_BAD_ACCESS when iterating through the KeyPaths and attempting to access $0.keyPath in the extension above. This also appears to happen when we change Restaurant and Sodas into classes. I'm hoping that access to KeyPaths for structs-within-structs (and similar) doesn't get lost during the pitch/development phase!

5 Likes
#52

On second thought (you could say I'm reflecting back on this :smiley:), if we're going to have users writing switch statements to discriminate apart various kinds of Type apart, I think we may as well have Type as a protocol, and have concrete types.

I.e. replace:

// I highly recommend the "mirror" terminology, to clear up variable names like this
let typeObject = Type(something)
switch typeObject.kind {
case .class: ...
case .struct: ...
case .function: ...
...
}

With:

switch Type(something) {
case let classObject as ClassType: ...
case let structObject as StructType: ...
case let functionObject as FunctionType: ...
...
}

That gives you non-nilable access to the fields that are pertinent to each case.

1 Like
#53

Maybe I overlooked it, but shouldn't (or can't) Type have a name property as well?
Whereby name returns the name of a struct or class as written in code. Similar to the fields of the type.
It would be nice and obvious to have, no?

Otherwise you have to go through a String initialiser (assuming this is still the current way to do it).

extension Type
{
	var name: String { String(describing: self.swiftType) }
}
1 Like
#54

The type system representation proposed here recently came up in the review of SE-0385:

I agree that we should settle on one family of types to represent the type system that can be used throughout various language features, including the Reflection APIs proposed here, custom reflection metadata as proposed in SE-0385, and possibly other areas of the language that need to express requirements on declarations in Swift source code.

For example, I'm working on the design and implementation for attached macros, which would also benefit from a way to specify requirements on the "attached to" declaration, very similar to what's possible in SE-0385. Consider the @Persisted property wrapper from the Realm package. It may be possible to express this API as a macro that generates accessors directly, rather than using a property wrapper:

@attached(accessor) macro Persisted = #externalMacro(...)

class Dog {
  @Persisted var name: String
  @Persisted var age: Int
}

The @Persisted property wrapper today requires the wrapped value to conform to _Persistable. There's currently no way to express this requirement in an attached macro declaration. We could invent an ad-hoc representation of the various kinds of declarations, mirroring what SE-0385 did with the attachedTo: initializer parameter, but I think it would be much better to have a consistent representation across these various features.

8 Likes