[Pitch] Property reflection

Would it let me get a property if it does not correspond to a property, or computed property?

For example, would I be able to get the value returned by a fullName() instance method that concatenates the firstName and lastName properties together and returns the result? Or would it only work if fullName were implemented as a computed property as opposed to an instance method?

Thanks

···

On May 27, 2016, at 2:59 PM, Austin Zheng <austinzheng@gmail.com> wrote:

Yes, it was pretty much meant as a KVC-like feature for Swift. Get a reference to a property from a string which would allow you to get and set its value.

On Fri, May 27, 2016 at 11:53 AM, Ricardo Parada <rparada@mac.com> wrote:

On May 26, 2016, at 9:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

Can you elaborate on what this API would be used for? KVC? For instance, I played with Mirror the other day and my code to get a value given the property name looked more like this:

let age = myPerson.value(forKey:”age”) as! Int

And this is what I did:

// KVC stands for key-value-coding… but I only know how to get values. I don’t know how to set values

protocol KVC {
   func value(forKey key: String) -> Any!
}

// Default implementation
extension KVC {
   func value(forKey key: String) -> Any! {
       let aMirror = Mirror(reflecting:self)
       for case let (label?, value) in aMirror.children {
           if label == key {
               return value
           }
       }
       return nil
   }
}

public struct Person : KVC {
   let firstName: String
   let lastName: String
   let age: Int

   func fullName() -> String {
       return "\(firstName) \(lastName)"
   }
}

let aPerson = Person(firstName:"John", lastName:"Doe", age:48)

// It works for stored properties
let lastName = aPerson.value(forKey:"lastName") as! String
print("Last name is \(lastName)")

// It does not work for instance methods, i.e. fullName is not a stored property
let fullName = aPerson.value(forKey:"fullName")
if fullName != nil {
   print("Full name is \(fullName)")
} else {
   print("Unable to get fullName via KVC")
}

Sent from my iPad

I wonder how the views would work with the value semantic of structs.

Not all value types have value semantics. It's important to not forget that. I would like to see a way to make that distinction clear in our code but that is a conversation for another thread.

I meant value types. Thanks for pointing out that. :-)

The view types proposed, if not being a compiler magic, would need to have a reference to the reflected instance. They are also escapable unless special rules for the view types are enforced. So… perhaps we would want less magic and special rules?

IMO these views are not necessary, and can be consolidated into typed/untyped get/set calls on `Reflectable`. However, if the intention is to amortize the type metadata lookup, or not to introduce any kind of lookup caching in the runtime, perhaps we can introduce instead property descriptors.

(A quick dump of my idea:
https://gist.github.com/andersio/9ff02257b5c89b35fd523dcd09e484e4\)

Property descriptors could be useful in the sense that they wouldn't need to refer back to the instance. But I would also like to see a way to get something like a lens into the property for a specific instance which is what the views allow for. This design doesn't allow for that. Maybe we want to allow you to query for property descriptors alone, lenses alone, or the combination in a view.

Just for the record, I was meaning that the user can request a typed descriptor of a particular named instance properties (and even class properties) at runtime from the metatype. Users may then use these typed descriptors to query or modify the value from a specific instance.

It is like KVC, but uses typed descriptors instead of string keypaths.

Anyway, the most important message I'd like to raise is that the mutation should be acted upon the owner of the property, if the reflection is supposed to cover also value types.

Specifically using Zheng’s initial idea as an example, the setter in “GetSetPropertyView” of a property in a struct instance should not cause any change to the original instance by the established behaviour of value types in Swift.

Let’s say even If there are some kind of pointer magic bypassing the value type restrictions at runtime, GetSetPropertyView cannot be escaped but only useable in the local scope. Otherwise, we would have a view that can somehow point to nothing. This doesn’t sound great either way.

···

On 27 May 2016, at 9:30 PM, Matthew Johnson <matthew@anandabits.com> wrote:
On May 27, 2016, at 5:45 AM, Anders Ha via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Best Regards
Anders

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Here’s an idea using enums. They work similar to a lens, where they are separate from the actual instance. So you can convert string keys or validate potential values without needing an instance.

The PropertyIdentifierProtocol is implemented by an string representable enum, piggy backing on its failable initializer for converting from strings.
The PropertyStateProtocol, implemented by an enum with associated values, encapsulates a property in a type-safe manner, allowing it to be extracted as an entity then applied to other instances. It has a failable initializer for validating a value.

https://gist.github.com/BurntCaramel/315ee4dfca0c240755b534e1a5ee183f

···

On 27 May 2016, at 11:30 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Property descriptors could be useful in the sense that they wouldn't need to refer back to the instance. But I would also like to see a way to get something like a lens into the property for a specific instance which is what the views allow for. This design doesn't allow for that. Maybe we want to allow you to query for property descriptors alone, lenses alone, or the combination in a view.

Would you be willing to elaborate? Are you thinking of model objects whose properties are all completely hidden using 'private' and whose APIs are available only through methods? What if there existed something like "private(reflectable)", for properties you really want to keep both private and exposed to the reflection machinery? Or maybe a property behavior to do the same thing once that feature becomes a reality...

Austin

···

On May 26, 2016, at 11:52 PM, David Hart <david@hartbit.com> wrote:

On 27 May 2016, at 04:44, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

That's what I want to use it for and I'm fairly sure we need access to private properties for that.

I'm also not fond of allowing what could be called a backdoor to accessing privates and intervals too but has anyone thought of enhancing the Mirror class instead of having something totally new? I've tried using it once but in the end it wasn't necessary for what I needed to do.

An opt-in into reflection would be interesting too as it could help remove the burden of the compiler to prepare each class and instance for reflection. Should a type be prepared for reflection one could add to it an attribute like @reflected. Then the compiler would add the necessary optimisations to it and its subclasses.

About serialisation, I still think the best way to deal with it is to use the Visitor pattern. Each class that needs serialisation implements the functions that say what exactly is needed to save and restore state. I don't see how anything automatic can be better than this.

···

-----Original Message-----
From: "Brent Royal-Gordon via swift-evolution" <swift-evolution@swift.org>
Sent: ‎27/‎05/‎2016 07:25 AM
To: "Matthew Johnson" <matthew@anandabits.com>
Cc: "swift-evolution" <swift-evolution@swift.org>
Subject: Re: [swift-evolution] [Pitch] Property reflection

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

I had some thoughts about an alternate design based partially on access control concerns. Sketching roughly, it looks like this:

* There is a pseudo-property called, say, `properties` on every type (and perhaps on every instance). (Alternate designs are available, like an `inout`-returning pseudo-function. It doesn't really matter.)

* `properties` is a dictionary, or at least something very like a dictionary: you can look something up by key or iterate over the available keys. Its keys are strings, and its values are lens functions. These lens functions return type Any; their setters throw if you assign an incompatible type.

* The contents of `properties` are the properties visible *at the site where it is called*. That means calling `properties` in different places will give you different results. If you import a type from another module and then add properties in an `internal` extension, you'll see the other module's `public` properties plus your `internal` ones.

* You can pass your `properties` dictionary to other code; that effectively delegates your access to that other code. Thus, if your main class body passes its `property` dictionary to a serialization library, that library can access your private properties. If you pass it `inout`, then the library can modify your private properties.

* There is no opting in, but I *think* the compiler has enough information to figure out what goes into `properties` at the sites where it is used, so you only have to pay for it if and when you use it. I could be wrong, though—there may be something there that I'm missing.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

I saw this syntax show up in the equatable/hashable context (could not convince myself of its value), are there other areas where it was put forward?

Not yet, but when we have the ability to run code at compile time it is possible we could even have user-defined derivable protocols somehow. I'm sure we will have the ability to synthesize more than just these two eventually.

···

Sent from my iPad

On May 27, 2016, at 3:07 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com> wrote:

On May 27, 2016, at 4:20 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Would you be willing to elaborate? Are you thinking of model objects whose properties are all completely hidden using 'private' and whose APIs are available only through methods? What if there existed something like "private(reflectable)", for properties you really want to keep both private and exposed to the reflection machinery? Or maybe a property behavior to do the same thing once that feature becomes a reality...

'private(reflectable)' would mean private for the purpose of reflection if we follow the same interpretation as 'private(set)'. Maybe you meant 'private public(reflectable)' - as in private, but public for the purpose of reflection only.

···

Sent from my iPad

On May 27, 2016, at 2:25 AM, Austin Zheng <austinzheng@gmail.com> wrote:

Austin

On May 26, 2016, at 11:52 PM, David Hart <david@hartbit.com> wrote:

On 27 May 2016, at 04:44, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

That's what I want to use it for and I'm fairly sure we need access to private properties for that.

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

I had some thoughts about an alternate design based partially on access control concerns.

The syntax in your design would be very convenient for sure.

I'm not sure how this alternative is related to access control. Austin's proposal could enforce access control in the same way and he mentioned he was inclined to enforce it the same way you are describing.

The reason I say it is tricky is because enforcing access control in this way would prevent things people want to do with reflection (things that have received a lot of discussion recently). Allowing a distinct visibility to be specified for reflection might be a way to strike a reasonable balance.

Your suggested approach doesn't allow for access to property metadata. Do you have any suggestions on how that would be exposed?

What about casting the lens to a typed lens? Would you envision that being supported?

I also think we would want a way to access a specific property by name if accessing the whole properties dictionary involved any non-trivial overhead (but maybe it wouldn't have to).

···

Sent from my iPad
On May 27, 2016, at 5:25 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

Sketching roughly, it looks like this:

* There is a pseudo-property called, say, `properties` on every type (and perhaps on every instance). (Alternate designs are available, like an `inout`-returning pseudo-function. It doesn't really matter.)

* `properties` is a dictionary, or at least something very like a dictionary: you can look something up by key or iterate over the available keys. Its keys are strings, and its values are lens functions. These lens functions return type Any; their setters throw if you assign an incompatible type.

* The contents of `properties` are the properties visible *at the site where it is called*. That means calling `properties` in different places will give you different results. If you import a type from another module and then add properties in an `internal` extension, you'll see the other module's `public` properties plus your `internal` ones.

* You can pass your `properties` dictionary to other code; that effectively delegates your access to that other code. Thus, if your main class body passes its `property` dictionary to a serialization library, that library can access your private properties. If you pass it `inout`, then the library can modify your private properties.

* There is no opting in, but I *think* the compiler has enough information to figure out what goes into `properties` at the sites where it is used, so you only have to pay for it if and when you use it. I could be wrong, though—there may be something there that I'm missing.

--
Brent Royal-Gordon
Architechies

Sent from my iPad

I wonder how the views would work with the value semantic of structs.

Not all value types have value semantics. It's important to not forget that. I would like to see a way to make that distinction clear in our code but that is a conversation for another thread.

I meant value types. Thanks for pointing out that. :-)

The view types proposed, if not being a compiler magic, would need to have a reference to the reflected instance. They are also escapable unless special rules for the view types are enforced. So… perhaps we would want less magic and special rules?

IMO these views are not necessary, and can be consolidated into typed/untyped get/set calls on `Reflectable`. However, if the intention is to amortize the type metadata lookup, or not to introduce any kind of lookup caching in the runtime, perhaps we can introduce instead property descriptors.

(A quick dump of my idea:
https://gist.github.com/andersio/9ff02257b5c89b35fd523dcd09e484e4\)

Property descriptors could be useful in the sense that they wouldn't need to refer back to the instance. But I would also like to see a way to get something like a lens into the property for a specific instance which is what the views allow for. This design doesn't allow for that. Maybe we want to allow you to query for property descriptors alone, lenses alone, or the combination in a view.

Just for the record, I was meaning that the user can request a typed descriptor of a particular named instance properties (and even class properties) at runtime from the metatype. Users may then use these typed descriptors to query or modify the value from a specific instance.

It is like KVC, but uses typed descriptors instead of string key paths.

I understand this. But this is not like a lens. It is much more awkward to use.

Anyway, the most important message I'd like to raise is that the mutation should be acted upon the owner of the property, if the reflection is supposed to cover also value types.

Specifically using Zheng’s initial idea as an example, the setter in “GetSetPropertyView” of a property in a struct instance should not cause any change to the original instance by the established behaviour of value types in Swift.

Let’s say even If there are some kind of pointer magic bypassing the value type restrictions at runtime, GetSetPropertyView cannot be escaped but only useable in the local scope. Otherwise, we would have a view that can somehow point to nothing. This doesn’t sound great either way.

You raise good points here. I am interested to hear what Joe Groff has to say about this. I believe the issues involved are the same as those involved with lenses into value types.

···

On May 27, 2016, at 9:22 AM, Anders Ha <hello@andersio.co> wrote:

On 27 May 2016, at 9:30 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
On May 27, 2016, at 5:45 AM, Anders Ha via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Best Regards
Anders

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

The private(reflectable) sounds reasonable, however, what about internal? It's impossible to include it in an invocation from within the module, but not including it when invoked from outside.

Would the behavior on private vars be non-reflectable by default and internal vars would be reflectable by default?

Charlie

···

On May 27, 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Would you be willing to elaborate? Are you thinking of model objects whose properties are all completely hidden using 'private' and whose APIs are available only through methods? What if there existed something like "private(reflectable)", for properties you really want to keep both private and exposed to the reflection machinery? Or maybe a property behavior to do the same thing once that feature becomes a reality...

Austin

On May 26, 2016, at 11:52 PM, David Hart <david@hartbit.com> wrote:

On 27 May 2016, at 04:44, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

That's what I want to use it for and I'm fairly sure we need access to private properties for that.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I'm talking of model objects that serializes certain properties (not necessarily all) that should have their setter be private/hidden for encapsulation reasons:

struct Person {
    private(set) var birthdate: NSDate
}

I'm not a fan of adding an attribute like private(reflectable). The proposal we are discussing is most probably coming post-Swift3 and we might as well wait until property behaviour a are brought in to the mix.

Sorry if my post feels negative, but I'm a huge supporter of Reflection APIs and I just want us to get it right :)

···

On 27 May 2016, at 09:25, Austin Zheng <austinzheng@gmail.com> wrote:

Would you be willing to elaborate? Are you thinking of model objects whose properties are all completely hidden using 'private' and whose APIs are available only through methods? What if there existed something like "private(reflectable)", for properties you really want to keep both private and exposed to the reflection machinery? Or maybe a property behavior to do the same thing once that feature becomes a reality...

Austin

On May 26, 2016, at 11:52 PM, David Hart <david@hartbit.com> wrote:

On 27 May 2016, at 04:44, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

That's what I want to use it for and I'm fairly sure we need access to private properties for that.

I think there is a consensus forming that we probably don't yet know enough about Swift's future capabilities to nail down specific designs yet. (Not that I want to stop people from talking about them if they wish.)

However, now that there is a conversation topic about reflection, I also wanted to ask the following questions:

1. What sorts of features are important, and what would you use them for?

For me, the list looks like the following:

- Describe the structure of a type. (We have that right now, with introspection using "Mirror".)

- Find all the properties on a type. Get and set their values, and get information about them (like their dynamic type).
- Find all the methods on a type. Be able to get a reference to and invoke a method on an instance of a type.
- Find all the subscripts and initializers on a type?

- Get a list of all concrete types opting in to extended reflection.
- Get a list of all concrete types visible to the caller that conform to some protocol(s), possibly with additional requirements on associated types.
- Get a list of all protocols.

- Create a reference to a type using a string, and perform certain actions using that type. (It would be interesting if you could say "try parsing this type into a concrete type that conforms to X, Y, and Z protocols", and if successful you get a metatype to use or something you can pass to a generic function.
- Create a reference to a method or property using a string, and use it as above.

- Reify methods and properties marked with the 'dynamic' keyword in such a way that where they are dispatched to can be controlled at runtime. For example, maybe a dynamic method of a class might use some default implementation, or if some condition is met forward invocations of it to a dynamic method on another class instead.

2. What sorts of priorities should inform Swift's reflection capabilities?

For me, the option to perform both "statically typed" and runtime type checked operations, when feasible, is a big one. I think something like variadic generics could allow for typesafe reflection on methods and functions at runtime - rather than having to call a performSelector: like method, it would be possible to get a full-fledged value of function type and use it just like any other method. This is something few languages have AFAIK.

The reflection machinery should cooperate with the rest of the language. If there must be a way to, for example, access private members of an instance through reflection, it should not be unconditional and should be carefully considered such that there are certain invariants that are still honored. Like Laurent said earlier, bad things can happen if you use reflection to subvert access control.

I think reflection should be opt-in in most cases. Reflection opting-in should be composable, inheritable, and retroactive, which covers the most common cases in which reflection would be useful: working with Cocoa/Cocoa Touch, for example.

One priority you don’t mention here that I think is worthwhile to consider is static reflection. Any features that make sense both statically and dynamically should have the same API statically and dynamically wherever possible.

···

On May 27, 2016, at 11:54 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Best,
Austin

On May 27, 2016, at 7:37 AM, plx via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I think this proposal should “simmer" until we know what property behaviors will look like (or find out they are never happening, etc.).

The interaction with “property behaviors” seems likely to be quite subtle, even for something simple like `lazy`.

For sake of argument, let’s say that a `lazy` declaration like so:

  class SomeClass {
    lazy var foo: Foo = Foo(bar: self.prepareBar())
  }

…gets implicitly-expanded into e.g. something like the below:

  class SomeClass {

    var foo: Foo {
      guard let foo = _foo else {
        _foo = Foo(bar: self.prepareBar())
        return _foo
      }
      return foo
    }

    private var _foo: Foo? = nil

  }

…which immediately exposes a few lazy-specific questions:

- should `foo` be exposed via reflection? (IMHO: yes)
- should `_foo` be exposed via reflection? (IMHO: probably not, but not certain)
- should `foo`’s lazy-ness be exposable? (IMHO: yes, but how exactly?)

…as well as a few more-general questions:

- should computed properties, in general, be exposed to reflection? (IMHO: probably not, but there are some exceptions…)
- if user-specified property behaviors get special exposure, how should that work?
- how should reflection work for enums/enums-with-payloads?

Finally, I worry a lot less about the details of getter/setter pairs than I worry about being able to use reflection for construction/initialization.

I don’t have any actual proposal on that front, but it seems like it should be factored into any reflection design.

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I believe CoreData uses synthesized computed properties that access values stored in an internal row cache (or something along those lines).

Yeah, by creating a subclass of the original class, but that's not the point - accessing private variables can have legit uses. Yes, it can be abused, but this is the developer's responsibility IMHO.

···

I'm in favor of reflecting all vars by default (even private ones) with an option to opt-out.

I much prefer the opt-in approach if we're going to allow for reflection to have different visibility. Access control should be explicit IMO.

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Sent from my iPad

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

I saw this syntax show up in the equatable/hashable context (could not convince myself of its value), are there other areas where it was put forward?

Not yet, but when we have the ability to run code at compile time it is possible we could even have user-defined derivable protocols somehow. I'm sure we will have the ability to synthesize more than just these two eventually.

so then IMHO the language as it exists today does not need ‘deriving’. I would even venture that the core team has a road-map for equatable/hashable. And if not, they do have some code in place that looks like it is very close to doing the trick today. So that leaves me thinking that it is either a bandwidth issue (not wanting to over commit and miss the deadline) and/or interaction with other decisions that might need to be made before (initializers being one - and that one seems to also have dependencies on 4+). Either way, I do not forsee a ‘deriving’. But I am truly nobody.

···

On May 27, 2016, at 3:18 PM, Matthew Johnson <matthew@anandabits.com> wrote:
On May 27, 2016, at 3:07 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com <mailto:laurent.mihalkovic@gmail.com>> wrote:

On May 27, 2016, at 4:20 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

or perhaps simply read what the people who have a lot of experience the topic have to say about it, and hopefully learn from the errors others have described. Serialization has long been a source of serious security issues in Java. Swift could do without these

-1

···

On May 27, 2016, at 4:05 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

Sent from my iPad

On May 26, 2016, at 9:44 PM, Austin Zheng <austinzheng@gmail.com <mailto:austinzheng@gmail.com>> wrote:

Thanks, as always, for the thoughtful feedback. (inline)

On Thu, May 26, 2016 at 7:20 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

Yes, that's a good start. I'll sleep on it.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

'deriving' is pretty much what I had in mind. If Equatable gets a magical attribute, keyword, or whatever, I'd like to use it for this feature as well.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

One big advantage methods have over properties is that you can (as of right now) get a reference to a method as a value of function type, but not a property.

True, but as with 'throws' this is a temporary limitation. I don't think we should design our reflection API around a temporary limitation. IIRC the intended plan for getting first class access to properties is a lens. If we model it as a property we will eventually get first class access to it as a lens (which means we play nice with lenses even though we don't know what they will look like yet).

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

I think it's worth consideration. Maybe views into properties can eventually conform to some sort of lens protocol or interface, allowing them to be composed and used as such.

I think Joe Groff has talked about lenses more than anyone else. Maybe he has some thoughts on this. But as I noted above, maybe exposing access as a property is the best first step. It's easy enough to wrap in a closure if you need to pass a function around before property references / lenses are introduced.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

```

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

As several have noted, this is what makes things tricky. People will want to write a library packaged as a module that can serialize private properties from types in a different module. We need to at least consider that use case.

I agree it is awkward. It is just a quick dump anyway. :)

Expanding on Groff’s idea of a `T -> inout U`-ish lens, perhaps the reflection API can reside in the metatype, and create lenses (with runtime type checking) instead? It would end up similarly to Zheng’s initial idea. But unlike Zheng’s view types, lenses would not capture but expects an instance as its input.

By the way, a read-write lens should be conceptually `inout T -> inout U` to cover also the value types. But then it would prevent it to be used with constant references (e.g. `self` in instance methods). New compiler magics can be engineered to tackle this though, or a distinction between a read-write lens for value type and reference type has to be made.

···

On 27 May 2016, at 10:43 PM, Matthew Johnson <matthew@anandabits.com> wrote:

On May 27, 2016, at 9:22 AM, Anders Ha <hello@andersio.co <mailto:hello@andersio.co>> wrote:

On 27 May 2016, at 9:30 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

Sent from my iPad

On May 27, 2016, at 5:45 AM, Anders Ha via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I wonder how the views would work with the value semantic of structs.

Not all value types have value semantics. It's important to not forget that. I would like to see a way to make that distinction clear in our code but that is a conversation for another thread.

I meant value types. Thanks for pointing out that. :-)

The view types proposed, if not being a compiler magic, would need to have a reference to the reflected instance. They are also escapable unless special rules for the view types are enforced. So… perhaps we would want less magic and special rules?

IMO these views are not necessary, and can be consolidated into typed/untyped get/set calls on `Reflectable`. However, if the intention is to amortize the type metadata lookup, or not to introduce any kind of lookup caching in the runtime, perhaps we can introduce instead property descriptors.

(A quick dump of my idea:
https://gist.github.com/andersio/9ff02257b5c89b35fd523dcd09e484e4\)

Property descriptors could be useful in the sense that they wouldn't need to refer back to the instance. But I would also like to see a way to get something like a lens into the property for a specific instance which is what the views allow for. This design doesn't allow for that. Maybe we want to allow you to query for property descriptors alone, lenses alone, or the combination in a view.

Just for the record, I was meaning that the user can request a typed descriptor of a particular named instance properties (and even class properties) at runtime from the metatype. Users may then use these typed descriptors to query or modify the value from a specific instance.

It is like KVC, but uses typed descriptors instead of string key paths.

I understand this. But this is not like a lens. It is much more awkward to use.

-
Note:
[swift-evolution] Proposal: Expose getter/setters in the same way as regular methods
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151214/003008.html

Anyway, the most important message I'd like to raise is that the mutation should be acted upon the owner of the property, if the reflection is supposed to cover also value types.

Specifically using Zheng’s initial idea as an example, the setter in “GetSetPropertyView” of a property in a struct instance should not cause any change to the original instance by the established behaviour of value types in Swift.

Let’s say even If there are some kind of pointer magic bypassing the value type restrictions at runtime, GetSetPropertyView cannot be escaped but only useable in the local scope. Otherwise, we would have a view that can somehow point to nothing. This doesn’t sound great either way.

You raise good points here. I am interested to hear what Joe Groff has to say about this. I believe the issues involved are the same as those involved with lenses into value types.

Best Regards
Anders

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

This is a good list of goals. A few quick comments:

I feel like a fundamental question that can be “added” to almost all of these is “how do modules appear for reflection?”

EG, for one such example, “Find all the methods on a type”:
- can you ask for only methods defined in its “home module”
- can you ask for only methods defined on it in some specific module
- if you ask for all of them, can you tell which methods come from which modules

…and relatedly you get into e.g. “if you do a method lookup by name, how do you disambiguate same-name methods defined in extensions from different modules”.

I don’t think it’s necessary to keep listing such possible questions, as the general thrust should be clear; any answer is probably going to be dependent on the final ABI/resiliency implementation and on how (if?) modules themselves become visible in the runtime (e.g. do they have a type/representation / etc., or not?).

I have a pretty specific motivating case that makes me concerned about these issues; I’ll detail it a bit in hopes it can get added to the “is this proposal reasonably feature-complete” consideration.

Something I would *like* to be able to do with the reflection system is do a reflection-based, fully-generic “custom-view configuration-experiment UI”; e.g. if you could write this:

  protocol ConfigurableUIComponent : AnyObject {
    associatedtype Configuration // perhaps: `:MutableReflectable` or similar?
    init()
    var configuration: Configuration { get set }
  }

…and then be able to write something like this:

  class ConfigurationExperimentationViewController<
    K
    where
    K:UIView,
    K:ConfigurableUIComponent> : UIViewController {

  // sets up a K() and installs that view into `self.view`
  // sets up a reflection-derived UI for manipulating K’s `configuration`,
  // e.g. an embedded table view with controls for each appropriate field in `K.Configuration`

}

…reasonably-straightforwardly in Swift, ideally without needing to e.g. add custom annotations / additional conformances to get a reasonable outcome (at least for many cases).

Although I’m pretty sure any reasonable read/write reflection system will suffice for the "easy scenarios”, there’s a lot of room for nuance / complication due to the presence of e.g. calculated properties, etc., and it’d be nice if the reflection system provided enough information to make reasonable decisions in such cases without requiring too much user support (from e.g. custom annotations, (etc.)).

Given the early state of the discussion hashing it out in detail now doesn’t seem necessary, but I hope concrete scenarios like the above can be kept under consideration as the design work continues.

···

On May 27, 2016, at 11:54 AM, Austin Zheng <austinzheng@gmail.com> wrote:

I think there is a consensus forming that we probably don't yet know enough about Swift's future capabilities to nail down specific designs yet. (Not that I want to stop people from talking about them if they wish.)

However, now that there is a conversation topic about reflection, I also wanted to ask the following questions:

1. What sorts of features are important, and what would you use them for?

For me, the list looks like the following:

- Describe the structure of a type. (We have that right now, with introspection using "Mirror".)

- Find all the properties on a type. Get and set their values, and get information about them (like their dynamic type).
- Find all the methods on a type. Be able to get a reference to and invoke a method on an instance of a type.
- Find all the subscripts and initializers on a type?

- Get a list of all concrete types opting in to extended reflection.
- Get a list of all concrete types visible to the caller that conform to some protocol(s), possibly with additional requirements on associated types.
- Get a list of all protocols.

- Create a reference to a type using a string, and perform certain actions using that type. (It would be interesting if you could say "try parsing this type into a concrete type that conforms to X, Y, and Z protocols", and if successful you get a metatype to use or something you can pass to a generic function.
- Create a reference to a method or property using a string, and use it as above.

- Reify methods and properties marked with the 'dynamic' keyword in such a way that where they are dispatched to can be controlled at runtime. For example, maybe a dynamic method of a class might use some default implementation, or if some condition is met forward invocations of it to a dynamic method on another class instead.

2. What sorts of priorities should inform Swift's reflection capabilities?

For me, the option to perform both "statically typed" and runtime type checked operations, when feasible, is a big one. I think something like variadic generics could allow for typesafe reflection on methods and functions at runtime - rather than having to call a performSelector: like method, it would be possible to get a full-fledged value of function type and use it just like any other method. This is something few languages have AFAIK.

The reflection machinery should cooperate with the rest of the language. If there must be a way to, for example, access private members of an instance through reflection, it should not be unconditional and should be carefully considered such that there are certain invariants that are still honored. Like Laurent said earlier, bad things can happen if you use reflection to subvert access control.

I think reflection should be opt-in in most cases. Reflection opting-in should be composable, inheritable, and retroactive, which covers the most common cases in which reflection would be useful: working with Cocoa/Cocoa Touch, for example.

Best,
Austin

On May 27, 2016, at 7:37 AM, plx via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I think this proposal should “simmer" until we know what property behaviors will look like (or find out they are never happening, etc.).

The interaction with “property behaviors” seems likely to be quite subtle, even for something simple like `lazy`.

For sake of argument, let’s say that a `lazy` declaration like so:

  class SomeClass {
    lazy var foo: Foo = Foo(bar: self.prepareBar())
  }

…gets implicitly-expanded into e.g. something like the below:

  class SomeClass {

    var foo: Foo {
      guard let foo = _foo else {
        _foo = Foo(bar: self.prepareBar())
        return _foo
      }
      return foo
    }

    private var _foo: Foo? = nil

  }

…which immediately exposes a few lazy-specific questions:

- should `foo` be exposed via reflection? (IMHO: yes)
- should `_foo` be exposed via reflection? (IMHO: probably not, but not certain)
- should `foo`’s lazy-ness be exposable? (IMHO: yes, but how exactly?)

…as well as a few more-general questions:

- should computed properties, in general, be exposed to reflection? (IMHO: probably not, but there are some exceptions…)
- if user-specified property behaviors get special exposure, how should that work?
- how should reflection work for enums/enums-with-payloads?

Finally, I worry a lot less about the details of getter/setter pairs than I worry about being able to use reflection for construction/initialization.

I don’t have any actual proposal on that front, but it seems like it should be factored into any reflection design.

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I'm also not fond of allowing what could be called a backdoor to accessing privates and intervals too but has anyone thought of enhancing the Mirror class instead of having something totally new? I've tried using it once but in the end it wasn't necessary for what I needed to do.

An opt-in into reflection would be interesting too as it could help remove the burden of the compiler to prepare each class and instance for reflection. Should a type be prepared for reflection one could add to it an attribute like @reflected. Then the compiler would add the necessary optimisations to it and its subclasses.

About serialisation, I still think the best way to deal with it is to use the Visitor pattern. Each class that needs serialisation implements the functions that say what exactly is needed to save and restore state. I don't see how anything automatic can be better than this.

I brought up serialization because a lot of people *want* to do it using a dynamic API. This is a part of the Objective-C vs Swift dynamism conversation that is happening right now. That conversation inspired Austin's pitch and also my comment that people's desire for this makes access control a tricky issue.

Regardless of *you* want to do serialization, I think it's important to recognize that this is a hot topic and one of the frustrations some people currently have with Swift. I think it's worth having the conversation about how we can better satisfy the use cases they have talked about and what role reflection might play in that.

I think most serialization use cases are probably better served by static rather than dynamic reflection. But it isn't clear how long we'll have to wait until we can do that.

···

Sent from my iPad

On May 27, 2016, at 6:15 AM, Leonardo Pessoa <me@lmpessoa.com> wrote:

From: Brent Royal-Gordon via swift-evolution
Sent: ‎27/‎05/‎2016 07:25 AM
To: Matthew Johnson
Cc: swift-evolution
Subject: Re: [swift-evolution] [Pitch] Property reflection

> This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

I had some thoughts about an alternate design based partially on access control concerns. Sketching roughly, it looks like this:

* There is a pseudo-property called, say, `properties` on every type (and perhaps on every instance). (Alternate designs are available, like an `inout`-returning pseudo-function. It doesn't really matter.)

* `properties` is a dictionary, or at least something very like a dictionary: you can look something up by key or iterate over the available keys. Its keys are strings, and its values are lens functions. These lens functions return type Any; their setters throw if you assign an incompatible type.

* The contents of `properties` are the properties visible *at the site where it is called*. That means calling `properties` in different places will give you different results. If you import a type from another module and then add properties in an `internal` extension, you'll see the other module's `public` properties plus your `internal` ones.

* You can pass your `properties` dictionary to other code; that effectively delegates your access to that other code. Thus, if your main class body passes its `property` dictionary to a serialization library, that library can access your private properties. If you pass it `inout`, then the library can modify your private properties.

* There is no opting in, but I *think* the compiler has enough information to figure out what goes into `properties` at the sites where it is used, so you only have to pay for it if and when you use it. I could be wrong, though—there may be something there that I'm missing.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Oh, this is really interesting. The idea being that you get an object representing a property like "age" based on a metatype (like "Person.type"), and then you pass in instances of the type to methods on that object to manipulate that property. That is definitely less stateful and less prone to strange issues like mutating a value type beneath its view.

···

On May 27, 2016, at 8:53 AM, Anders Ha <hello@andersio.co> wrote:

I agree it is awkward. It is just a quick dump anyway. :)

Expanding on Groff’s idea of a `T -> inout U`-ish lens, perhaps the reflection API can reside in the metatype, and create lenses (with runtime type checking) instead? It would end up similarly to Zheng’s initial idea. But unlike Zheng’s view types, lenses would not capture but expects an instance as its input.

By the way, a read-write lens should be conceptually `inout T -> inout U` to cover also the value types. But then it would prevent it to be used with constant references (e.g. `self` in instance methods). New compiler magics can be engineered to tackle this though, or a distinction between a read-write lens for value type and reference type has to be made.

-
Note:
[swift-evolution] Proposal: Expose getter/setters in the same way as regular methods
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151214/003008.html

Anyway, the most important message I'd like to raise is that the mutation should be acted upon the owner of the property, if the reflection is supposed to cover also value types.

Specifically using Zheng’s initial idea as an example, the setter in “GetSetPropertyView” of a property in a struct instance should not cause any change to the original instance by the established behaviour of value types in Swift.

Let’s say even If there are some kind of pointer magic bypassing the value type restrictions at runtime, GetSetPropertyView cannot be escaped but only useable in the local scope. Otherwise, we would have a view that can somehow point to nothing. This doesn’t sound great either way.

You raise good points here. I am interested to hear what Joe Groff has to say about this. I believe the issues involved are the same as those involved with lenses into value types.

Best Regards
Anders

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

That’s a good point. Talking about the issues involved and stating the reasons for our decision is giving the use case consideration. I’m not advocating either way. I’m just saying that we need to make a conscious decision with a good and clearly stated rationale.

-Matthew

···

On May 27, 2016, at 9:50 AM, L Mihalkovic <laurent.mihalkovic@gmail.com> wrote:

On May 27, 2016, at 4:05 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Sent from my iPad

On May 26, 2016, at 9:44 PM, Austin Zheng <austinzheng@gmail.com <mailto:austinzheng@gmail.com>> wrote:

Thanks, as always, for the thoughtful feedback. (inline)

On Thu, May 26, 2016 at 7:20 PM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

Yes, that's a good start. I'll sleep on it.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

'deriving' is pretty much what I had in mind. If Equatable gets a magical attribute, keyword, or whatever, I'd like to use it for this feature as well.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

One big advantage methods have over properties is that you can (as of right now) get a reference to a method as a value of function type, but not a property.

True, but as with 'throws' this is a temporary limitation. I don't think we should design our reflection API around a temporary limitation. IIRC the intended plan for getting first class access to properties is a lens. If we model it as a property we will eventually get first class access to it as a lens (which means we play nice with lenses even though we don't know what they will look like yet).

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

I think it's worth consideration. Maybe views into properties can eventually conform to some sort of lens protocol or interface, allowing them to be composed and used as such.

I think Joe Groff has talked about lenses more than anyone else. Maybe he has some thoughts on this. But as I noted above, maybe exposing access as a property is the best first step. It's easy enough to wrap in a closure if you need to pass a function around before property references / lenses are introduced.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

```

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

As several have noted, this is what makes things tricky. People will want to write a library packaged as a module that can serialize private properties from types in a different module. We need to at least consider that use case.

or perhaps simply read what the people who have a lot of experience the topic have to say about it, and hopefully learn from the errors others have described. Serialization has long been a source of serious security issues in Java. Swift could do without these

-1