[Pitch] Property reflection

[Austin (Austin Zheng)]

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for
reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what
reflection in Swift might look like one day, and it's certainly not the
only possible design for such a feature. Reflection comes in many different
forms, and "no reflection" is also an option. Deciding what sort of
reflection capabilities Swift should support is a prerequisite to
stabilizing the runtime API, which I imagine has resilience consequences.
I'm not really interested in *defending* this specific proposal per se, as
I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

https://gist.github.com/austinzheng/699d47f50899b88645f56964c0b7109a

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler
to generate whatever hooks, metadata, and support code is necessary. Once a
type conforms, the interface to the reflection features naturally present
themselves as protocol methods. It would be great to allow an extension to
retroactively enable reflection on a type vended by another module,
although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the
following categories: typed vs untyped, get-only versus get-set. A view is
a struct representing a property on an instance of a type (or maybe a
metatype, for type properties). It allows you to get information about that
property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it
later to a get-set view, if the underlying property is get-set. If you
don't care about setting, though, you can just work exclusively with
get-only views.)

It supports both typed and untyped access. You can ask for a property view
specifically for (e.g.) a `String` property, and if you get one you can be
assured that your getting and setting operations will be type safe. You can
also ask for an "untyped" property view that exposes the value as an Any,
and allows you to try (and possibly fail, with a thrown error) to set the
value.

The requirements part of it is composable. For example, you can imagine a
future "FullyReflectable" protocol that simply inherits from
"PropertyReflectable", "MethodReflectable", and other reflectable
protocols. Or maybe a library requires reflection access to types that it
needs to work with, and it can create its own protocols that inherit from
"PropertyReflectable" and naturally enforce reflection support on the
necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't
necessarily see all the types, though, and the interface is pretty
straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination
is that it would only expose the properties you'd be able to directly
access), or property behaviors (I think get-set behavior is fundamental to
properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or
existentials.

Anyways, thanks for reading all the way to the end, and any feedback,
criticism, or alternative proposals would be greatly appreciated.

Best,
Austin

[Callionica_Swift]

This is important and really useful. There's a lot to like here. I have
been thinking about property reflection recently too. Don't have time to
write too much on it now, but was wondering what was the thinking behind
suggesting an opt-in? For me, I would hope that getting access to a
single property object that holds the getter and setter functions would
be almost as simple as the way we obtain a function reference for a method
(by naming the item in code) and would hope that it would work for all
types.

-- Callionica

···

On Thursday, May 26, 2016, Austin Zheng via swift-evolution < swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for
reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what
reflection in Swift might look like one day, and it's certainly not the
only possible design for such a feature. Reflection comes in many different
forms, and "no reflection" is also an option. Deciding what sort of
reflection capabilities Swift should support is a prerequisite to
stabilizing the runtime API, which I imagine has resilience consequences.
I'm not really interested in *defending* this specific proposal per se,
as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler
to generate whatever hooks, metadata, and support code is necessary. Once a
type conforms, the interface to the reflection features naturally present
themselves as protocol methods. It would be great to allow an extension to
retroactively enable reflection on a type vended by another module,
although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the
following categories: typed vs untyped, get-only versus get-set. A view is
a struct representing a property on an instance of a type (or maybe a
metatype, for type properties). It allows you to get information about that
property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it
later to a get-set view, if the underlying property is get-set. If you
don't care about setting, though, you can just work exclusively with
get-only views.)

It supports both typed and untyped access. You can ask for a property view
specifically for (e.g.) a `String` property, and if you get one you can be
assured that your getting and setting operations will be type safe. You can
also ask for an "untyped" property view that exposes the value as an Any,
and allows you to try (and possibly fail, with a thrown error) to set the
value.

The requirements part of it is composable. For example, you can imagine a
future "FullyReflectable" protocol that simply inherits from
"PropertyReflectable", "MethodReflectable", and other reflectable
protocols. Or maybe a library requires reflection access to types that it
needs to work with, and it can create its own protocols that inherit from
"PropertyReflectable" and naturally enforce reflection support on the
necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't
necessarily see all the types, though, and the interface is pretty
straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my
inclination is that it would only expose the properties you'd be able to
directly access), or property behaviors (I think get-set behavior is
fundamental to properties, although "behavior metadata" on the views might
be useful).

I'd also have to figure out how it would operate with generic types or
existentials.

Anyways, thanks for reading all the way to the end, and any feedback,
criticism, or alternative proposals would be greatly appreciated.

Best,
Austin

[anandabits (Matthew Johnson)]

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

···

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[plx]

I think this proposal should “simmer" until we know what property behaviors will look like (or find out they are never happening, etc.).

The interaction with “property behaviors” seems likely to be quite subtle, even for something simple like `lazy`.

For sake of argument, let’s say that a `lazy` declaration like so:

  class SomeClass {
    lazy var foo: Foo = Foo(bar: self.prepareBar())
  }

…gets implicitly-expanded into e.g. something like the below:

  class SomeClass {

    var foo: Foo {
      guard let foo = _foo else {
        _foo = Foo(bar: self.prepareBar())
        return _foo
      }
      return foo
    }

    private var _foo: Foo? = nil

  }

…which immediately exposes a few lazy-specific questions:

- should `foo` be exposed via reflection? (IMHO: yes)
- should `_foo` be exposed via reflection? (IMHO: probably not, but not certain)
- should `foo`’s lazy-ness be exposable? (IMHO: yes, but how exactly?)

…as well as a few more-general questions:

- should computed properties, in general, be exposed to reflection? (IMHO: probably not, but there are some exceptions…)
- if user-specified property behaviors get special exposure, how should that work?
- how should reflection work for enums/enums-with-payloads?

Finally, I worry a lot less about the details of getter/setter pairs than I worry about being able to use reflection for construction/initialization.

I don’t have any actual proposal on that front, but it seems like it should be factored into any reflection design.

···

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[L_Mihalkovic]

It might be good to learn from the mistakes java made with its reflection API because it might help avoid facing the impedence mismatch java faced a couple years ago when they tried to model the language on top of reflection.
The point is I think that starting in a small corner and trying to walk your way up is like the quantum mechanical way of trying to capture a lion (put the cage on the table and wait for the lion to show up inside), which in this case means do something somewhere, repeat, and hope it collectively makes sense when it's all done.
IMO the design does not emerge spontaneously from the syntax, it is the syntax that is created because it serves some higher order principles.

···

On May 27, 2016, at 3:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[rparada (Ricardo Parada)]

Can you elaborate on what this API would be used for? KVC? For instance, I played with Mirror the other day and my code to get a value given the property name looked more like this:

let age = myPerson.value(forKey:”age”) as! Int

And this is what I did:

// KVC stands for key-value-coding… but I only know how to get values. I don’t know how to set values

protocol KVC {
   func value(forKey key: String) -> Any!
}

// Default implementation
extension KVC {
   func value(forKey key: String) -> Any! {
       let aMirror = Mirror(reflecting:self)
       for case let (label?, value) in aMirror.children {
           if label == key {
               return value
           }
       }
       return nil
   }
}

public struct Person : KVC {
   let firstName: String
   let lastName: String
   let age: Int

   func fullName() -> String {
       return "\(firstName) \(lastName)"
   }
}

let aPerson = Person(firstName:"John", lastName:"Doe", age:48)

// It works for stored properties
let lastName = aPerson.value(forKey:"lastName") as! String
print("Last name is \(lastName)")

// It does not work for instance methods, i.e. fullName is not a stored property
let fullName = aPerson.value(forKey:"fullName")
if fullName != nil {
   print("Full name is \(fullName)")
} else {
   print("Unable to get fullName via KVC")
}

···

On May 26, 2016, at 9:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

[Anders]

I wonder how the views would work with the value semantic of structs.

The view types proposed, if not being a compiler magic, would need to have a reference to the reflected instance. They are also escapable unless special rules for the view types are enforced. So… perhaps we would want less magic and special rules?

IMO these views are not necessary, and can be consolidated into typed/untyped get/set calls on `Reflectable`. However, if the intention is to amortize the type metadata lookup, or not to introduce any kind of lookup caching in the runtime, perhaps we can introduce instead property descriptors.

(A quick dump of my idea: https://gist.github.com/andersio/9ff02257b5c89b35fd523dcd09e484e4\)

Best Regards
Anders

···

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Austin (Austin Zheng)]

Thanks, as always, for the thoughtful feedback. (inline)

These names are a good place to start but I agree that it would be nice to
improve them. I will give it some thought. One comment for now - you use
both `get` / `set` and `read` / `write`. It’s probably better to pick one.

Yes, that's a good start. I'll sleep on it.

It's opt-in: types have to conform to a special protocol for the compiler
to generate whatever hooks, metadata, and support code is necessary. Once a
type conforms, the interface to the reflection features naturally present
themselves as protocol methods. It would be great to allow an extension to
retroactively enable reflection on a type vended by another module,
although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go
in that direction for Equatable, Hashable, and other synthesized
conformances).

'deriving' is pretty much what I had in mind. If Equatable gets a magical
attribute, keyword, or whatever, I'd like to use it for this feature as
well.

It uses "views": there are four types of views, two of each in the
following categories: typed vs untyped, get-only versus get-set. A view is
a struct representing a property on an instance of a type (or maybe a
metatype, for type properties). It allows you to get information about that
property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

One big advantage methods have over properties is that you can (as of right
now) get a reference to a method as a value of function type, but not a
property.

I am wondering how this might relate to lenses. If we’re going to
introduce those it feels like the property value should be introduced as a
lens. I’m unsure of exactly what that would look like but I it is
definitely worth thinking about.

I think it's worth consideration. Maybe views into properties can
eventually conform to some sort of lens protocol or interface, allowing
them to be composed and used as such.

Another option if we don’t go with a lens is a simple property (`var value
{ get }` and `var value { get set }`). IIRC we are going to have throwing
computed properties eventually so you could still throw from the setter.

```

I'm not yet sure how it should interact with access control (my
inclination is that it would only expose the properties you'd be able to
directly access),

This one is tricky. I am generally be opposed to any way to get around
access control. But people are going to implement things like
serialization using this which may require access to private properties. I
think we want to try to understand the consequences of different options
and when in doubt decide in favor caution.

My personal preference would be to honor access control, and consider
ser/de separately (especially since there are so many other possible
considerations for that feature). Access control in Swift isn't just
another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be
turning JSON or XML received from a network call into model objects for an
app. There are quite a few Objective-C libraries that use KVC to implement
this functionality.

or property behaviors (I think get-set behavior is fundamental to
properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes
which we would also want to have available. It’s probably better to make
available anything that could be useful and isn’t too costly to provide.
It could all live behind a `metadata` property so we don’t clutter the
interface of the views themselves.

Absolutely. It's not clear right now what property behaviors will end up
looking like, but if possible I'd definitely like to reify them and expose
them to the end user (like how the T.Type is exposed in the gist). At the
very least (and this is definitely insufficient), there should be a list of
strings describing the behaviors.

···

On Thu, May 26, 2016 at 7:20 PM, Matthew Johnson <matthew@anandabits.com> wrote:

I'd also have to figure out how it would operate with generic types or
existentials.

Anyways, thanks for reading all the way to the end, and any feedback,
criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Austin (Austin Zheng)]

Thank you! If you have other ideas I'd love to hear them, even if they're
completely different. I think this is a topic that deserves to 'simmer' for
a while.

You bring up an important point: should reflection be opt-in or opt-out?
I've seen arguments for both options, and I think it's definitely a point
that needs to be very carefully discussed. Importantly, opt-out makes this
a sort-of-zero cost abstraction; only types that need the functionality
take the performance and metadata size hit. This is something that the core
team seems to find important.

If it is opt-in, though, there are certainly a bunch of framework base
types that should be opted-in, so that all their subclasses can be
reflected upon, providing most of the practical benefits of default opt-in.
Giving UIView and NSView this functionality would be a starting point, and
*maybe* NSObject (although that would have effects upon all of Swift
Foundation).

Best,
Austin

···

On Thu, May 26, 2016 at 7:42 PM, Callionica (Swift) < swift-callionica@callionica.com> wrote:

This is important and really useful. There's a lot to like here. I have
been thinking about property reflection recently too. Don't have time to
write too much on it now, but was wondering what was the thinking behind
suggesting an opt-in? For me, I would hope that getting access to a
single property object that holds the getter and setter functions would
be almost as simple as the way we obtain a function reference for a method
(by naming the item in code) and would hope that it would work for all
types.

-- Callionica

On Thursday, May 26, 2016, Austin Zheng via swift-evolution < > swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for
reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what
reflection in Swift might look like one day, and it's certainly not the
only possible design for such a feature. Reflection comes in many different
forms, and "no reflection" is also an option. Deciding what sort of
reflection capabilities Swift should support is a prerequisite to
stabilizing the runtime API, which I imagine has resilience consequences.
I'm not really interested in *defending* this specific proposal per se,
as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler
to generate whatever hooks, metadata, and support code is necessary. Once a
type conforms, the interface to the reflection features naturally present
themselves as protocol methods. It would be great to allow an extension to
retroactively enable reflection on a type vended by another module,
although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the
following categories: typed vs untyped, get-only versus get-set. A view is
a struct representing a property on an instance of a type (or maybe a
metatype, for type properties). It allows you to get information about that
property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it
later to a get-set view, if the underlying property is get-set. If you
don't care about setting, though, you can just work exclusively with
get-only views.)

It supports both typed and untyped access. You can ask for a property
view specifically for (e.g.) a `String` property, and if you get one you
can be assured that your getting and setting operations will be type safe.
You can also ask for an "untyped" property view that exposes the value as
an Any, and allows you to try (and possibly fail, with a thrown error) to
set the value.

The requirements part of it is composable. For example, you can imagine a
future "FullyReflectable" protocol that simply inherits from
"PropertyReflectable", "MethodReflectable", and other reflectable
protocols. Or maybe a library requires reflection access to types that it
needs to work with, and it can create its own protocols that inherit from
"PropertyReflectable" and naturally enforce reflection support on the
necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't
necessarily see all the types, though, and the interface is pretty
straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my
inclination is that it would only expose the properties you'd be able to
directly access), or property behaviors (I think get-set behavior is
fundamental to properties, although "behavior metadata" on the views might
be useful).

I'd also have to figure out how it would operate with generic types or
existentials.

Anyways, thanks for reading all the way to the end, and any feedback,
criticism, or alternative proposals would be greatly appreciated.

Best,
Austin

[Austin (Austin Zheng)]

(inline)

It might be good to learn from the mistakes java made with its reflection API because it might help avoid facing the impedence mismatch java faced a couple years ago when they tried to model the language on top of reflection.

Of course. What might those mistakes be? It would be interesting to see how the relatively limited expressivity of Java informed the design of its reflection APIs (which are very different from those of Objective-C as far as I know).

The point is I think that starting in a small corner and trying to walk your way up is like the quantum mechanical way of trying to capture a lion (put the cage on the table and wait for the lion to show up inside), which in this case means do something somewhere, repeat, and hope it collectively makes sense when it's all done.

There is an overarching (albeit vague) design I have in mind, of which this is one piece. I wanted to gather some preliminary feedback to see whether or not this was something worth pursuing further. Should I share the overarching design instead?

···

On May 27, 2016, at 12:54 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com> wrote:

IMO the design does not emerge spontaneously from the syntax, it is the syntax that is created because it serves some higher order principles.

On May 27, 2016, at 3:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

[Brent_Royal-Gordon]

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

I had some thoughts about an alternate design based partially on access control concerns. Sketching roughly, it looks like this:

* There is a pseudo-property called, say, `properties` on every type (and perhaps on every instance). (Alternate designs are available, like an `inout`-returning pseudo-function. It doesn't really matter.)

* `properties` is a dictionary, or at least something very like a dictionary: you can look something up by key or iterate over the available keys. Its keys are strings, and its values are lens functions. These lens functions return type Any; their setters throw if you assign an incompatible type.

* The contents of `properties` are the properties visible *at the site where it is called*. That means calling `properties` in different places will give you different results. If you import a type from another module and then add properties in an `internal` extension, you'll see the other module's `public` properties plus your `internal` ones.

* You can pass your `properties` dictionary to other code; that effectively delegates your access to that other code. Thus, if your main class body passes its `property` dictionary to a serialization library, that library can access your private properties. If you pass it `inout`, then the library can modify your private properties.

* There is no opting in, but I *think* the compiler has enough information to figure out what goes into `properties` at the sites where it is used, so you only have to pay for it if and when you use it. I could be wrong, though—there may be something there that I'm missing.

···

--
Brent Royal-Gordon
Architechies

[anandabits (Matthew Johnson)]

I wonder how the views would work with the value semantic of structs.

Not all value types have value semantics. It's important to not forget that. I would like to see a way to make that distinction clear in our code but that is a conversation for another thread.

The view types proposed, if not being a compiler magic, would need to have a reference to the reflected instance. They are also escapable unless special rules for the view types are enforced. So… perhaps we would want less magic and special rules?

IMO these views are not necessary, and can be consolidated into typed/untyped get/set calls on `Reflectable`. However, if the intention is to amortize the type metadata lookup, or not to introduce any kind of lookup caching in the runtime, perhaps we can introduce instead property descriptors.

(A quick dump of my idea:
https://gist.github.com/andersio/9ff02257b5c89b35fd523dcd09e484e4\)

Property descriptors could be useful in the sense that they wouldn't need to refer back to the instance. But I would also like to see a way to get something like a lens into the property for a specific instance which is what the views allow for. This design doesn't allow for that. Maybe we want to allow you to query for property descriptors alone, lenses alone, or the combination in a view.

···

Sent from my iPad

On May 27, 2016, at 5:45 AM, Anders Ha via swift-evolution <swift-evolution@swift.org> wrote:

Best Regards
Anders

On 27 May 2016, at 9:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

Not just deserialization, but also for example, if you take upon something like Swift version of CoreData, some vars may be private, but the framework will need access to them. Of course, this could be handled by some annotation giving access to the variable via reflection.

I believe CoreData uses synthesized computed properties that access values stored in an internal row cache (or something along those lines).

I'm in favor of reflecting all vars by default (even private ones) with an option to opt-out.

I much prefer the opt-in approach if we're going to allow for reflection to have different visibility. Access control should be explicit IMO.

···

Sent from my iPad

On May 27, 2016, at 8:25 AM, Charlie Monroe <charlie@charliemonroe.net> wrote:

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[hartbit (David Hart)]

That's what I want to use it for and I'm fairly sure we need access to private properties for that.

···

On 27 May 2016, at 04:44, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

[Panajev (Goffredo Marocchi)]

You made me think about JNI and now I am sad... You monster :P.

···

Sent from my iPhone

On 27 May 2016, at 08:54, L. Mihalkovic via swift-evolution <swift-evolution@swift.org> wrote:

It might be good to learn from the mistakes java made with its reflection API because it might help avoid facing the impedence mismatch java faced a couple years ago when they tried to model the language on top of reflection.
The point is I think that starting in a small corner and trying to walk your way up is like the quantum mechanical way of trying to capture a lion (put the cage on the table and wait for the lion to show up inside), which in this case means do something somewhere, repeat, and hope it collectively makes sense when it's all done.
IMO the design does not emerge spontaneously from the syntax, it is the syntax that is created because it serves some higher order principles.

On May 27, 2016, at 3:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[anandabits (Matthew Johnson)]

Thanks, as always, for the thoughtful feedback. (inline)

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

Yes, that's a good start. I'll sleep on it.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

'deriving' is pretty much what I had in mind. If Equatable gets a magical attribute, keyword, or whatever, I'd like to use it for this feature as well.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

One big advantage methods have over properties is that you can (as of right now) get a reference to a method as a value of function type, but not a property.

True, but as with 'throws' this is a temporary limitation. I don't think we should design our reflection API around a temporary limitation. IIRC the intended plan for getting first class access to properties is a lens. If we model it as a property we will eventually get first class access to it as a lens (which means we play nice with lenses even though we don't know what they will look like yet).

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

I think it's worth consideration. Maybe views into properties can eventually conform to some sort of lens protocol or interface, allowing them to be composed and used as such.

I think Joe Groff has talked about lenses more than anyone else. Maybe he has some thoughts on this. But as I noted above, maybe exposing access as a property is the best first step. It's easy enough to wrap in a closure if you need to pass a function around before property references / lenses are introduced.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

```

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

My personal preference would be to honor access control, and consider ser/de separately (especially since there are so many other possible considerations for that feature). Access control in Swift isn't just another safety feature, it's also a boundary for things like the optimizer.

To be honest, I expect the first big use of a feature like this to be turning JSON or XML received from a network call into model objects for an app. There are quite a few Objective-C libraries that use KVC to implement this functionality.

As several have noted, this is what makes things tricky. People will want to write a library packaged as a module that can serialize private properties from types in a different module. We need to at least consider that use case. Supporting reflection-specific visibility might be a reasonable way to strike a balance. But I'm sure it would also receive a lot of resistance on both sides (those who don't think it's necessary to make internals more visible and those who want them to be visible without extra boilerplate).

···

Sent from my iPad

On May 26, 2016, at 9:44 PM, Austin Zheng <austinzheng@gmail.com> wrote:

On Thu, May 26, 2016 at 7:20 PM, Matthew Johnson <matthew@anandabits.com> wrote:

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

Absolutely. It's not clear right now what property behaviors will end up looking like, but if possible I'd definitely like to reify them and expose them to the end user (like how the T.Type is exposed in the gist). At the very least (and this is definitely insufficient), there should be a list of strings describing the behaviors.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Austin (Austin Zheng)]

I think there is a consensus forming that we probably don't yet know enough about Swift's future capabilities to nail down specific designs yet. (Not that I want to stop people from talking about them if they wish.)

However, now that there is a conversation topic about reflection, I also wanted to ask the following questions:

1. What sorts of features are important, and what would you use them for?

For me, the list looks like the following:

- Describe the structure of a type. (We have that right now, with introspection using "Mirror".)

- Find all the properties on a type. Get and set their values, and get information about them (like their dynamic type).
- Find all the methods on a type. Be able to get a reference to and invoke a method on an instance of a type.
- Find all the subscripts and initializers on a type?

- Get a list of all concrete types opting in to extended reflection.
- Get a list of all concrete types visible to the caller that conform to some protocol(s), possibly with additional requirements on associated types.
- Get a list of all protocols.

- Create a reference to a type using a string, and perform certain actions using that type. (It would be interesting if you could say "try parsing this type into a concrete type that conforms to X, Y, and Z protocols", and if successful you get a metatype to use or something you can pass to a generic function.
- Create a reference to a method or property using a string, and use it as above.

- Reify methods and properties marked with the 'dynamic' keyword in such a way that where they are dispatched to can be controlled at runtime. For example, maybe a dynamic method of a class might use some default implementation, or if some condition is met forward invocations of it to a dynamic method on another class instead.

2. What sorts of priorities should inform Swift's reflection capabilities?

For me, the option to perform both "statically typed" and runtime type checked operations, when feasible, is a big one. I think something like variadic generics could allow for typesafe reflection on methods and functions at runtime - rather than having to call a performSelector: like method, it would be possible to get a full-fledged value of function type and use it just like any other method. This is something few languages have AFAIK.

The reflection machinery should cooperate with the rest of the language. If there must be a way to, for example, access private members of an instance through reflection, it should not be unconditional and should be carefully considered such that there are certain invariants that are still honored. Like Laurent said earlier, bad things can happen if you use reflection to subvert access control.

I think reflection should be opt-in in most cases. Reflection opting-in should be composable, inheritable, and retroactive, which covers the most common cases in which reflection would be useful: working with Cocoa/Cocoa Touch, for example.

Best,
Austin

···

On May 27, 2016, at 7:37 AM, plx via swift-evolution <swift-evolution@swift.org> wrote:

I think this proposal should “simmer" until we know what property behaviors will look like (or find out they are never happening, etc.).

The interaction with “property behaviors” seems likely to be quite subtle, even for something simple like `lazy`.

For sake of argument, let’s say that a `lazy` declaration like so:

  class SomeClass {
    lazy var foo: Foo = Foo(bar: self.prepareBar())
  }

…gets implicitly-expanded into e.g. something like the below:

  class SomeClass {

    var foo: Foo {
      guard let foo = _foo else {
        _foo = Foo(bar: self.prepareBar())
        return _foo
      }
      return foo
    }

    private var _foo: Foo? = nil

  }

…which immediately exposes a few lazy-specific questions:

- should `foo` be exposed via reflection? (IMHO: yes)
- should `_foo` be exposed via reflection? (IMHO: probably not, but not certain)
- should `foo`’s lazy-ness be exposable? (IMHO: yes, but how exactly?)

…as well as a few more-general questions:

- should computed properties, in general, be exposed to reflection? (IMHO: probably not, but there are some exceptions…)
- if user-specified property behaviors get special exposure, how should that work?
- how should reflection work for enums/enums-with-payloads?

Finally, I worry a lot less about the details of getter/setter pairs than I worry about being able to use reflection for construction/initialization.

I don’t have any actual proposal on that front, but it seems like it should be factored into any reflection design.

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[L_Mihalkovic]

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

These names are a good place to start but I agree that it would be nice to improve them. I will give it some thought. One comment for now - you use both `get` / `set` and `read` / `write`. It’s probably better to pick one.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

What do you think of using the `deriving` syntax for this (assuming we go in that direction for Equatable, Hashable, and other synthesized conformances).

I saw this syntax show up in the equatable/hashable context (could not convince myself of its value), are there other areas where it was put forward?

···

On May 27, 2016, at 4:20 AM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

On May 26, 2016, at 8:25 PM, Austin Zheng via swift-evolution <swift-evolution@swift.org> wrote:

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

How did you arrive at `get` and `set` methods?

I am wondering how this might relate to lenses. If we’re going to introduce those it feels like the property value should be introduced as a lens. I’m unsure of exactly what that would look like but I it is definitely worth thinking about.

Another option if we don’t go with a lens is a simple property (`var value { get }` and `var value { get set }`). IIRC we are going to have throwing computed properties eventually so you could still throw from the setter.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access),

This one is tricky. I am generally be opposed to any way to get around access control. But people are going to implement things like serialization using this which may require access to private properties. I think we want to try to understand the consequences of different options and when in doubt decide in favor caution.

or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

Not just behavior metadata. Someday we might have user-defined attributes which we would also want to have available. It’s probably better to make available anything that could be useful and isn’t too costly to provide. It could all live behind a `metadata` property so we don’t clutter the interface of the views themselves.

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

[Austin (Austin Zheng)]

Yes, it was pretty much meant as a KVC-like feature for Swift. Get a
reference to a property from a string which would allow you to get and set
its value.

···

On Fri, May 27, 2016 at 11:53 AM, Ricardo Parada <rparada@mac.com> wrote:

On May 26, 2016, at 9:25 PM, Austin Zheng via swift-evolution < > swift-evolution@swift.org> wrote:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

Can you elaborate on what this API would be used for? KVC? For instance,
I played with Mirror the other day and my code to get a value given the
property name looked more like this:

let age = myPerson.value(forKey:”age”) as! Int

And this is what I did:

// KVC stands for key-value-coding… but I only know how to get values. I
don’t know how to set values

protocol KVC {
   func value(forKey key: String) -> Any!
}

// Default implementation
extension KVC {
   func value(forKey key: String) -> Any! {
       let aMirror = Mirror(reflecting:self)
       for case let (label?, value) in aMirror.children {
           if label == key {
               return value
           }
       }
       return nil
   }
}

public struct Person : KVC {
   let firstName: String
   let lastName: String
   let age: Int

   func fullName() -> String {
       return "\(firstName) \(lastName)"
   }
}

let aPerson = Person(firstName:"John", lastName:"Doe", age:48)

// It works for stored properties
let lastName = aPerson.value(forKey:"lastName") as! String
print("Last name is \(lastName)")

// It does not work for instance methods, i.e. fullName is not a stored
property
let fullName = aPerson.value(forKey:"fullName")
if fullName != nil {
   print("Full name is \(fullName)")
} else {
   print("Unable to get fullName via KVC")
}

[L_Mihalkovic]

When compared to the java language itself, the reflection API was close-enough but lossy. So when it came time to implement javax.lang.model over reflection there were naturally gaps (similar notions were wrongly viewed as different problems). Narrowed down and translated back to swift, a lesson might be that any solution that does not present some clean symmetry in its treatment of nominal vs structural types might be a non-starter (you can read the compiler source code to see an example of how far they have been able to maintain some symmetry between them).

just a thought.

···

On May 27, 2016, at 10:31 AM, Austin Zheng <austinzheng@gmail.com> wrote:

(inline)

On May 27, 2016, at 12:54 AM, L. Mihalkovic <laurent.mihalkovic@gmail.com <mailto:laurent.mihalkovic@gmail.com>> wrote:

It might be good to learn from the mistakes java made with its reflection API because it might help avoid facing the impedence mismatch java faced a couple years ago when they tried to model the language on top of reflection.

Of course. What might those mistakes be? It would be interesting to see how the relatively limited expressivity of Java informed the design of its reflection APIs (which are very different from those of Objective-C as far as I know).

The point is I think that starting in a small corner and trying to walk your way up is like the quantum mechanical way of trying to capture a lion (put the cage on the table and wait for the lion to show up inside), which in this case means do something somewhere, repeat, and hope it collectively makes sense when it's all done.

There is an overarching (albeit vague) design I have in mind, of which this is one piece. I wanted to gather some preliminary feedback to see whether or not this was something worth pursuing further. Should I share the overarching design instead?

IMO the design does not emerge spontaneously from the syntax, it is the syntax that is created because it serves some higher order principles.

On May 27, 2016, at 3:25 AM, Austin Zheng via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hi swift-evolution,

For those who are interested I'd like to present a pre-pre-proposal for reflection upon a type's properties and solicit feedback.

First of all, some caveats: this is only a very small piece of what reflection in Swift might look like one day, and it's certainly not the only possible design for such a feature. Reflection comes in many different forms, and "no reflection" is also an option. Deciding what sort of reflection capabilities Swift should support is a prerequisite to stabilizing the runtime API, which I imagine has resilience consequences. I'm not really interested in defending this specific proposal per se, as I am looking for a jumping-off point to explore designs in this space.

Anyways, here is a gist outlining the public API to the feature:

An exploration of a possible KVC-like API for a future version of Swift. · GitHub

A couple of notes regarding the proposal:

The API names need improvement. Suggestions welcome.

It's opt-in: types have to conform to a special protocol for the compiler to generate whatever hooks, metadata, and support code is necessary. Once a type conforms, the interface to the reflection features naturally present themselves as protocol methods. It would be great to allow an extension to retroactively enable reflection on a type vended by another module, although I have no idea how feasible that is.

It uses "views": there are four types of views, two of each in the following categories: typed vs untyped, get-only versus get-set. A view is a struct representing a property on an instance of a type (or maybe a metatype, for type properties). It allows you to get information about that property (like its name) and try getting and setting its values.

(You can get a get-only view to a property, and then try and upgrade it later to a get-set view, if the underlying property is get-set. If you don't care about setting, though, you can just work exclusively with get-only views.)

It supports both typed and untyped access. You can ask for a property view specifically for (e.g.) a `String` property, and if you get one you can be assured that your getting and setting operations will be type safe. You can also ask for an "untyped" property view that exposes the value as an Any, and allows you to try (and possibly fail, with a thrown error) to set the value.

The requirements part of it is composable. For example, you can imagine a future "FullyReflectable" protocol that simply inherits from "PropertyReflectable", "MethodReflectable", and other reflectable protocols. Or maybe a library requires reflection access to types that it needs to work with, and it can create its own protocols that inherit from "PropertyReflectable" and naturally enforce reflection support on the necessary types.

It looks a bit cumbersome, but there's room for refinement. Users won't necessarily see all the types, though, and the interface is pretty straightforward:

myPerson.typedReadWriteProperty<Int>("age")?.set(30)

try myPerson.allNamedProperties["age"]?.set(30)

I'm not yet sure how it should interact with access control (my inclination is that it would only expose the properties you'd be able to directly access), or property behaviors (I think get-set behavior is fundamental to properties, although "behavior metadata" on the views might be useful).

I'd also have to figure out how it would operate with generic types or existentials.

Anyways, thanks for reading all the way to the end, and any feedback, criticism, or alternative proposals would be greatly appreciated.

Best,
Austin
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution