`shared` members and metatypes for generic types

Namespacing is one of the primary use cases. That is exactly what the constant and nested types examples show.

I'm not really clear on what you're replying to. I know it's one of the primary use cases, and I accepted that it's a good one (hence “besides namespaces” in the quoted sentence).

What would the syntax for this look like? static let x = 42 where T: _?

This is discussed in the metatypes section of the pitch. Here is an example of what they are asking:

func foo<T>() {
    // Can I access T.Shared here?  If T is Array<Int> this would be the same as Array<_>.
    // If so, what syntax do I use?
    // If T is a non-generic type T.Shared == T (i.e. the syntax used above is self-referential in this case)

    // Can I access T.self.shared here?  If T is Array<Int> this would be the same as Array<_>.self.  
    // If so, what syntax do I use?
    // If T is a non-generic type T.self.shared == T.self (i.e. the syntax used above is self-referential in this case)
}

If the language allows where on properties, then I guess that would work. But it doesn't (yet?), so you'd need an extension:

extension Foo where T: _ {
    static let x = 42
}

And ideally would be spelled extension Foo<T>, but that's something else that isn't in the language (yet?).

You can't do anything because: "Generic parameter 'T' is not used in function signature". :wink:

But let's add a parameter t: T.
You could write t as? Array<_>.
And T.self as? Array<_>.Type.

I observe your example for getting the shared type does not involve knowing it's an Array. But is there something you can you do with your shared type if it comes from an unconstrained generic type T you know nothing about?

Edit: I'm still not entirely sure I grasped what you want to accomplish here.

And neither is stored properties in extensions!

Ahh, my mistake! :slight_smile:

This is not the same as what I have pitched. Here you only have access via dynamic cast. You do not have access in the type system itself. What I have pitched is that Shared would be akin to an associated type that is available on all types. For generic types it would resolve to the type with no generic arguments bound. For non-generic types it would simply be a recursive reference (since there are no generic arguments to "unbind").

It exists as an independent type in the type system. You can compare it in constraints, so for example you could determine that T and U are instances of the same generic type (although they may have different generic arguments bound). You could also use a constraint where T.Shared: MyProtocol and proceed to use additional capabilities on the shared type.

First, this falls cleanly out of the shared metatype design I pitched and seems like an obvious thing to support under that model. Second, this is the feature that enables higher-kinded types to be encoded in a more robust manner than we can implement today.

I have been giving a lot of thought to the existential design many of you are interested in. I am coming to the conclusion that this approach is a reasonable way to generalize the design I have pitched.

However, I don't believe it serves the motivating use cases very well. Progressive disclosure and judicious use of syntactic sugar are a very important elements in the design of Swift. I don't think users should have to understand a sophisticated existential-based design like this in order to use the basic namespacing capability required by the motivating use cases. IMO, some kind of sugar indicating "namespace only / no bound arguments" is warranted even in the presence of the existential-based design. This being the case, I think it is better to start simple, focus on the concrete use cases we have, and generalize if sufficient motivating use cases for generalization arise.

I'm not advocating for a full existential-based model with the extension Foo<_> syntax. If, as you've stated, major use cases include namespacing and supporting stored properties specifically when no generic arguments are relevant (your point is well taken there), then to my mind something much simpler can happen under the surface. For example, it would seem to me that the following would suffice:

  • Notionally, for all types that have an extension Foo<_>, the compiler creates an independent non-instantiable type (let's refer to it here as AnyFoo_withSpecialMangling; obviously, it would not be visible as such to the end user) that can have stored static properties, its own conformances, and everything else that an independent type can have.
  • In contexts where the compiler today would complain that it can't infer the generic argument for a Foo<_>, instead the call is parsed as one to a member on AnyFoo_withSpecialMangling.
  • I'm unclear on why the emphasis on the shared metatype being available in a generic context. It seems to be unnecessary for the stated main use cases, but if it's necessary, I fail to see why Foo<_>.self and Foo<_>.Type are problematic ways of spelling it.

We can reject Dictionary<K, _> and Dictionary<_, V> until such time as we decide to support those. Should the need arise for a full existential model, it can then be retrofitted into the language in a source-compatible way.

This scheme is compatible with the idea of progressive disclosure. The compiler already barks at users using the notation Foo<_>. It's only a small step from there to wondering if you can actually have something named Foo<_>. Should we embark on this journey towards existentials as the underlying model, then we have an in-built mechanism for ensuring progressive disclosure as we actually build the feature itself with progressive stages of complexity (after all, we have to teach ourselves as we go).

5 Likes

I quickly scrolled over all messages and didn't found any mention to this alternative. This just works:

let fooOfInt: Foo = .makeFooOfInt()

It looks like the compiler is partly able to do what you want to achieve. In that case I think we may want to relax the inferring rule or add a special case where Foo.makeFooOfInt() means the same as I just demonstrated.


This is only solving one bit of the issues you want to fix with your pitch.

I remember this, but I am not quite sure where to find a link to it. It came up in a discussion of allowing Self and associated type constraints in protocols. Any<ProtocolName> would tell the compiler to build a type erasing thunk around the protocol that let you use it as a type (and that type would be spelled Any<ProtocolName>)

This syntax could then also be used for other things which needed compiler generated type erasure too (e.g. Any<Array>)

I liked the syntax at the time (and I kind of still do), but we seem to be moving in another direction syntax wise as swift has evolved.

Crazy idea. What if we were to spell it kind of like we would any other type which is internal to another type?

struct Foo<T> {
    shared { //No access to T inside this block
        static let message = "hello"
    }
}

Things declared inside the block would be accessible from any specialization of Foo, and anything static would also be available without defining T on Foo.

let msg = Foo.message 

If you need to refer to the shared type itself for some reason, then we could just call that Foo<_>.

The advantages:

  • You don't have to type "shared" everywhere
  • It feels like other namespaces, so it should be easy to pick up once seen
  • Stored static properties
  • Factories would work on unspecialized Foo
  • We can declare nested types (that don't reference T) on generic types

Disadvantages compared to the original proposal:

  • I don't know how you would conform it to protocols
  • Is it still useful for HKT?
3 Likes

What about using the term "unspecialized" instead of "shared" with my idea above?

struct Foo<T> {
    unspecialized {  //No access to T within this block
        static let message = "hello"

        enum Zog {
            case yes
            case no
            case getABucket
        }
    }

    // T can be accessed here
}

I thought this was about static stored properties. Those are allowed in extensions when the type is not generic.

The metatype section of your pitch doesn't make this obvious to me. It indeed says this is possible, but without an example or a use case or an explanation of what it enables.

If we need that reflection capability, it could be added with unboundType(from: T.self), or T.unboundSelf (though the later could clash with a static member of the same name). I don't think there's a way to get the base class of a class from its type currently, otherwise I'd say it should follow the same model.

I have a clear memory of jumping to attention when one of the core team members (@Joe_Groff maybe?) remarked that generalized existentials would likely cover not only protocols, but also generic classes and structs, e.g. Any<Array> (or whatever the syntax is).

The memory is clear because that’s been a longstanding point of serious pain in one of my projects, and I was excited that my pet problem might be addressed. I don't remember now whether that conversation was on swift-evo, email, or twitter. It was a loooong time ago, and I can’t seem to dig it up. Perhaps I made it up, and perhaps I misunderstood! But I do have a specific memory of being excited about being able to have the existential Resource for the generic class Resource<ContentType>.

This writeup proposed it, for example.

Yes, exactly. I'm loosely tossing out the idea that all static members should not have access to the associated type by default, and would need to explicitly declare their dependence on the associated type by piggybacking on some existing syntax — and more importantly some existing mental model.

This is not necessarily a workable idea; I don’t have a fully considered answer here. The “extension on the existential type” approach might have more going for it. Either way, I’m just looking for a way to avoid creating yet another whole new mental model for a feature that is esoteric and, let’s face it, not strictly necessary. I keep thinking of somebody saying that Swift is “a crescendo of special cases stopping just short of generality.” (Ah, it was here.)

This is a useful but small feature proposal, and it feels like the language surface area should be correspondingly small.

1 Like

Why can't the compiler work this out for itself without a keyword?

In your example the compiler knows that message doesn't depend on T and therefore should allow Foo.message. Conversely I think the compiler should ban Foo<Int>.message, since that is misleading.

Because the body could use a type parameter without it being visible in the signature, and we need to know whether a given symbol is "shared" (or whatever we call this) without looking at the body.

1 Like

But to check the 'shared' declaration you have to look at the body to make sure it didn't use the generics, therefore just look at the body and note if it uses a generic parameter or not. There will be little difference in compilation time between the two and it is simpler to let the compiler do it and more in keeping with a language that is meant to by syntactically neat.

To check an explicit "shared" attribute, you only have to type-check the body if you are compiling that body. To infer an implicit "shared" attribute, you have to type-check the body if you are type-checking any use of the declaration.

This is why we don't infer types across declaration boundaries, among other things. The "shared" attribute is subject to the same forces.

2 Likes

This is interesting, I was not aware this is possible. This is only possible for factories because of dot shorthand. This would not work for static methods or properties that do not return Foo though.

1 Like

I wouldn't want to require a block like that so it would be a redundant feature, especially given the shared extension included in the pitch which also avoids repeating shared on every declaration.

This capability is included in the pitch.

That depends on whether you expose a Shared "associated type"` on all types, including access to this type in an unconstrained generic context. You have pitched syntax, not semantics so it isn't clear what behavior you intend.

This feels like a more verbose version of unbound.

“Specialized” has a specific meaning—a version of a generic type specially compiled with some of its type parameters hardcoded—and we even have an internal @_specialize attribute with this meaning. The feature we’re discussing here is not that.

Thanks, I did not know that! That makes two behaviors of Swift I've learned from this thread tonight. :slight_smile:

What I meant by "falls cleanly out of then design I pitched" is that we need a place to put the shared members. This immediately points to the need for a shared metatype whether that is arrived at directly or indirectly via the existential model. The design I chose was to use a declaration modifier to place members in this metatype. Since the metatype exists it we should be able to refer to it in code and using the same name as the declaration modifier is the obvious choice.

The one concrete use case I am currently aware of for this metatype is in the encoding of higher-kinded types. In its simplest form, it would look something like this:

protocol TypeConstructor {
    associatedtype TypeArgument
}
struct Apply<SharedType, Argument> {
    private let value: Any
    init<T: TypeConstructor >(
       _ value: T
    ) where T.Shared == SharedType, T.Argument == Argument {
        self.value = value
    }
    func unapply<To: TypeConstructor>(
        to: To.Type
    ) -> To where To.Shared == SharedType, To.Argument == Argument {
        // This cast is guaranteed to succeed as long as the type conforming to
        // TypeConstructor correctly defines `Argument` to match its *single* type argument
        // This guarantee is derived from the guarantee that `Shared` is common to all types constructed
        // using the type constructor and *is not* the `Shared` type of *any other types*.
        return value as! To
    }
}

This encoding can be extended to work with multiple generic arguments. It can also be implemented today using a custom type for a type tag and access control can be used to mostly eliminate the potential of incorrect sharing of a type tag. However, the Shared type in this pitch would make it somewhat less boilerplate-y and eliminate the risk of tag overlap.

I imagine there are some other interesting, sophisticated techniques that can make use of the Shared type in generic context, I just don't know what they are yet.

To summarize, we need this metatype to exist and we know it can be useful so I don't know why we would want to hide it from programmers.