`shared` members and metatypes for generic types

For me, this has been e.g. for APICall<T> - a base "abstract" class for API calls that return various values. I needed to track ongoing API calls, etc., so I needed an array of [APICall]. Yeah, sure, you can create a protocol a work around that this way, but I've ended up (for various reasons) creating a APICallBase that is not generic and contains some basic common implementation. I believe I've done so in one or two more cases that I have a non-generic Base class to work around this.

As per the topic itself, I've found the following usecase:

class Preferences {
    struct Key<T>: RawRepresentable {
        var rawValue: String
    }

    func bool(for key: Key<Bool>) -> Bool { ... }
    func value<T>(for key: Key<T>) -> T? { ... }
    func value<T>(for key: Key<T>, defaultValue: T) -> T { ... }
}

This is a class representing typed preferences. The ideal usage would be something like this:

extension Preferences.Key {
    static let myCoolPreference = Key<Bool>(rawValue: "MyCoolPreference")
}

extension Preferences {
    var isMyCoolPreferenceEnabled: Bool {
        get { return self.bool(for: .myCoolPreference) }
        set { ... }
    }
}

What's currently impossible is to create the extension of Key and then address it with the short style .myCoolPreference. Maybe I'm missing something, but something like this would create nice and type-safe preferences wrappers. You can nowadays declare various constants for accessing defaults, but they are untyped.

Of course, there is a workaround:

class Preferences {
     // Untyped
    struct Key: RawRepresentable {
        var rawValue: String
    }

     // Typed
    struct TypedKey<T>: RawRepresentable {
        var rawValue: String
    }
    func bool(for key: TypedKey<Bool>) -> Bool { ... }
    func value<T>(for key: TypedKey<T>) -> T? { ... }
    func value<T>(for key: TypedKey<T>, defaultValue: T) -> T { ... }
}

Then you declare the static let on the untyped Key and address it as self.bool(for: Key.myCoolPreference). Works, but isn't a nice solution.

It is jargony and that's why I don't like it that much. I don't think this feature is that esoteric. Constants and factories are not esoteric at all. I think the feature would be pretty widely used by anyone writing generic types.

I don't like it because there are many ways to interpret what "generic" refers to. In fact, shared members could be generic:

extension Foo {
    shared func makeFoo<T>(_ array: [T]) -> Foo<Int>
}

It would be extremely strange to see nongeneric in place of shared there!

As I mentioned earlier, the name doesn't matter to me beyond the criteria that it should not be actively confusing as nongeneric would be above, and ideally it would be accessible / meaningful to people who just want to use it for simple things like constants and factories.

I don't have any concrete examples for you, except that by refactoring metatypes we also would like to merge MemoryLayout into the new metatype itself and provide access to it though a property that won't eventually collide with any static code out there. In that sense MemoryLayout is a type that actually provides some information about the generic type it captures, similar to how a metatype provides access to some of the types members. That makes it a perfect candidate to be merged into a metatype later. If you now look at it from that angle you may want to create a standalone type similar to MemoryLayout that would provide the functionality you try to achieve in your pitch. Later on if we're going to refactor metatypes we could just merge the new type and provide good prefix.

1 Like

I'd suggest namic such as static(shared) var foo: Int = 0 or similar instead of replacing the static altogether. Just add a modifier to static so that it's clear that it's static, but applies to the entire type.

That wouldn't work at all. MemoryLayout provides the same information for all types. The entire point of this pitch is to allow the implementation of each generic type to define its own set of shared members.

FWIW, I still don't see how this would substantially complicate metatype refactoring. IIRC @beccadax was involved in that discussion quite a bit as well. Brent, if you're familiar with metatype refactoring, do you believe this concern is warranted?

I wasn't saying it will make it impossible to refactor metatypes, but rather it can but not necessarily would
make it complicated (I'm not 100% sure though). Sorry if I confused you.

If you're able to demonstrate a real difficulty this would introduce you have my ear. Short of that, if this were to make it to proposal and review I'm comfortable letting the core team consider any vague concerns that are raised. They are in a better position than most of us to make that kind of determination.

Since properties must be computed, everything that could make use of the proposed feature could be implemented in an extension. Therefore, I think a possible syntax for this that doesn't require a new keyword would be to allow extensions spelled:

extension Foo where T : _ {
// or eventually, `extension Foo<_>` when that syntax is supported
  static var message: String { return "hello" }
  static func makeDefault() -> Foo<Int> { /* ... */ }
}

This would allow this information to be associated with the definition of the member rather than the call site, without need for a new keyword.

8 Likes

The main benefit of where T: _ is that it can blank only a certain portion of the generic types in scope, but I don't really know if there are many use case for this. I can't say I'm convinced either, but I find it makes the intent clearer than any keyword name I've seen so far. It also fits well with Foo<_>.Type which I think is clearer than Foo.Shared.

1 Like

Would be good to overcome this shortcoming, preferably without a new keyword or even a where constraint.

...and agreed re HKTs; far preferable that the Swift type system eventually evolve to support this.

If I'm understanding you correctly, that's too narrow a view. An “existential type,” at least as I understand the term, is any kind of abstract type that gathers together a family of concrete types by exposing some characteristic they have in common (e.g. a shared method, such as all sequences having a count), while preserving each value’s more specific type information at runtime.

Existentials certainly aren’t specific to protocols. We most commonly the term for “a generic type with unknown type parameters,” as in Java’s List<?>, but even in this narrower context, the term isn’t just for protocols/interfaces. And “existential” also refers to things other than unknown type parameters: Any is an existential in Swift, and IIRC the compiler authors refer to it as such; this proposal uses the word “existential” to refer to the family of types having a given superclass. (The rather dense paper that introduced them is titled “Abstract Types Have Existential Type.”) And of course all protocols — with associated types or without — are existential types.

Am I off base here? I'm pretty sure I'd heard core team members say they expect generalized existentials to include at the very least class, struct, and enum types with unknown type parameters. Maybe a suitable Swift authority can clarify.


If I am understanding the term right, then these types would all exist with generalized existentials (just making up a syntax):

Any<Array> // Array of something; `count` still exposed, subscript returns `Any` and is read-only
[Any<Array>] // Array of arrays of unknown elements; could be used to sum counts
Any<Array where .Element: Encodable> // Would cover e.g. Array<Int> and Array<[String:String]>
Any<Foo> // Your Foo up top

Past threads have even proposed spelling generalized existentials like this:

Array
[Array]
Array where .Element: Encodable
Foo

However they’re spelled, presumably these existential types all have metatypes.

At the point we have these generalized existentials, it’s not clear to me that there’s a difference between Any<Foo>.Type or Foo.Type and your proposal’s Foo.Type.

Thus, when Brent says:

…he’s describing exactly how I would expect generalized existentials to behave. A reasonable implementation of generalized existentials would make Array.Type a supertype of Array<Int>.Type, for example.


The point of all of the above is that this proposal makes more sense viewed in the context of generalized existentials — given my (possibly flawed) understanding of where they are heading, anyway — and in that context it might be possible for it not to define so much new structure, but rather achieve the same functionality by relaxing existing restrictions on static. Does that make sense?


Right, so if you do want to make that function depend on T, you’ll need to change the signature. Whether that change is spelled shared func makeFooAndPrintIt() or static func makeFooAndPrintIt<T>() or something else, you’re still changing the signature to indicate that the function needs a type parameter. It’s not clear to me that there’s a substantive difference, that shared prevent some kind of ambiguity or breakage that other spellings allow.

This pitch specifically provides for stored properties. Are you suggesting that use case should be dropped?

I don't understand what you mean here.

Right, I understand that. I think it would take some concrete examples of use cases to convince me that the additional complexity of this approach is warranted.

I can see this. However, it also means that we would need to support Dictionary<_, String> and Dictionary<String, _> in addition to Dictionary<_, _>. I believe the syntax should follow from the model we decide to adopt.

The logical conclusion of this model is the fully generalized generic type existentials @Paul_Cantrell is interested in. I haven't heard of any problems people are facing that requires a tool like this to solve. Is this approach pragmatically useful of just academically interesting? I can't tell at the moment.

Some additional questions I have about this approach are:

  • Does this approach stored properties? How?
  • Is it possible to access the shared type of a type argument (i.e. all generic arguments are _) from an unconstrained generic context? How?
  • Is it possible to access the shared metatype instance (i.e. Foo<_>.self) of a type argument from an unconstrained generic context? How?
  • If the answer to the above is affirmative, do non-generic types have a recursive reference to themselves using the same syntax?
  • Is it possible to conform the shared type to protocols independently of the primary type? How?

I consider the above to be essential capabilities of the pitch that haven't been explicitly matched by this alternative yet. If you believe these are important capabilities I hope someone supporting this design can demonstrate how they would work under the existential / _ approach. If you believe these are not important capabilities or are opposed to introducing them please explain why.

I'm interested in exploring this direction further but am somewhat skeptical the benefits would justify the added complexity. Also importantly, I don't believe it serves the motivating use cases I described nearly as well. Perhaps there are other motivating use cases that would change the overall picture but they haven't been described yet.

1 Like

What I was referring to is prior discussions on SE, @Austin's proposal draft Enhanced Existential Support from a couple years ago, and the section in the generics manifesto I linked above. As I mentioned above, it is possible to take a broader view of existentials than this. However, this has not been the case in the Swift community until now as far as I know. That is why I said it would be an independent feature from what has usually been referred to as "generalized existentials" (perhaps more accurately, generalized protocol and superclass existentials).

I have followed the discussion of generalized existentials very closely and this is the first I've heard of anything like that being discussed in the SE community. If this was discussed previously and I missed it somehow I would love to have a link!

Is your Any<Array> syntax intended to have the same meaning as the Array<_> others have used above?

Can you find links to the threads you have in mind here?

This makes perfect sense given the understanding you have of where generalized existentials is heading. If you are correct I must have missed something somewhere along the way and would love to catch up. It would definitely influence my thoughts on the best path here.

The problem here is that static func makeFooAndPrintIt<T>() does not mean "uses T defined by the type. Instead, it introduces a shadowing type parameter local to the function:

struct F<T> {
    static func foo<T>(_ t: T) {
        print(type(of: t))
    }
}
func callF() {
    F<Int>.foo("hello")
}

Others have suggested using T: _ syntax up thread to make this distinction. That syntax is clever, but seems far less accessible to novices.

Further, it relies on an orthogonal language feature that does not exist yet for individual member declarations: static func bar() where T: Equatable produces "'where' clause cannot be attached to a non-generic declaration". This means that all members declared using T: _ must be declared in an extension without an additional language enhancement. That's not the worst thing, but it is an unfortunate consequence of going in that direction (at least for now).

Do you have something in mind? I don't believe this can be solved without some kind of new syntax.

1 Like

It seems that there are two problems here you're discussing. First, there's no way to create any members in a generic type that can be accessed without specifying all the generic arguments (stored properties or otherwise). Second, there's no way to create stored properties in generic types, even when we're content to access them while specifying all generic arguments. These seem orthogonal concerns.

By itself, shared would solve the first issue. However, by itself, shared would not address the second issue. Fundamentally, the following example would have to be made to work, and then shared would in turn enable its being defined in a way that can be accessed without specifying T:

struct Foo<T> {
    static let message = "hello"
}

The syntax I propose (extension Foo<_>, essentially, once we're allowed to write extension Array<Int>) would similarly address the first issue but not the second. It is true that it would not also address the combination of the two issues, but that would be included under enabling same-module stored properties in extensions, an idea which has significant traction.

You critiqued (rightly) that merely allowing users to write Foo<_> at the call site isn't enough, because members could use the generic argument internally without its being apparent in the signature. I'm saying that, in my proposed design as in yours, we have a way of indicating explicitly to the compiler at the point where members are defined that they make no use of the generic arguments.

1 Like

I don't think this necessarily follows, at least for the initial implementation. Dictionary<_, String> and similar could be rejected until they are (possibly) implemented in future. I think a particularly appealing part of a syntax like this, besides avoiding a new keyword, is the capability for future extensions like these.

Stored properties in same-file (or same-module?) extensions are a commonly requested feature already, independent of this feature. So perhaps this should be solved more generally.

I think namespacing is the most compelling use case from the initial pitch, so I would like to see more ideas generally. Factory methods have been pitched a few times as independently useful, and could possibly just get special treatment in generic types, and you say that using this to encode higher-kinded types is just a workaround pending proper language support.

1 Like

I disagree. I'm speculating here, but I don't believe there is currently a place in the runtime to store static properties of generic types. Doing so is much more complicated when generic arguments are relevant to storage than when they are not (i.e. when Foo<Int> requires separate storage from Foo<String>. If this speculation is correct, then providing a place to store properties is inherent to the problem and is dramatically simplified by introducing the shared type.

If anyone is able to provide facts that concur with or invalidate the above speculation please do so.

This is simply not true. The design I have pitched here absolutely does address the second issue. If you are discussing a modified version of the design I have described please describe exactly what that design looks like.

This is not a-priori true. This is your opinion about how you believe things should proceed.

I see. This design doesn't include a new keyword but it does include new syntax. To be perfectly precise, my design doesn't include a new keyword either, but instead a new declaration modifier.

As I mentioned in my previous reply, I believe syntax should follow from the model. I don't think using a declaration modifier is problematic if the model I have pitched is the right one. On the other hand, if the existential-based model is that right one then clearly something similar to the _ syntax is more appropriate.

Much more important, however, are the semantic questions. You discussed stored properties in depth but didn't address any of the other questions I asked in my previous reply. I am interested in what you have to say about those as well:

Stored properties in extensions is not the issue. Stored static properties in generic types is the issue. It sounds like you are taking the same position as @xwu. If so, please see my replies to him and comment on the issues I discuss there.

Can you elaborate on this and perhaps provide examples of the kind of ideas you mean by "more ideas generally"?

Only that I didn't find the use cases besides namespaces that compelling, for the reasons I mentioned, and I would be interested in seeing more practical use cases in general.

Currently there is no static stored properties allowed in generic types. But if all the generic parameters are unbound you're no longer in a generic context, so they could be allowed as they are elsewhere.

I don't understand the first two questions. Can you clarify by showing what you mean using the shared syntax?

I think the logical way to allow that would be:

extension Foo: SomeProtocol where T: _ {
    ...
}
1 Like