Questions about implementing parameterized extensions

Hi all, I wanted to spawn a thread about some of the issues I'm running into while implementing parameterized extensions to hopefully get some feedback from some compiler developers, but to also give the community insight into where I am in the current implementation. I've recently picked this back up as Tuples are EHC are coming to a close, and am running into some of the same issues I was running into a while back.

One of the main issues I'm running into is that an extension's generic parameters don't show up in the type definition. This is an issue when asking to get a type's context substitution map because we walk the type and find any bound generic and apply it to the respectful generic parameter found in the generic signature. However, we can't do this for a generic parameter who doesn't show up in the type at all. My naïve solution was to manually attempt to look at all generic requirements and try to fish out the substituted type from these requirements. For example:

struct Generic<A> {}

// extension<B> Generic where A == B? {}
extension<B> Generic<B?> {
  struct Nested {
    let value: B
  }
}

// let x = Generic<Int?>.Nested(value: 128)
let x = Generic.Nested(value: 128)
print(x)

Here we have a nested type Nested whose generic signature looks something like this:
<A, B where A == B?> and when we come across a type like Generic<Int?>.Nested we have to calculate the substitution map here. Currently, we generate something like this:

A => Int?
B => B

This is incorrect though because B must be substituted to the correct type Int, but because this Int never appears in Generic<Int?>.Nested, we don't assign it to B. My solution looks through this generic signature, <A, B where A == B?> and uses a type walker over the same-type requirements looking for B as the second type. If I managed to find B, I use the same type walker, who recorded the list of actions took to find B, on the substituted type we generated for A, which was Int? and we stop at Int and assign this to B. This seems to work for many of the simple examples, however a similar situation happens at runtime.

For mangling, I've added a new node named GenericExtension who follows the same formula as extensions, however in their generic signature it tells us exactly how many generic parameter the extension introduced allowing me to easily know how many and what kind of generic parameters to create during type decoding. However, the generic parameter introduced by the extension is never bound to any type in mangling because once again, the parameter doesn't appear in the type signature. This is an issue when the runtime is attempting to instantiate metadata from a demangle node because it has the bound generic type Generic<Int?> in the example above, but the generic parameter count and what we've demangled aren't equal. I.e. in the Nested's generic context it states it has 2 generic parameters with the generic parameter introduced by the extension being a key argument, but in the mangling we only supply it with the bound generic type Generic<Int?> meaning we only have 1 generic argument.

I could do something similar where I walk the generic signature of the demangled extension node, and try to fish out a substituted type for it's generic parameters, but maybe there are other solutions? Perhaps mangle the generic argument for the extension in the mangling, something like a bound generic extension?

Sorry if this should really be communicated through the PR, or if this was hard to follow! This is sort of where I am right now with parameterized extensions. I have a lot of the simple use cases working (like functions and computed properties within these extensions), but it's making sure that everything works is my priority. Maybe there just isn't something I considered yet that is causing some of these issues, or maybe I'm thinking about this incorrectly. Any help would be appreciated!

(For reference, my recent work is here: Parameterized extensions by Azoy · Pull Request #25263 · apple/swift · GitHub)

cc: @Slava_Pestov @Joe_Groff @Douglas_Gregor (sorry for the ping!)

3 Likes

My solution looks through this generic signature, <A, B where A == B?> and uses a type walker over the same-type requirements looking for B as the second type.

This is already implemented as GenericSignature::getConcreteType().

I use the same type walker, who recorded the list of actions took to find B , on the substituted type we generated for A , which was Int? and we stop at Int and assign this to B .

We have a utility to recover type substitutions given an original type and a substituted type. It sounds like it might be useful here. Take a look at TypeBase::substituteBindingsTo().

For mangling, I've added a new node named GenericExtension who follows the same formula as extensions, however in their generic signature it tells us exactly how many generic parameter the extension introduced allowing me to easily know how many and what kind of generic parameters to create during type decoding. However, the generic parameter introduced by the extension is never bound to any type in mangling because once again, the parameter doesn't appear in the type signature.

Can the mangling be in terms of the real generic signature of the nested type, and not the generic signature of the parent type? This way the mangling will at least match the runtime representation of the generic requirements in the nested type's metadata.

However, now printing the name of the nested type in a sensible way is still going to require some magic, but it's okay if that doesn't work in the first iteration.

Also, don't we have a similar problem with nested types inside constrained extensions today, with the runtime demangler having to recover additional conformances beyond those from the parent type itself?

You could also punt on this issue and say that nested types are not yet supported inside parametrized extensions, but I know that's an unsatisfying answer.

Also, don't forget to test type aliases as well.

1 Like

Right, but this returns the whole type like Optional<B> instead of B. I'd have to walk the type anyway to see if it contained B because I can only query for hasTypeParameter.

Ah yes this is exactly what I was looking for! Because of this I ended up removing the type walker altogether and instead use Type::findIf() to make sure the second type contained the missing substitution we're looking for, and then just using this to find it.

1 Like

@Alejandro, I'm super-happy to see this being worked on! I am still in the process of learning about the implementation issues you're facing, so I'm afraid this posting may only seem marginally on-topic…

Is there some reason parameterized constraints are being limited to extensions? It seems to me that one should be able to create the same kinds of constraints on declarations as well, e.g.:

struct X<D> where<K, V> D == Dictionary<K, V> { … }

While I'm all for making incremental progress, I'm a little concerned about creating inconsistencies in the generics system, and also that we might be making choices that don't generalize well to the whole scope of parameterized constraints.

Just out of curiously. Would your X<D> type enforce that the generic type parameter will always be any dictionary? That would be a great constraint feature!