Generally speaking, I wouldn't expect to be able to switch between the two extension signatures above without breaking ABI. While it might be possible to allow that now with some clever compatibility hacks, it seems like it would introduce a lot of complexity. I also think it's not a coincidence this has come up in two recent proposals, and I think there's a decent chance that if we tried to maintain compatibility, some future extension/generics feature would eventually force a break.
From a user perspective, I also don't think this is a huge problem. If I write extension Array in a framework I'm distributing, there's no reason for me to ever rewrite it as extension<T> Array<T> because it's entirely equivalent (there's no expressiveness gain). In the unlikely event the extension does get refactored or rewritten in an incompatible way, tooling like the api digester should be able to warn about these issues.
It's possible there I'm failing to consider some scenarios, but I'd be in favor of allowing rewrites like the one above on a 'best effort basis', while discouraging the authors of resilient frameworks from doing so and counting on tooling to prevent accidental breakage.
Any progress on this feature? It‘s one of the most wanted generics feature for me. I have too many code that needs to be deleted after this becomes reality.
There's been a lot of changes made that have probably invalidated what I have so far, but now that things seem to have settled down at least a little bit I plan to start pushing this again once tuples (hopefully) have EHC. I can't provide any specific timeline as to when I'll have solved all the current issues, but this is next on my radar Also, now having some experience with the runtime machinery, I'm hoping that I'll have a somewhat of an easier time understanding what needs to be done for the runtime component.
You don‘t have to tackle this completely on your own. There are a lot of people that would probably want to push this through. We just need to keep this discussion alive and active.
Yeah, I don't see an inherent reason why the unconstrained, one-to-one mapping case would need a different mangling (generic parameters do not have their names in the mangling, only their order), so I think it's worth special-casing that one.
Adding that sort of capability should only affect compile-time lookup; the runtime representation of the type would still be the same as it always is. (This is important for binary compatibility.)
Even if it did affect mangling, I'd expect us (the Swift community as proposal reviewers) to decide that member lookup still wouldn't find generic parameters used to define extensions. You can define multiple extensions with the same signature, which means you could get many different aliases all available on a type that all resolve to the same underlying type.
I expect this problem to be worse too because the generic parameters for a type are relevant outside the type (specifically, for constrained extensions in today's syntax), but the generic parameters for an extension don't feel like they're available outside the extension, at least not to me.
Well these things are hard to understand, at least for me as I'm not nearly experienced in these field like you guys who work(ed) on the compiler. I didn't meant that qualified lookup, as I understand it, would allow new kind of forms such as [Int?].Wrapped.self. To me that's illegal. What I was rather asking if we had qualified lookup was, if struct G<Element> {} would allow us to use write G<Int>.Element, which would then resolve to Int in this particular case, if it would change anything in the mangling.
I've seen many areas where we actually want the generic parameter name to be inherited, but it's currently not possible due to this missing feature. So we have to either fallback to a protocol with an associated type or an alternative type alias, which won't be matching the generic parameter name.
Yeah, I was thinking about the combination of this pitch and your member lookup pitch. To use the dummy type:
struct G<Element> {}
extension <InnerElement: Hashable>
G<Set<InnerElement>> {}
print(G<Int>.Element.self) // okay
print(G<Set<Int>>.Element.self) // also okay
print(G<Set<Int>>.InnerElement.self) // should this be okay?
print(G<Int>.InnerElement.self) // error for sure
One could argue™ that the generic parameters on the extension should appear on the type it's extending because it'd be useful in some cases. And it would! But in cases where you don't expect that to happen, you've now added a weird extra member to the base type, and therefore I'd suggest that the member-lookup-finds-generic-parameters feature only apply to the original type and not extensions.
I think the way about it. As you mentioned it the 3rd (& 4th) example is "weird" and I wouldn't expect that. My pitch was about generic type parameters on types only. Honestly I have no idea if this could/should be extended to generic functions one day. I mention it because to me personally the generic type parameter name on functions can be whatever, as we always ignore it, except for the readability of the APIs we write. That's how I think about the generic type parameters on parameterized extensions. That brings me back to why I also view the 3rd (& 4th) example as illegal.
I'm back with some more design questions that I believe needs to be hashed out.
I discussed earlier about the one-to-one mapping of extension parameters and the extended type and many have expressed that they expect those to canonicalize to the same thing. I agree. Generalizing this also means that one can swap between conditional conformance syntaxes like we've discussed way up this thread. However, although the one-to-one mapping would be a special case, how does this extend to other potential use cases?
// ok, children in this extension are treated as children like
// extension Array {}
extension<T> Array<T> {}
struct Foo<A, B> {}
// ok, this makes sense to also treat this extension as
// extension Foo {}
extension<T, U> Foo<T, U> {}
// this can rewritten to something more readable as
// extension Foo where B == A? {}
// the question is, should we rewrite the following extension
// as the more simpler one to allow users to swap between the two
// to preserve ABI compatibility?
extension<U> Foo<U, U?> {}
// ok, this is a one-to-one mapping, but the extension's parameter
// adds a requirement that is being imposed on Element.
// we could take these requirements and simply apply them to
// the parameter that's being one-to-oned.
extension<T: Equatable> Array<T>: Equatable {}
// this is an example of where we can't perform any sort of
// rewrite because we're making use of the introduced parameter
// in the signature without any same-type to relate T to.
// consider the third example in this code block, we were able to
// rewrite that extension because the first parameter had a one-to-one
// with A == U, so we could naturally express B in terms of A.
// this is not the case with the following because there is no other
// parameter to express Element in.
extension<T> Array<T?> {}
Obviously there are hundreds of examples I could give to demonstrate the fact that we (the compiler) can rewrite some parameterized extensions to be compatible with extensions we can already write today. In my opinion, I believe that it should perform this rewrite to as many extensions as it can (obviously it won't where it can't). I know @owenv expressed that swapping between compatible extension signatures shouldn't be compatible with each other, but others like @griotspeak and @jrose expressed that the one-to-one should be special and do expect those to be compatible. My question is, to what extent (assuming we're ok with some extensions being compatible with each other), should they be compatible? Should we aggressively rewrite all compatible parameterized extensions? Should we only do a one-to-one when there's a single generic parameter? Would love if others could chime in and express their opinion on this as well.
On the topic of these generic parameters being introduced into the extended type, I agree with others that these generic parameters should be private to the extension.
If there's anything here that you'd like explained more in detail, or maybe I explained something poorly, please let me know and I'll try to reword it to make it easier to understand. Please let me know your thoughts, concerns, or questions!
Whatever rewrites you/we implement, it must be fixed by the time of the next release, because that's part of ABI stability.
The completely unconstrained case is most important.
The "only adds constraints" case seems easy enough to implement.
Anything more and I'm worried we'd have trouble explaining the rules to people, which is still important even if we continue to productize the ABI checker. So, your Foo<U, U?> case would be nice if it were the same, but I'm not sure it's obvious how that example differs from the Array<T?> case. So whatever stopping place you/we pick, we should be able to describe it well. (But read on.)
So let's try to describe these cases:
"An extension without generic parameters may not be rewritten as an extension with generic parameters or vice versa; doing so is not ABI-compatible."
"An extension with generic parameters may only be rewritten as an extension without generic parameters or vice versa only when there are no constraints on any of the parameters, and the parameters map one-to-one to the original type's generic arguments. For example, extension <K, V> Dictionary<K, V> is equivalent to extension Dictionary, but extension <V, K> Dictionary<K, V> is not. Similarly, extension <T> Array<T> is equivalent to extension Array, but extension <T> Array<T> where T: Equatable is not equivalent to extension Array where Element: Equatable".
[After describing this, the limitation on constraints seems weird.]
"An extension written without generic parameters implicitly introduces generic parameters with the same names as the type's original generic parameters. That is, extension Array where Element: Equatable is shorthand for extension <Element> Array<Element> where Element: Equatable. Rewriting between these two forms is ABI-compatible."
I like this because even if it isn't implemented this way it's very easy for people to reason about, and it makes it clear that you're just changing the names of generic parameters if you want to use T instead, which will always be allowed for extensions.
"An extension with generic parameters where all parameters are used directly as generic arguments may be rewritten as an extension without generic parameters and vice versa without breaking ABI compatibility. However, if any parameters are not used as top-level generic arguments, the extension must remain parameterized."
This is what you were asking about with the Foo<U, U?> rewrite, but I think it's actually the correct general principle for when it's possible to do a rewrite. It's a bit harder to explain but on the other hand I think it's 100% correct: that is, we know how to map every such parameterized extension to a non-parameterized extension automatically (through unification, I guess), and we know that if someone writes a parameterized extension that does introduce a new parameter, it wasn't possible to express the same thing using a non-parameterized extension.
EDIT: I suppose equality constraints might make a newly-introduced parameter redundant, and therefore I think it's important that we'd still stick to the rule as written and not attempt additional unification.
Conclusion: I think #3 (very explainable) or #4 (correct and complete, but possibly harder to implement) would be good ways to go. But the most critical thing is that we can't add support for #4 later without breaking ABI.
Why is the last extension not the same? You said it's only important if there are no constraints. This extension has no constraints, it just has the order of the generic type parameters flipped. Is this also important?
Can you also elaborate a little more why these two are different?
What kind of 'difference' are we talking about in this case?
I'm more confused because with #3 you said that extension Array where Element: Equatable is the same as extension <Element> Array<Element> where Element: Equatable and then you also said "it makes it clear that you're just changing the names of generic parameters if you want to use T instead, which will always be allowed for extensions", which to me means that it's also equivalent to extension <T> Array<T> where T: Equatable.
@Alejandro I'm really happy to see this feature moving forward again. It would allow me to delete a huge chunk of duplicative code and make a lot of things simpler. ( this could land in Swift 5.3)
These are all attempts at "how we would explain a restriction to the user if we/Alejandro chose not to lift it", with the items in the second list corresponding to the items in the first list. You're right that a normal Swift developer probably won't think of the order of generic parameters as being ABI; they are, unfortunately. (Maybe we should have sorted by name and not allowed you to rename them instead, but we didn't.)
What is the plan for overlap detection? I looked at the PR and I don't see anything additional there for detecting overlap. For instance, if we have
struct S<A> { ... }
struct X { ... }
protocol PT { ... }
protocol PS { func f() }
extension<T : PT> S : PS where A == T { func f() { print("Generic") } }
extension S : PS where A == X { func f() { print("Specialized") } }
extension X : PT { ... }
let s : S<X> = ...
s.f()
What are the semantics of this? Do they depend on which modules the different declarations are in?
Do we provide an error or warning about the ambiguity?
Do we have syntax that lets the user manually pick which one they want?
This problem is already present with the existence of conditional conformances, and I am afraid by adding more ways of writing them (in this case, making equality constraints available, not just conformance constraints), we might exacerbate the issue. It seems even more of a problem with the "Future Directions" part where you allow blanket extensions over any generic type, because that can overlap with anything else (you can look at Haskell's OverlappingInstances extension, for example of the headaches this can cause). For now, at least we have one level of nesting in the extended type (e.g. S or Array or Pair) which makes it somewhat less likely.
this could land in Swift 5.3)