SE-0267 — `where` clauses on contextually generic declarations

Evolution Proposal Reviews
1

The review of SE-0267 — where clauses on contextually generic declarations begins now and runs through October 31, 2019.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager (via email or direct message in the Swift forums).

What goes into a review of a proposal?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift.

When reviewing a proposal, here are some questions to consider:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Thanks,
Joe Groff
Review Manager

16 Likes
2

This is an additive change with no impact on the ABI and existing code.

While this doesn’t change the ABI, I think it will open up new surface area in the ABI, because these two declarations:

struct Box {
    func sequence() -> [Box<Wrapped.Element>] where Wrapped: Sequence { ... }
}

extension Box where Wrapped: Sequence {
    func sequence() -> [Box<Wrapped.Element>] { ... }
}

Will, AFAIK, have different name manglings. In a binary framework, switching from one to the other will be an ABI-breaking change.

We’re already coping with this issue in generic declarations—if sequence() had a generic parameter, you could use a where clause in much the same way, which would introduce an ABI break just like this—so in some sense, this ship has already sailed. Still, I don’t like opening up more code to this issue.

(We can’t correct the ABI for generic types; that would be an ABI break. And yes, there is at least one public Apple API that would break.)

On the other hand, the current half-support for these kinds of generic constraints is very irregular, and even though it’s not in scope for this specific proposal, adding this might bring us closer to supporting where Element: Equatable in protocols, which I very badly want. So I’m torn by this proposal. I’m not sure what to recommend—I just know that these are aspects of it that we should think about.

3 Likes
3

If this code were added it makes sense that it could break the ABI. My understanding is this proposal gives the ability to add code like this which shouldn't effect anything. Am I misunderstanding something?

4 
protocol P {
    // error: Instance method requirement 'foo(arg:)' cannot add constraint 'Self: Equatable' on 'Self'
    func foo() where Self: Equatable  
}

I understand why a protocol requirement isn't allowed to be conditional by constraining Self or associated types. Can you clarify whether declarations in protocol extensions will be allowed to use constraints based on Self or associated types? It seems like this should be allowed and would be very useful.

4 Likes
5

+1. Very excited about this restriction going away. It’s one of those things that can usually be worked around but I not-infrequently run up against the limitation nonetheless and would see quality of life and clarity of code improve with the proposed changes.

How much effort did you put into your review?

A quick read of the proposal.

6

Can we adjust the mangling rule so that the new syntax become equivalent to the old one (with where clause being at the extension declaration)? We're adding new syntax anyway.

7

I don't think we can, because we already allow a where clause to add requirements to outer generic parameters when the member declaration has generic parameters of its own. That mangling would have to change, too.

2 Likes
8

Is this the sort of thing that could be special-cased in some way? For example, in future versions of the compiler the mangling for one variant of the syntax is always used, regardless of what is written in the source, unless there is some annotation on the method that it should use the other version of the mangling? It seems a shame to be stuck with having a purely syntactic difference break binary compatibility. I realise that would break source compatibility for frameworks that used this when compiling with new compilers, but that seems an acceptable cost to pay.

(This is a slightly different suggestion to @Lantua's as I understand it; rather than making the new syntax equivalent to the old one in the mangling, I'm suggesting that a canonical mangling be chosen going forward when something could be expressed in multiple ways syntactically).

9

Slava's point is the real sticking point to making this ABI-compatible.

X Unconstrained extension
(or original decl)		 Constrained extension
Unconstrained method Foo.bar() (Foo where T: Hashable).bar()
Constrained method Foo.(bar() where T.Comparable) (proposed) (Foo where T: Hashable).(bar() where T: Comparable)

Canonicalizing the bottom-left cell to be the same format as the top-right isn't so bad, but the bottom-right cell already exists, and is already distinct from the top cell. We might have been able to canonicalize that too, but now it's too late, and so I'm not sure it's worth making a special case for the bottom-left case that won't apply to the bottom-right.

2 Likes
10

Obviously this is entirely an Apple policy decision, but it might not necessarily be too late. Let's say, for the sake of argument, that all forms are canonicalised to the bottom-right as of the next compiler version unless there is a method annotation to use the 5.0 mangling. If a library author producing a binary-stable library (probably Apple) compiles with the new compiler they would receive a warning that they need to specify the mangling through the method annotation to avoid an ABI break.

For anyone producing new binary-stable libraries after the new compiler version and any existing clients of the old libraries nothing would change; existing clients can assume that a library built using an older compiler version uses the non-canonical manglings.

11

To be clear, the bottom right is the problem. The canonicalization has to be to the bottom left or top right even when written as one of the other three quadrants with constraints in them.

I feel like it'd be a pretty nasty move to have released Xcode support for binary frameworks in Xcode 11 and then immediately break it. Is it worth it to make this a short-term issue rather than a long-term one, though? Maybe.

(Note that we don't have to break binary frameworks built with Xcode 11, but purveyors of such frameworks would need to update their code for Xcode 12 or whatever.)

2 Likes
12

Why is the bottom right cell the problem and not the bottom left? Are there non-cosmetic issues with mangling the bottom left like the bottom right instead of the other way around?

13

It seems to me like we could version breaking ABI rules like this in a way that we only take advantage of them if you target newer compiler or runtime versions (and only ever have). That would be another axis of complexity to have to reason about, though.

A more straightforward approach to these sorts of ABI fixes might be to just implement them behind an ABI version flag. That way other platforms that don't have ABI constraints can benefit right away, as can new platforms that might eventually be introduced that can take an ABI break.

2 Likes
14

The whole bottom row represents a second way to write things already expressible by the top-right cell. We didn't bother to do that for the bottom-right row, so we already have this problem today:

extension Foo where T: Hashable {
  func bar<U>(_: U) where T: Comparable {}
}

extension Foo where T: Hashable, T: Comparable {
  func bar<U>(_: U) {} // could be equivalent to above, but isn't
}

SE-0267 exposes the problem in a lot more places, and while there's an obvious way to canonicalize the newly-allowed declarations to an existing syntax (bottom-left cell -> top-right cell*), we have both the top-right and bottom-right cells in existence today. So the options would be:

  1. Moving a constraint between the extension and the member is always breaking (as proposed).
  2. Moving a constraint between the extension and the member is okay if the member is not itself generic (still ABI compatible but a much more subtle rule).
  3. Moving a constraint between the extension and the member is always okay (ABI-breaking for instances of the bottom-right cell without extra annotations, in order to canonicalize them to the top cell).

* We can't canonicalize to the bottom-right cell because it's not obvious which constraints go on the extension and which go on the function. If it's "all of them that can go on the extension", that's the top-right cell. We could in theory canonicalize to the bottom-left cell, but that'd be extra breaking since it's the one cell that isn't possible today, and it loses some nice properties for partial names.

6 Likes
15

I suppose, based on our experience with opaque result types, that any new mangling will make this feature not back-deployable by default.

16

The proposal does not mention a requirement for newer runtime, which means this should still be back-deployable.

17

Only new type manglings need new runtime support, though the Swift 5.1 has additional fallbacks that can allow for the compiler to avoid needing to emit new manglings when targeting older OSes. Our existing mangling scheme ought to be able to cover naming symbols using this feature.

18

This is what Slava originally said on this matter:

Though I don't see how the lexical structure breaks when we exchange constraints between the extension and the member.

Does it matter at all to the demangler where the constraints are placed?

Yes, that will be allowed too since you can already constrain the extension instead.

19

I think you misunderstand, so let me be more clear with a code example.

I want to be able to write this:

extension Sequence {
    func toSet() -> Set<Element> where Element: Hashable {
        return Set(self)
    }
}

Instead of this:

extension Sequence where Element: Hashable {
    func toSet() -> Set<Element> {
        return Set(self)
    }
}

It isn't clear from the proposal whether that is possible. It is clear that you cannot use constraints on protocol requirements. What about symbols declared in extensions?

There is a good reason to disallow this in the context of protocol requirements. It would be confusing to have a requirement with such constraints. What would that even mean in a case where Self or an associated type does not meet the constraints?

On the other hand, the meaning is quite clear in the code I have written above. The only reason I can imagine for disallowing the former is implementation difficulty. Ideally if we're going to lift this limitation we would do it everywhere it is sensible to do so.

20

I did mention it in a generalized form in the introduction note, but apparently it should be made more eye-catching:

Only declarations that already support a generic parameter list and being constrained via a conditional extension fall under this enhancement.

Unless the constraint is on a protocol requirement (we haven't modelled those yet), there is no misunderstanding :slight_smile:.

P.S There are some tests for protocol extensions too.
https://github.com/apple/swift/pull/23489/files#diff-f058e6035325de0d3eeeb9cf08946e69