Source break between Swift 6.2.4 and 6.3

Development Compiler
1

there is a source break in the Swift 6.3 release compiler, with respect to Swift 6.2.4.

in all versions of Swift since Swift 5, an operator overload on a type T with signature
(Self, Self) -> Whatever
would always be preferred over an operator with signature
(Self, any T) -> Whatever.

that meant that a program like this

extension Error where Self: Equatable {
    private func equals(_ other: any Error) -> Bool {
        (other as? Self).map { $0 == self } ?? false
    }
}
extension Error {
    static func == (lhs: Self, rhs: any Error) -> Bool {
        if let lhs: any Error & Equatable = lhs as? any Error & Equatable {
            return lhs.equals(rhs)
        } else {
            return false
        }
    }
}

struct BarbieError: Equatable, Error {}

let a: any Error = BarbieError.init()
let b: any Error = BarbieError.init()

print(a == b)

was well-formed and would dispatch from (Self, any T) -> Whatever down to (Self, Self) -> Whatever.

in Swift 6.3, the overload resolution precedence is reversed, (Self, any T) -> Whatever will be preferred over (Self, Self) -> Whatever. that causes the example program above to fall into infinite recursion, and crash.

an LLM-assisted search surfaced an issue reported by @Frizlab about a month ago.

that bug was blamed on literals in default arguments, but in this example, there are no literals involved.

1 Like
2

@Slava_Pestov or @xedin, is this worth fixing or is this also better kept this way?

3

It seems reasonable to prefer overload with any Error because it’s an exact match.

4

If the constraint solver produces more than one solution, we attempt to pick the best one, which is where the "this overload better/more specific/etc than this overload" rules are ultimately encoded. The crux of the issue, however, is that the solver implements various optimizations to skip certain overload choices along the way---it has to, to get decent performance---and sometimes, these optimizations will skip a choice that would lead to a "better" solution, if attempted, so we might not consider certain potential solutions at all.

So the overload ranking rules didn't actually change in this example, but the performance optimizations changed. In fact 6.3 replaces some older optimizations with something more principled, but as you saw some quirks remain. For example, if you change your method to wrap the self reference with a call to a generic identity function, like this, you'll get the old Swift 6.2 behavior back, because it defeats an optimization and falls back to considering all overloads of == again:

func identity<T>(_ t: T) -> T { return t }

extension Error where Self: Equatable {
    private func equals(_ other: any Error) -> Bool {
        (other as? Self).map { $0 == identity(self) } ?? false
    }
}

As a stylistic point, I would personally recommend not declaring "overlapping" overloads, where more than one overload may apply to the same given set of argument types, precisely for this reason that any change in compiler behavior might lead to runtime behavioral differences. (At the very least, these code paths should be covered by tests that are run with all compiler versions you need to support!)

This goes doubly so for "retroactive" overloads. You're overloading an operator from outside your module, ==, and your new overload involves a type from outside your module, namely any Error. (I call this a "retroactive overload" because this sort of thing can cause problems similar to declaring a retroactive conformance of a type from outside your module to a protocol from outside your module.) Nothing prevents some other module from declaring the same overload, which could cause a source break either in your module or something else that imports yours.

8 Likes
5

it is not an exact match. other has been cast to Self, so in the body of the closure, both self and $0 have static type Self.

6

you are right that this is a Bad API, and you should certainly try to push people off of patterns like this whenever you see them, but it remains a reality on the ground that this is an exceedingly common pattern in many code bases, particularly in code written by beginning and intermediate-level programmers who tend to use a lot of existentials. ironically the problem is exacerbated by efforts to move such code bases off of existentials, and onto generics, which involves introducing overloads that mix older existential APIs with “cleaner” generic APIs.

“how do i compare two any Equatable” (and its parent, “why does any P not conform to P”) is such a common question that crops up when learning Swift that i have little doubt when i inherit a code base that it will contains instances of this pattern.

7

Sorry, I looked at the bottom ==, not the one in the closure.

8

I finally got a chance to take a look at what is going on here and I don't think this is intentional (the optimizer shouldn't be prioritizing existential erasure in this case), and it doesn't happen if the same overloading pattern occurs with non-operators, both (Self, Self) -> ... and (Self, any Error) -> ... overloads would have the same score for a member for example.

As Slava mentioned in his reply we did change optimizations in 6.3 and even though we tried to make the new optimizer behave as closely as possible to the old there are still situations like this that slipped through. Could you please file a GitHub issue for this?

7 Likes
9
1 Like
10

This is not optimization related. The infinite recursion can already be seen after SILGen.

The map closure in Error<>.equals directly calls Error.== which itself calls Error<>.equals:

// closure #1 in Error<>.equals(_:)
// Isolation: nonisolated
sil private [ossa] @$ss5ErrorP3nixSQRzrlE6equals33_99B4105BFEF8B89FB51B048E96307926LLySbsAA_pFSbxXEfU_ : $@convention(thin) <Self where Self : Equatable, Self : Error> (@in_guaranteed Self, @in_guaranteed Self) -> (@out Bool, @error_indirect Never) {
// %0 "$return_value"                             // user: %17
// %1 "$error"
// %2 "$0"                                        // users: %14, %4
// %3 "self"                                      // users: %11, %5
bb0(%0 : $*Bool, %1 : $*Never, %2 : $*Self, %3 : @closureCapture $*Self):
  debug_value %2, let, name "$0", argno 1, expr op_deref // id: %4
  debug_value %3, let, name "self", argno 2, expr op_deref // id: %5
  %6 = metatype $@thick Self.Type                 // user: %14
  %7 = alloc_stack $any Error                     // users: %16, %12, %10
  %8 = alloc_existential_box $any Error, $Self    // users: %10, %9
  %9 = project_existential_box $Self in %8        // user: %11
  store %8 to [init] %7                           // id: %10
  copy_addr %3 to [init] %9                       // id: %11
  %12 = load [take] %7                            // users: %15, %14
  // function_ref static Error.== infix(_:_:)
  %13 = function_ref @$ss5ErrorP3nixE2eeoiySbx_sAA_ptFZ : $@convention(method) <τ_0_0 where τ_0_0 : Error> (@in_guaranteed τ_0_0, @guaranteed any Error, @thick τ_0_0.Type) -> Bool // user: %14
  %14 = apply %13<Self>(%2, %12, %6) : $@convention(method) <τ_0_0 where τ_0_0 : Error> (@in_guaranteed τ_0_0, @guaranteed any Error, @thick τ_0_0.Type) -> Bool // user: %17
  destroy_value %12                               // id: %15
  dealloc_stack %7                                // id: %16
  store %14 to [trivial] %0                       // id: %17
  %18 = tuple ()                                  // user: %19
  return %18                                      // id: %19
} // end sil function '$ss5ErrorP3nixSQRzrlE6equals33_99B4105BFEF8B89FB51B048E96307926LLySbsAA_pFSbxXEfU_'

Whereas when wrapping in an identity function, the closure calls the witness method which does a simple struct compare:

// closure #1 in Error<>.equals(_:)
// Isolation: nonisolated
sil private [ossa] @$ss5ErrorP3nixSQRzrlE6equals33_E619977560B7C2782CE1898FF188B53DLLySbsAA_pFSbxXEfU_ : $@convention(thin) <Self where Self : Equatable, Self : Error> (@in_guaranteed Self, @in_guaranteed Self) -> (@out Bool, @error_indirect Never) {
// %0 "$return_value"                             // user: %14
// %1 "$error"
// %2 "$0"                                        // users: %11, %4
// %3 "self"                                      // users: %9, %5
bb0(%0 : $*Bool, %1 : $*Never, %2 : $*Self, %3 : @closureCapture $*Self):
  debug_value %2, let, name "$0", argno 1, expr op_deref // id: %4
  debug_value %3, let, name "self", argno 2, expr op_deref // id: %5
  %6 = metatype $@thick Self.Type                 // user: %11
  %7 = alloc_stack $Self                          // users: %13, %12, %11, %9
  // function_ref identity<A>(_:)
  %8 = function_ref @$s3nix8identityyxxlF : $@convention(thin) <τ_0_0> (@in_guaranteed τ_0_0) -> @out τ_0_0 // user: %9
  %9 = apply %8<Self>(%7, %3) : $@convention(thin) <τ_0_0> (@in_guaranteed τ_0_0) -> @out τ_0_0
  %10 = witness_method $Self, #Equatable."==" : <Self where Self : Equatable, Self : ~Copyable, Self : ~Escapable> (Self.Type) -> (borrowing Self, borrowing Self) -> Bool : $@convention(witness_method: Equatable) <τ_0_0 where τ_0_0 : Equatable, τ_0_0 : ~Copyable, τ_0_0 : ~Escapable> (@in_guaranteed τ_0_0, @in_guaranteed τ_0_0, @thick τ_0_0.Type) -> Bool // user: %11
  %11 = apply %10<Self>(%2, %7, %6) : $@convention(witness_method: Equatable) <τ_0_0 where τ_0_0 : Equatable, τ_0_0 : ~Copyable, τ_0_0 : ~Escapable> (@in_guaranteed τ_0_0, @in_guaranteed τ_0_0, @thick τ_0_0.Type) -> Bool // user: %14
  destroy_addr %7                                 // id: %12
  dealloc_stack %7                                // id: %13
  store %11 to [trivial] %0                       // id: %14
  %15 = tuple ()                                  // user: %16
  return %15                                      // id: %16
} // end sil function '$ss5ErrorP3nixSQRzrlE6equals33_E619977560B7C2782CE1898FF188B53DLLySbsAA_pFSbxXEfU_'
1 Like
11

By optimizer I think Pavel is referring to the constraint system optimizer (CSOptimizer.cpp), which handles optimizing overload selection and choices within those overloads.

1 Like