Roadmap for improving the type checker

[Slava_Pestov]

@Nobody1707

I still think it's odd that there are apparently no heuristics to prefer that each literal is the annotated type in cases like this.

The old "shrink" performance hack in Swift 6.2 did something like what you're describing, actually. It was added by @xedin in 2017, and it was a big performance win at the time.

The downside is these types of heuristics require special knowledge of the behavior of math operators, so they don't generalize to other complex overloaded APIs (eg, Collection manipulation with closures, SwiftUI, etc).

@Nobody1707

Edit: Although, it did actually type check on main, so maybe you already added something equivalent.

That's because @xedin recently removed "shrink" it in favor of the more general disjunction selection optimization described in my post, which handles more complex expressions in a more general way.

[Slava_Pestov]

@Finagolfin

I don't want the correct diagnostic that the "expression is erroneous", if it takes minutes to get it. Instead, when the checking is taking forever, I want it to bail as it does now, and give me the candidates that it could not figure out the type for.

I don't think anyone is suggesting letting the type checker run for several minutes on a single expression, or that this will be the only way to get good diagnostics

@Finagolfin

This could be done by maintaining a list of each candidate type for each variable and method in the expression and then when the checker bails, point me at the variable and method where the most candidate types were tried. In this Hooper URL case, I figure it would be channel and the + operator before and after it, since the rest of that expression should be pretty easy to check.

Separating "the rest of the expression" from the "invalid part" is sort of how the old diagnostics engine worked, before its was replaced as described here.

In the old "CSDiag" design, if the solver failed to find a solution, we would then attempt type-checking different sub-expressions independently to "narrow down" the sub-expression with the problem in it. However this would often produce misleading diagnostics (if you've used a pre-Swift 5 compiler you'll remember) and ultimately the approach was discarded entirely.

@Finagolfin

For other more complicated expressions with multiple errors or ambiguities, give me a -dump-type-checker-stats flag that will give me the types-tried counts for all variables in the failing expression, so I have all the relevant data, not just the top candidates.

However it's true that presenting diagnostics in a better way than just a list would help with complex ambiguities or situations where there are multiple things that are wrong. There are UI improvements that one could imagine making where a text editor or IDE gives more information than just a flat list of errors. I think it's important to get the the basic diagnostic experience right first, though.

@Finagolfin

I understand that this Hooper example is not representative, because most devs would not make such a simple mistake.

It is somewhat representative, and fixing the Hooper example won't require anything too heroic. Similar examples already produce the right diagnostic immediately, it's just a fun pathological case that happens to fall over completely today.

[Slava_Pestov]

@kabiroberai

What if, instead of modifying the way existing numeric literals are checked, we added a new integer-only literal spelling?

This could be one way to stage in such a change without breaking source, but it would be weird if best practices then dictated that basically all literals should use an i suffix to get good type checker behavior in pathological instances.

[dfabulich (Dan Fabulich)]

I think you and Finagolfin are kinda talking past each other here.

@Slava_Pestov

@Finagolfin

But the type checker knows exactly which portion of the expression gave it trouble, ie channel and the + operator next to it, so it should tell me that:

But therein lies the rub—it actually doesn’t In fact, in the Hooper example, it doesn’t even get as far as deciding that the expression is erroneous. All it knows is that the problem is too hard.

I know that (and so does Fingolfin). But knowing which expression is "too hard" is exactly what the current error message doesn't tell us.

Today, when the type checker bails (especially in SwiftUI code), the error message just says "Uh oh, something went wrong in this giant @ViewBuilder." Rather than make this happen less frequently, I primarily want better information about what to do when it does happen.

"I was really working hard on type checking but I had to give up when I reached line 98" would be an enormous help. It would also help to know "I worked the hardest on line 150, 172, and 193."

Now, before you assume, I know that the problem might not "really" be on line 98, 150, 172, or 193, in that case. But if the checker is working overtime on line 150, I can usually at least help out by adding some more type information; maybe I'll write VerticalAlignment.top instead of just .top. A little more help from me usually gives the type checker enough room to complete its job and give me a useful diagnostic.

Just tell us where the type checker is struggling. We'll give you the type hints you need to solve the problem.

@Slava_Pestov

This could be one way to stage in such a change without breaking source, but it would be weird if best practices then dictated that basically all literals should use an i suffix to get good type checker behavior in pathological instances.

See, that's the miscommunication here. You're assuming that "reasonable time" errors will necessarily be completely opaque, and so if you give up and let us help the type checker, we'll all just have to add explicit type hints everywhere in order to avoid "reasonable time" errors, completely undermining the expressiveness of the language everywhere, forever.

But if the type checker would tell us where it's struggling, we could add i just when/where it's needed, just when the type checker tells us it needs help, not "basically all literals."

[Slava_Pestov]

@dfabulich

Today, when the type checker bails (especially in SwiftUI code), the error message just says "Uh oh, something went wrong in this giant @ViewBuilder.

Ah, yeah. With result builders and multi-statement closures, it ought be possible to at least give a better source location that points at a statement inside the closure instead of the top-level expression. That’s a good idea.

[kabiroberai]

@Slava_Pestov

it would be weird if best practices then dictated that basically all literals should use an i suffix to get good type checker behavior in pathological instances.

I agree with what @dfabulich said: in my head this feels like the same thing as annotating a closure to help the checker; doesn't necessarily have to be a preemptive thing.

Coming back to the main point though, I'm really looking forward to seeing this roadmap materialize — especially around result builders. Having conducted dozens of iOS interviews recently, many with interns and new grads, I feel like this category of errors is one of the most common hurdles for developers who are new to Swift, second only to concurrency snags.

[Slava_Pestov]

@kabiroberai

I agree with what @dfabulich said: in my head this feels like the same thing as annotating a closure to help the checker; doesn't necessarily have to be a preemptive thing.

Ok! I should also admit that out of everything in the roadmap, the mixed literal thing was probably the least well-thought through on my part, and there’s a good chance it won’t be helpful at all. It was just something I noticed the current implementation struggled with while playing around with my expression fuzzer.

[xwu (Xiaodi Wu)]

@kabiroberai

I agree with what @dfabulich said: in my head this feels like the same thing as annotating a closure to help the checker; doesn't necessarily have to be a preemptive thing.

You can already annotate any integer literal you'd like with its type using the type coercion spelling (123 as Int), which folks designing Swift way back when have said they intended to be the Swift counterpart to C-style postfix letters in literals: there's no need to wait for another syntax to do this if you need.

[Slava_Pestov]

@xwu

You can already annotate any integer literal you'd like with its type using the type coercion spelling (123 as Int),

Another alternative spelling is Int(123), because of SE-0213.

[filip-sakel]

I apologize if this was already discussed but does subtype casting for collection types (Arrays, Dictionaries, Sets) have a significant performance impact? If so, is it rare enough to deprecate and eventually remove?

Also, with result builders causing a significant portion of these issues, is it worth specifically examining them? I remember seeing that buildExpression tends to complicate type checking but I couldn’t find that in documentation. Thus, I am wondering if we could perhaps isolate sub expressions/ partial results to make the compiler able to give more targeted diagnostics even when the overall expression is too complex.

[Slava_Pestov]

@filip-sakel

I apologize if this was already discussed but does subtype casting for collection types (Arrays, Dictionaries, Sets) have a significant performance impact? If so, is it rare enough to deprecate and eventually remove?

If you want one specific conversion to pick on, I’d say the CGFloat <-> Double ones are the most difficult to type check efficiently in hindsight, because they tend to appear in expressions that also involve the heavily overloaded math operators. The ranking rules are also rather subtle so we’ve seen surprising overload choices made in such expressions as well.

Optional conversions can also be bad, because converting a type Optional<T> to Optional<U>, where T and U are type variables, can be done in two ways: you can either convert the payload (T to U), or you can bind U to Optional<T>, so we have to introduce a disjunction.

In general though we cannot hope to phase out most implicit conversions, for example being able to implicitly upcasting a subclass to a superclass, or erasing a concrete type to an existential, is pretty fundamental to the language and it would be terrible if that was spelled explicitly (even if breaking the entire world of existing code was somehow acceptable).

(And once you’re introducing conversion constraints instead of binding type variables together in most positions, the specific set of conversions you allow don’t matter too much, with a few exceptions like the CGFloat one and optionals.)

[bbrk24]

Here's a misleading diagnostic I ran into just now:

 487 |         if computedSize == .zero && widget is TimePicker {
     |                         |- error: binary operator '==' cannot be applied to operands of type 'Size' and 'Duration'

computedSize's type is a custom struct called Size. I thought it was SIMD2<Int> and so I was using this as a shorthand to check if its width and height are both zero. For reasons beyond my comprehension, the compiler decided that obviously I meant Duration.zero on the RHS of the equality operator.

Where did it possibly pull Duration from?

[Slava_Pestov]

@bbrk24

For reasons beyond my comprehension, the compiler decided that obviously I meant Duration.zero on the RHS of the equality operator.

Where did it possibly pull Duration from?

Well, there is a == overload for Duration, and Duration has a member named zero

You’re absolutely right though that even considering this choice is a waste of time, this is what changes to operator lookup is all about in my post.

[bbrk24]

@Slava_Pestov

Well, there is a == overload for Duration, and Duration has a member named zero

Sure, but there's a hundred other types that satisfy this as well (e.g. all the primitive number types). I was just curious why it chose Duration specifically.

@Slava_Pestov

You’re absolutely right though that even considering this choice is a waste of time, this is what changes to operator lookup is all about in my post.

Makes sense, thanks!

[grynspan (Jonathan Grynspan)]

@Slava_Pestov

If you want one specific conversion to pick on, I’d say the CGFloat <-> Double ones are the most difficult to type check efficiently in hindsight, because they tend to appear in expressions that also involve the heavily overloaded math operators. The ranking rules are also rather subtle so we’ve seen surprising overload choices made in such expressions as well.

Rabble rabble muckrake muckrake… we should just redefine CGFloat as a typealias of Double. There are no 32-bit Apple platforms still supported by the Swift compiler (right…?) and we are not beholden to ABI elsewhere.

[bbrk24]

@grynspan

There are no 32-bit Apple platforms still supported by the Swift compiler (right…?)

Apple Watch, I think. Well it's "arm64_32" because it has 64-bit registers and 32-bit pointers...

[grynspan (Jonathan Grynspan)]

Right, I guess the userland CoreGraphics API there will still present it as a 32-bit float. Blergh. Either way, if the type checker struggles with it, perhaps treat it as a typealias for the purposes of typechecking.

[xwu (Xiaodi Wu)]

@bbrk24

Sure, but there's a hundred other types that satisfy this as well (e.g. all the primitive number types). I was just curious why it chose Duration specifically.

There is a note that goes with the diagnostic, and it says:

Overloads for '==' exist with these partially matching parameter lists: (Duration, Duration), (Int, Int), (Int128, Int128), (Int16, Int16), (Int32, Int32), (Int64, Int64), (Int8, Int8), (UInt, UInt), (UInt128, UInt128), (UInt16, UInt16), (UInt32, UInt32), (UInt64, UInt64), (UInt8, UInt8)

So the list is sorted alphabetically and Duration is the first one.

[Slava_Pestov]

@grynspan

Rabble rabble muckrake muckrake… we should just redefine CGFloat as a typealias of Double. There are no 32-bit Apple platforms still supported by the Swift compiler (right…?) and we are not beholden to ABI elsewhere.

That would be ABI breaking for many reasons, because it will change the mangled names of functions that take or receive CGFloat, for example.

[x-sheep (Lennard Sprong)]

Isn't CGFloat part of Foundation on non-Apple platforms as well?