Regarding Swift type inference compile-time performance

square_peg · 2021-06-18T19:54:15.187Z

Hey all!
This came up during a discussion today and I honestly wasn't sure if my assumptions were correct so looking for some clarity here!

What are the compile-time performance differences between the four declarations below?

(I have added how I think it works (based on reading the Type Inference document) in comments along with my doubts.)

let a = "hello, world!" // type is inferred
let b = String("hello, world!") // type is inferred from String(...) and then passed to the root (the constant b)
let c: String = .init("hello, world!") // type inference is not required
let d: String = "hello, world!" // type inference is not required

Assumptions: type inference adds a small compile-time performance penalty

Note: I searched this forum and few other websites for related discussions, including the Swift reference, but couldn't find anything conclusive

Edit 0: Added "compile-time" to the word "performance" to avoid any confusion

Ray_Fix · 2021-06-18T20:02:23.919Z

Hi! There is no runtime penalty, AFAIK. There might be some minuscule compile-time difference but I wouldn't worry about it, honestly. So then it just comes down to a question of style. The style guide that I use prefers a.

typesanitizer · 2021-06-18T20:06:53.321Z

We can measure it.

I've multiplied it by a 1000 to get a better signal.

Benchmark #1: xcrun swiftc -typecheck a.swift
  Time (mean ± σ):     175.7 ms ±   3.5 ms    [User: 82.9 ms, System: 81.9 ms]
  Range (min … max):   171.0 ms … 182.8 ms    16 runs

Benchmark #1: xcrun swiftc -typecheck b.swift
  Time (mean ± σ):     224.8 ms ±   2.8 ms    [User: 131.1 ms, System: 81.7 ms]
  Range (min … max):   220.2 ms … 228.2 ms    13 runs

Benchmark #1: xcrun swiftc -typecheck c.swift
  Time (mean ± σ):     672.3 ms ±   8.0 ms    [User: 568.3 ms, System: 93.7 ms]
  Range (min … max):   662.4 ms … 685.1 ms    10 runs

Benchmark #1: xcrun swiftc -typecheck d.swift
  Time (mean ± σ):     213.3 ms ±   2.0 ms    [User: 119.8 ms, System: 81.6 ms]
  Range (min … max):   210.2 ms … 216.5 ms    13 runs

Here's the test script using the hyperfine utility.

#!/usr/bin/env python3
import os

filenames = ["a", "b", "c", "d"]
code = [
  'let a{} = "hello, world!"',
  'let b{} = String("hello, world!")',
  'let c{}: String = .init("hello, world!")',
  'let d{}: String = "hello, world!"'
]

for (i, filename) in enumerate(filenames):
    with open(filename + ".swift", "w") as f:
        s = ""
        for j in range(1000):
            s += (code[i] + '\n').format(j)
        f.write(s)
    os.system("hyperfine 'xcrun swiftc -typecheck {}'".format(filename + ".swift"))

Jon_Shier · 2021-06-18T20:13:53.373Z

Odd, I would've expected b and c to have the same performance. Is : String = .init not equivalent to String.init?

Jens · 2021-06-18T20:35:22.307Z

This is in line with my experience. I removed all .init(…)s from a project that had lots of them because they took up so much of the total compile time.

Jon_Shier · 2021-06-18T20:38:15.292Z

I also would've expected d to be a bit faster than a since an explicit type is provided, which should short circuit the literal inference.

Jon_Shier · 2021-06-18T20:46:02.100Z

There also seems to be an across the board performance regression in Swift 5.5. On my machine (2020 iMac, 10-core i9), Xcode 12.5:

Benchmark #1: xcrun swiftc -typecheck a.swift
  Time (mean ± σ):      58.9 ms ±   0.8 ms    [User: 43.3 ms, System: 14.2 ms]
  Range (min … max):    57.3 ms …  61.1 ms    48 runs
 
Benchmark #1: xcrun swiftc -typecheck b.swift
  Time (mean ± σ):     103.6 ms ±   1.3 ms    [User: 87.0 ms, System: 15.3 ms]
  Range (min … max):   100.9 ms … 106.4 ms    28 runs
 
Benchmark #1: xcrun swiftc -typecheck c.swift
  Time (mean ± σ):     456.2 ms ±   6.1 ms    [User: 426.8 ms, System: 27.8 ms]
  Range (min … max):   447.7 ms … 469.7 ms    10 runs
 
Benchmark #1: xcrun swiftc -typecheck d.swift
  Time (mean ± σ):      95.1 ms ±   1.3 ms    [User: 77.7 ms, System: 16.0 ms]
  Range (min … max):    92.9 ms …  98.1 ms    30 runs

Xcode 13b1:

Benchmark #1: xcrun swiftc -typecheck a.swift
  Time (mean ± σ):     110.4 ms ±   0.9 ms    [User: 68.8 ms, System: 39.4 ms]
  Range (min … max):   108.8 ms … 112.7 ms    26 runs
 
Benchmark #1: xcrun swiftc -typecheck b.swift
  Time (mean ± σ):     156.6 ms ±   2.1 ms    [User: 113.6 ms, System: 40.7 ms]
  Range (min … max):   153.1 ms … 160.2 ms    18 runs
 
Benchmark #1: xcrun swiftc -typecheck c.swift
  Time (mean ± σ):     592.0 ms ±   5.4 ms    [User: 531.5 ms, System: 56.0 ms]
  Range (min … max):   585.3 ms … 604.4 ms    10 runs
 
Benchmark #1: xcrun swiftc -typecheck d.swift
  Time (mean ± σ):     146.6 ms ±   1.7 ms    [User: 103.1 ms, System: 41.2 ms]
  Range (min … max):   143.0 ms … 149.3 ms    20 runs

I did add a --warmup 1 to the hyperfine command as well.

typesanitizer · 2021-06-18T20:51:26.369Z

cc @xedin should this be fixed by [5.5][CSBindings] Account for literal bindings when checking "subtype of existential" property by xedin · Pull Request #37672 · apple/swift · GitHub or Fix some performance regressions [5.5] by slavapestov · Pull Request #37801 · apple/swift · GitHub ?

xedin · 2021-06-18T21:04:03.279Z

There where a couple of perf fixes that went into 5.5 branch recently. Could you try with the most recent snapshot?

Jens · 2021-06-18T21:11:06.657Z

I added this to @typesanitizer's test:

  'let e{} = String.init("hello, world!")'

and it is as slow as c:

  'let c{}: String = .init("hello, world!")',

How come e and c are so much slower to type check than b:

  'let b{} = String("hello, world!")',

?

I thought there was no difference at all between String("…") and String.init("…").

Jon_Shier · 2021-06-18T21:31:16.514Z

Actually, the 6/14 toolchain seems worse:

Benchmark #1: /Library/Developer/Toolchains/swift-5.5-DEVELOPMENT-SNAPSHOT-2021-06-14-a.xctoolchain/usr/bin/swiftc -typecheck a.swift
  Time (mean ± σ):     126.6 ms ±   2.4 ms    [User: 89.5 ms, System: 35.3 ms]
  Range (min … max):   123.7 ms … 133.6 ms    23 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-5.5-DEVELOPMENT-SNAPSHOT-2021-06-14-a.xctoolchain/usr/bin/swiftc -typecheck b.swift
  Time (mean ± σ):     178.8 ms ±   1.6 ms    [User: 141.2 ms, System: 35.7 ms]
  Range (min … max):   175.7 ms … 181.4 ms    16 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-5.5-DEVELOPMENT-SNAPSHOT-2021-06-14-a.xctoolchain/usr/bin/swiftc -typecheck c.swift
  Time (mean ± σ):     721.0 ms ±   3.3 ms    [User: 666.9 ms, System: 52.1 ms]
  Range (min … max):   716.2 ms … 726.5 ms    10 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-5.5-DEVELOPMENT-SNAPSHOT-2021-06-14-a.xctoolchain/usr/bin/swiftc -typecheck d.swift
  Time (mean ± σ):     167.0 ms ±   1.7 ms    [User: 129.1 ms, System: 35.9 ms]
  Range (min … max):   163.2 ms … 169.6 ms    18 runs

Jens · 2021-06-18T21:34:14.842Z

Are these toolchains compiled differently than the GM / release ones (maybe not as optimized)?

Jon_Shier · 2021-06-18T21:36:13.716Z

main toolchain also isn't any better:

Benchmark #1: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a.xctoolchain/usr/bin/swiftc -typecheck a.swift
  Time (mean ± σ):     124.1 ms ±   0.9 ms    [User: 89.0 ms, System: 33.2 ms]
  Range (min … max):   122.9 ms … 126.0 ms    23 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a.xctoolchain/usr/bin/swiftc -typecheck b.swift
  Time (mean ± σ):     178.1 ms ±   1.7 ms    [User: 141.7 ms, System: 34.4 ms]
  Range (min … max):   175.9 ms … 182.8 ms    16 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a.xctoolchain/usr/bin/swiftc -typecheck c.swift
  Time (mean ± σ):     717.6 ms ±   5.5 ms    [User: 665.0 ms, System: 50.4 ms]
  Range (min … max):   710.3 ms … 726.7 ms    10 runs
 
Benchmark #1: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2021-06-12-a.xctoolchain/usr/bin/swiftc -typecheck d.swift
  Time (mean ± σ):     166.1 ms ±   1.2 ms    [User: 129.4 ms, System: 34.7 ms]
  Range (min … max):   164.2 ms … 168.5 ms    18 runs

Jon_Shier · 2021-06-18T21:38:25.126Z

Jens:

Are these toolchains compiled differently than the GM / release ones (maybe not as optimized)?

I think the big difference would be building with assertions, which I'm pretty sure are disabled for the toolchain builds. But Apple builds Xcode's default toolchain internally, so I'm not sure if there's any difference there.

cukr · 2021-06-18T21:44:28.206Z

Jens:

I thought there was no difference at all between String("…") and String.init("…") .

In the past there was no difference, but today (after SE-0213) String("…") is equivalent to "..." as String which creates String directly from the literal.

The String.init("…") first makes a String from a literal, and then calls copy init on it.

godbolt.org

Compiler Explorer - Swift

struct Test: ExpressibleByStringLiteral { init(stringLiteral: String) { print("literal constructor") } init(_: Test) { print("copy constructor") } } let a = Test("...") print("---") let b = Test.init("...")

xedin · 2021-06-18T21:46:28.450Z

There toolchains have assertions enabled so that might contribute, if possible it would be interesting what happens in —no-assertions build.

xwu · 2021-06-18T23:45:26.314Z

cukr:

The String.init("…") first makes a String from a literal, and then calls copy init on it.

Correct, “c” and “e” actually have different user-facing semantics from the remaining examples. They differ in more than just code style from the rest, but in fact have different behavior.

The other cases differ by the presence/absence or location of the type annotation, and all perform rather similarly.

By contrast, “c” and “e” (that is, writing out .init with a type that is expressible by a literal) explicitly tells the compiler that you want first to create an instance of, if possible, the default literal type and then to convert it via an unlabeled initializer to the type you specify.

Looking through all unlabeled initializers defined on String and figuring out which is the best candidate among those that can take a single argument of a type that is expressible by a string literal is a chunk more work. SE-0213 explicitly preserves the .init spelling as the escape hatch to request this behavior, which used to apply to the String("…") spelling as well that is now special-cased to be a synonym of "…" as String.

Jon_Shier · 2021-06-19T00:13:56.121Z

It's definitely issue. Weren't there assertion disabled toolchains available early on once toolchains started becoming available? I seem to remember there being two downloads. In any case, I can do a build and test it out if you can give me the command.

xedin · 2021-06-19T00:19:58.945Z

Use ./utils/build-script -R --no-assertions --skip-build-benchmarks true it would build a release compiler without assertions.

xedin · 2021-06-19T00:23:15.762Z

Although it's understandable while there is a performance difference between different expressions as others already mentioned it's concerning to me that there is an overall difference between releases for such simple expressions, I suspect it's property wrappers which contributed here so PR 37801 might have improved that.