Enormous (7x) performance regression between 6-04-a and 7-11-a

i’ve been debugging abnormally slow runtimes from the Swift PNG benchmarks, and noticed an almost 7-fold slowdown between the 06-04-a snapshot and the 07-11-a snapshot (the two closest snapshots available in swiftenv).

these are the benchmark outputs from the june compiler snapshot (runtime is in the second column from the left):

$ swiftenv local DEVELOPMENT-SNAPSHOT-2020-06-04-a
$ utils/compression-benchmark rgb8-color-photographic
[3/3] Linking compression-benchmark
...
0 30.479 182693 rgb8-color-photographic
1 31.302 181957 rgb8-color-photographic
2 32.014 180382 rgb8-color-photographic
3 33.196 179513 rgb8-color-photographic
4 35.906 178860 rgb8-color-photographic
5 37.236 178675 rgb8-color-photographic
6 38.62 178226 rgb8-color-photographic
7 42.536 177858 rgb8-color-photographic
8 96.765 172951 rgb8-color-photographic
9 134.961 171440 rgb8-color-photographic

and these are the benchmark outputs from the july compiler snapshot:

$ swiftenv local DEVELOPMENT-SNAPSHOT-2020-07-11-a
$ utils/compression-benchmark rgb8-color-photographic
[3/3] Linking compression-benchmark
...
0 192.352 182693 rgb8-color-photographic
1 190.574 181957 rgb8-color-photographic
2 188.439 180382 rgb8-color-photographic
3 186.19 179513 rgb8-color-photographic
4 190.519 178860 rgb8-color-photographic
5 191.405 178675 rgb8-color-photographic
6 190.047 178226 rgb8-color-photographic
7 193.462 177858 rgb8-color-photographic
8 255.013 172951 rgb8-color-photographic
9 329.279 171440 rgb8-color-photographic

as you can see, there is an almost 7x slowdown for the lowest compression levels, which narrows to a ~3x slowdown for the highest compression levels. the compression levels 0-9 use three different compression algorithms, so it’s not just one single algorithm that was adversely affected.

as far as i can tell, the problem still exists in the most recent snapshot (08-04-a)

all benchmarks were run on this commit of Swift PNG (87c8912d61910babd4ceff0786b256a7e78371be)

4 Likes