Swift's unpredictable efficiency

Nonetheless, we can still write regression tests for these cases in the optimizer's own test suite, to ensure that certain abstractions get broken down by the optimizer.

3 Likes

Ah, OK. Can the optimizer's test suite be found anywhere under GitHub - apple/swift: The Swift Programming Language ? (I tried but couldn't find it.)


Is there some strategy for dealing with cases like these, except trusting that enough people will find the urge to perform enough experiments like this and file enough bugs?


Assuming that there are more people like me out there, ie non-compiler hackers yearning for increased efficiency and predictability of the optimizer, willing to spend some but not too much of their time helping to improve the situation in some way, what are some recommended ways to contribute?

(I wish there is some other way than filing individual bugs for each specific case we can find (for the most recent snapshot I guess), which would for example mean at least 6 bugs for the little demonstration in the OP.)

2 Likes

cc @Erik_Eckstein. Filing bugs for the individual issues you encounter is a good idea regardless. Ideally, we'd be able to use your examples as-is as regression tests to validate that compiler transforms continue to work on these abstractions without breakage due to standard library changes or changes elsewhere in the optimizer. Erik, would it be possible for the benchmark suite to work with ad-hoc tests like Jens' here without boilerplate, since at least some of these problems are context-dependent?

3 Likes

My suggestion would be to commit specific a-b benchmarks and then file an SR that states that the performance should be the same. Then we can validate when it is fixed and then maintain that performance over time.

2 Likes

Let me try to tackle this as a higher-level point. There are optimizations that are inherently sensitive to the exact code sequence in the program, potentially including contextual information. These will always come across as "unpredictable" in the sense that most users won't intuitively understand why their program got slower; I don't think that's a good reason to not pursue them. Instead, I think we should empower users to take more control over the performance of their program, in the following ways:

  • We should try to ensure that the optimizer isn't more powerful than the user: it should generally be possible to do important high-level optimizations (like minimizing ARC operations or specializing a generic algorithm) in source code rather than relying on the optimizer to independently discover the opportunity.

  • We should publish "optimization remarks" so that users can see what optimizations have been applied to their code, assuming this is possible to do without overwhelming them; they can they use this in conjunction with the first point to reclaim performance that seems to be lost "unpredictably".

Part of the goal of the ownership features is to provide better tools for addressing the first point.

20 Likes

I was just gonna say the same when I read your first point xD
I agree, most of what's related to memory overhead can be optimized with ownership by the user :slight_smile:

I would in general however, try to stay away from telling the users to "optimize their code" if that would imply a less readable or less maintainable code. I'm not talking about ownership here, but rather about "hey, this struct layout hits an optimizer bug so try to go for this layout instead". When you have to change the meaning of your code to less precisely fit your algorithm, that's when things tend to go wrong from what I've seen.

3 Likes

Right, I absolutely agree that we should continue to aim for the optimizer to optimize more cases. Test cases for that are always welcome.

The sorts of annotations we're talking about for ownership will hopefully not be too onerous to adopt in natural code, though.

4 Likes

I think that would be most useful as a delta between revisions: "[piece of code] can no longer be optimized because [reason]".

1 Like

All the of the commentary here is great, and my subsequent comments are not intended to detract from it.

However, I do want to make a couple of points.

In my opinion, the compiler should be better than the developer at optimising. The set of assembly language routines I can write that run faster than compiled code has reduced to almost zero over the years to the point where the only ones left are small and dependant on contextual knowledge not captured in code. Long may that trend continue. Without me having to decorate code with annotations.

Finally, whilst these results should be addressed... my experience in writing OysterKit starting in Swift 1 through to now is that release after release performance has improved. Sometimes dramatically. These results look worry-some, but with such a small footprint any fluctuations are exaggerated. Across a complete application or framework my experience (and measurements) tell me things just get better release after release.

1 Like

I am also worried about the number of annotations being added; it is a complete anathama for an easy to learn language to require annotations. I don’t buy that annotations are part of progressive disclosure when there so many annotations.

I am also concerned that annotations are becoming baked into the ABI. For example @inlineable should not be part of the ABI; ideally it would not exist, it’s the compiler’s job.

3 Likes

Which attributes are required? I guess there's @escaping, then a few Objective-C and other interop ones (@objc, @IB*). What would the number of attributes have to do with whether they are progressively disclosed? Which algorithm should the compiler use to predict the future so it can work out which parts of one module are safe to inline into a different module?

Strictly speaking, I think these are not attributes of the swift language itself, but of an Apple specific extension thereof, right?

Inlining is the compiler's job, but it's normally impossible to inline across ABI boundaries, and modules with long-term ABI compatibility concerns may not want inlining of old implementations to happen, so it's necessary for binary frameworks to opt in to allowing inlining. As Swift's build system improves to allow for cross-module optimization as part of its normal build process, @inlinable will become irrelevant for most Swift users.

I agree with your general sentiment that relying on annotations is unreasonable. I think there's a bare minimum expected level of optimization that we still aren't reliably reaching yet, and we should work on filling those gaps before dreaming up new language features.

8 Likes

Two points re inlining:

  1. You link your apps against known versions of the library (except for Apple's system libraries - which I will come back to) and therefore old code inlining is not a problem for non-system-3rd-party libraries. The compiler can decide that a function is potentially inlineable and automatically add @inlineable (and supply the source/AST/SIL/LLVM to inline) when compiling the library.
  2. Apple supplied system libraries get changed from under you, i.e. a system software update happens. In this case I think a better solution is an on device relinking of the code including inlining. This works very well in the Java world including Android. An advantage of having control over the whole of the stack is that this is possible for Apple to do also.

That's what I meant, sorry I wasn't clear. For most user modules, @inlinable should be irrelevant in the fullness of time.

That would be an interesting thing for Apple to explore, though there are many tradeoffs. "Works well" is subjective given Android's poor memory and energy efficiency compared to iOS. Apple's platform is fairly aggressively optimized for AOT compilation, and it's easier to optimize memory usage by sharing pages from dynamic libraries and pre-linking the system libraries into the shared cache. Some of that benefit could still be preserved with on-device recompilation, but the more you inline and specialize library code per application, the more per-application code size you pay for and less systemic savings you get.

2 Likes

Even in the short term, @inlinable doesn't really help with this performance problem since it can only be applied to public declarations. We either need a compilation mode that allows cross module optimization or a new attribute that isn't tied to compiling in resilient mode.

1 Like

Isn't that what @usableFromInline is for?

See comments to @Joe_Groff also; but to address your (@jawbroken) comments specifically.

The three specific attributes you mention:

  1. @escaping: Can't see why the compiler can't do this for me; but see comments to @Joe_Groff re linking system libraries.
  2. @objc: Probably needed when you write a Swift class to be used by Obj-C. Not needed when a Obj-C class to be used by Swift. OK with this as part of progressive disclosure since you are doing something quite advanced if you are writing Swift to be used by Obj-C and not extending an Obj-C protocol or class.
  3. @IB*: Other IDEs (non-Xcode) that allow users to graphically compose windows don't require these annotations, so not strictly necessary. However I am less bothered by these because they are more Xcode than Swift. Ideally though get rid of them because the obscure code and I have found them to be a source of bugs.

You also mention progressive disclosure. I don't find it very progressive if as soon as you scratch the surface and do something like pass a closure you have to start annotating (@escaping) or as soon as you want to write a library that is as performant as an Apple one you have to start annotating (@inlineable). To me these are ordinary things for a programmer to do, not advanced concepts that justify extra language features. Though obviously this is a subjective judgement by me, others might find closures advanced for example.

2 Likes

I would definitely support making this Apple only.

Definitely think there is potential to do better than Android. I think their:

  1. Higher memory usage is due to garbage collection and keeping class files and compiled code in memory.
  2. Higher energy usage is due to doing a lot of compilation on device. I was only suggesting re-linking after an OS update.