Performance annotations

Would it perhaps be possible to leverage a similar approach as result builders (the idea of connecting the AST to function calls) to have a higher ordered linguistic function scope definition? For example in the case of no runtime allocations - that the compile pass would see one of these attributions and then execute at compile time parts of the code as a pseudo extension to the compiler. That way we could build specializations like these for all sorts of things, not just one of paint bucket effects like no allocations, or no locks etc, but a full control of a subset of the AST.

Ruby has some functionality that does wild stuff like that; where it can even control the parser - what I am suggesting is not as widely scoped as that, but something more in the spirit of Swift's strong type system.

This might be too lofty of an idea to pursue for something so specifically targeted, but maybe it is an indication that since we can easily think of numerous useful cases that a longer term goal might be a good consideration here.

There are no nodes in the AST which represent “copy” or “move”. And the effects may need to propagate to callers (e.g. if they affect global FP mode). This makes them pretty different from result builders.

Oh for sure... it would need more than just AST and much more than just what result builders do today, but the concept was to expose the internals of the compiler's internal representation to code provided by the client - by no means is this a fully fleshed out idea. Just a random drive-by noodling.

In terms of the annotations, however, I wonder if it'd be a more natural fit to attach these as effects to functions in the same way as async and throws .

We want to do something more "lightweight" than integrating these annotations into the type system. async/throws like annotations mean that people have to annotate a lot of functions, which can be a pain. Also, it would mean that we have to annotate thousands of stdlib functions. And that would repeat every time we'd like to add a new performance annotation. That's not really practical.

Could either of these annotations ever be reasonably applied to an async function? I am thinking the answer is likely no?

There are possibilities to support this. Async functions use a stack managed by the runtime using a bump pointer allocator. For example, the user could specify the size of a pre-allocated stack space whenever a task is created, so that the bump pointer allocated will never overflow into heap space. On an overflow the program would abort with a runtime error. Though, I'm not sure if such an approach would make sense in practice.

The even more common case is that someone has a function that is trivial and can be fully slurped up with inline, but not marked as either - it is perfectly reasonable as an API author to change the implementation of something completely inlined. Normally changing that without changing the signature is legitimate and does not break ABI or API. Now with these annotations we run the risk of potentially breaking API.

Yes. But I assume it's a seldom case that a trivial function would be changed from e.g. non-allocating to allocating. We could add complementary attributes, e.g. @allocating, which can be put on inlinable functions for which the library author want to keep it open to change them in the future.

Maybe uncheckedPerformance or something could work?

Sounds good to me.

One compromise could be to introduce @performance(...) with the next Swift compiler in a way that just ignores any unknown attribute in there.

I like that idea.

3 Likes
Yes, I thought the same. And memory needed to hold all those audio samples can be quite big.

Just as an example:

48kHz x 2 bytes x 2 channels x 10 sec x 88 keys x 10 instruments ~ 1.6 GB

That's for raw audio like wav, half of that for PCM, but it should also be possible to uncompress AAC or MP3 compressed audio in real time so that memory can be 10x-20x times smaller.

This is what "mlock" description says:

getrlimit + RLIMIT_MEMLOCK returns some very large number so the actual limit is "a system-wide ``wired pages'' limit" and I wouldn't be surprised it is small. No idea how to view this limit on iOS / macOS.

1 Like

As I think more about it, I wonder how much of a problem this will be. The notion of "it’s okay that the called function allocates or locks" really depends on your perspective.

As a library author, I might want to annotate certain performance attributes to guide me during my implementation. For example, I've encountered situations where nested functions introduce invisible yet very expensive allocations, and I'd like to be informed if that should happen (including in any called functions). But it's unreasonable to add these flags to every function in the project, so I'm more likely to add them at a coarse level, covering high-level operations (such as public functions), and carve out exceptions for certain code which I expect to allocate.

But then, a consumer of my library may not be okay with the allocations that I flagged as being okay. From their perspective, I may be one of those leaf functions which they're trying to ensure doesn't allocate. And of course, they may are depending on my @noAllocations attribute being absolutely truthful:

So there are these 2 separate meanings of something like @noAllocations - internally, it means "no allocations except ones that I specifically allow", and externally, it has to mean "no allocations whatsoever".

Perhaps that was obvious, but I don't see it mentioned in the pitch, and I think it is worth calling out.

7 Likes

I thought the thorough and throughout annotation was part of the pitch.. I believe it has to be eventually. We can start with, say, a small subset of stdlib, and activate this annotation system based on explicit opt-in, perhaps even on a file by file base, like ARC. There will be pain initially but that's solvable IMHO.

Yes, it is important to have this distinction, e.g. @noAllocationsBut can have internal escape hatches, @noAllocationsEver can not (obviously names must be better).

This sounds very similar to @noImplicitCopy from the roadmap, and I actually think they should work the same way:

  • @noImplicitAllocation would cause the compiler to warn if an allocation occurs.
  • @noImplicitLock would cause the compiler to warn if a lock occurs.

Like @noImplicitCopy, I feel this should not be part of a function’s interface at all. The ability to overrule these attributes means no promises are actually made, and not having Implicit in the name would bury that reality even further.

3 Likes

As a completely distinct set of features, there could be compiler-enforced guarantees that allocations or locks do not occur. This would need to have no exceptions, including calls to other functions.

Anything more lenient than that would just be a false friend, worse than nothing. If there must be exceptions, they should be treated in the same manner as Swift.unsafeBitCast(_:to:).

I’m not convinced this would actually be useful. Maybe for an RTOS?

What about subclasses and protocol conformances? Shall it be like so:

 class Base {
    @noAllocations func foo() {}
    func bar() {}
}

class Derived: Base {
    override func foo() {} // Error
    @noLocks override func foo() {} // probably ok
    @noAllocations override func foo() {} // ok
    @noLocks override func bar() {} // ok
}

protocol Proto {
    @noAllocations func foo()
    func bar()
}

struct S: Proto {
    func foo() {} // Error
    @noLocks func foo() {} // probably ok
    @noAllocations func foo() {} // ok
    @noLocks func bar() {} // ok
}

I've seen some discussion around the unsafePerformance escape hatch but I want to ask, why do we need this? The "Escape hatch" section doesn't have rationale other than "to silence the performance diagnostics".

As a consumer of a library that exposes an API marked with @noAllocations, I would be very surprised to find allocations happening, because the compiler promised me there wouldn't be any.

If a method needs to allocate and "it's okay that the called function allocates or locks" then why mark it with @noAllocations?

4 Likes

The most common need for unsafePerformance, I would guess, is to label code that isn't or can't be annotated with the new attributes, either because it's imported from C, or from an existing Swift package that hasn't yet adopted these attributes, but which is known to no-allocations or no-locks-safe. It could also be used to selectively call into APIs that aren't no-allocations safe in full generality, but which are known to be safe in specific circumstances, or in code that is mostly expected to be no-allocations-safe but in which allocations may be allowed locally in limited circumstances.

3 Likes

Perhaps the semantics would be greatly clarified if we called it, by analogy to withoutActuallyEscaping, withoutActuallyAllocating and withoutActuallyLocking?

6 Likes

Riffing on what you are suggesting, I imagine we want it to be a catchall term like withoutActuallyViolatingPerformanceAssertions? Maybe that is too verbose, I don't know.

I also think that this is preferable. IMO, security is an important enough concern that it should have the monopoly on things called "unsafe" in the language. Locking or allocating can cause performance issues, but in the general case, taking (and later dropping) a lock or allocating don't cause security issues.

I think that withoutActually{PerformanceCharacteristic} is my preferred wording too.

3 Likes

Is the "it's okay that the called function allocates or locks" case feature requirement? If not, I'm also +1 on the withoutActually... naming(s), with the caveat that they should be clarified to only be appropriate when the callee is known to not allocate/lock. IMO it should not be a supported use case to call something that does actually allocate/lock from within the corresponding withoutActually... construct.

(I agree with @Karl that the "it's okay" situation is undesirable anyway—if it's okay that the callee allocates/locks, then the caller is probably an inappropriate place to apply the performance annotation in question.)

1 Like

Ideally API's that do allocations in some cases should be split into separate, say, "alloc" + "safe" + "dealloc" functions, otherwise checks would never be truthful, like advisory locks on unix. Until this ideal future comes, there indeed might be some leeways, like temporary exceptions.

Shall we require some explicit opt-in permission (e.g. compiler setting?) to call "maybe ok" functions in contexts that require strict noLocks/noAllocations? I also think it would be wonderful if there's some diagnostic facility (similar to thread sanitizer) that can verify claims made about noLocks/noAllocations in runtime and complain in case of mismatch.

Re the withoutActuallyLocking name - it sounds a bit misleading as it is not what "actually" happening, if locks are happen to be taken inside.

Let me explore the space a bit, hope it is self explanatory:

@performance(locks=no)
func callbackNoLocks() {}

@performance(locks=yes)
func callbackHasLocks() {}

@performance(locks=maybe)
func callbackMaybeLocks() {}

func realtimeProc(_ execute: @performance(locks=no) () -> Void) { ... }

func test() {
    realtimeProc(callbackNoLocks)    // ok
    realtimeProc(callbackHasLocks)   // error
    realtimeProc(callbackMaybeLocks) // allowed but shall need compiler opt-in
}

@performance(locks=maybe)
func foo() {
    // the following would allow putting @performance(locks=no) on function signature if there is nothing else worse
    @performance(locks=no) {
        someUnknownStuff()     // ok but needs compiler opt-in. runtime diagnostic checks may fire
        callbackNoLocks()      // ok. runtime diagnostic checks may fire
        callbackMaybeLocks()   // compilation error
        callbackHasLocks()     // compilation error
    }
    // the following would allow putting @performance(locks=unknown) on function signature if there is nothing else worse
    @performance(locks=maybe) {
        someUnknownStuff()     // ok but needs compiler opt-in. runtime diagnostic checks may fire
        callbackNoLocks()      // ok. runtime diagnostic checks may fire
        callbackMaybeLocks()   // ok. runtime diagnostic checks may fire
        callbackHasLocks()     // compilation error
    }
}

I think the best way to do this would be to wrap the function with different entry-points, where the @noAllocations version sets the appropriate parameters to ensure the wrapped function really does not allocate.

Otherwise, yeah - a function which selectively allocates shouldn't be tagged at the function level as not allocating. Because it can. And taking advantage of this feature may require some additional work in order to provide the guarantee; that's fine IMO.

7 Likes

I concur: all Swift code is expected to be as efficient as possible with regards to allocation. Unless you’re making a specific guarantee, there’s no point in explicitly saying so.

As I said earlier, I think there might be an argument for an “unsafe” variant that skips checks, since the compiler won’t be able to confirm that for everything. But there should be no room for tolerating violations of that, in the same way that the possibility of incorrectly using Swift.unsafeBitCast(_:to:) is beyond consideration.

Indeed, that'll take some time until compiler can notice all unsafe places and until all calls are annotated (*). OTOH, runtimes checks are relatively easy to implement, in leaf functions like malloc / pthread_mutex_lock, etc (perhaps a few places like these for starters to cover 95% of usual cases, add another dozen of those to cover 99%, refine later to cover the rest):

@noLocks func foo()
    someUnknownCall() // error, annotate or put in "assumingNoLocks" brackets.
    assumingNoLocks {
        someUnknownCall() // ok to compile, happens to be ok in runtime.
        someOtherUnknownCall() // ok to compile, crash boom bang at runtime, as it actually locks.
    }
}