Performance annotations

As I think more about it, I wonder how much of a problem this will be. The notion of "it’s okay that the called function allocates or locks" really depends on your perspective.

As a library author, I might want to annotate certain performance attributes to guide me during my implementation. For example, I've encountered situations where nested functions introduce invisible yet very expensive allocations, and I'd like to be informed if that should happen (including in any called functions). But it's unreasonable to add these flags to every function in the project, so I'm more likely to add them at a coarse level, covering high-level operations (such as public functions), and carve out exceptions for certain code which I expect to allocate.

But then, a consumer of my library may not be okay with the allocations that I flagged as being okay. From their perspective, I may be one of those leaf functions which they're trying to ensure doesn't allocate. And of course, they may are depending on my @noAllocations attribute being absolutely truthful:

So there are these 2 separate meanings of something like @noAllocations - internally, it means "no allocations except ones that I specifically allow", and externally, it has to mean "no allocations whatsoever".

Perhaps that was obvious, but I don't see it mentioned in the pitch, and I think it is worth calling out.

7 Likes

I thought the thorough and throughout annotation was part of the pitch.. I believe it has to be eventually. We can start with, say, a small subset of stdlib, and activate this annotation system based on explicit opt-in, perhaps even on a file by file base, like ARC. There will be pain initially but that's solvable IMHO.

Yes, it is important to have this distinction, e.g. @noAllocationsBut can have internal escape hatches, @noAllocationsEver can not (obviously names must be better).

This sounds very similar to @noImplicitCopy from the roadmap, and I actually think they should work the same way:

  • @noImplicitAllocation would cause the compiler to warn if an allocation occurs.
  • @noImplicitLock would cause the compiler to warn if a lock occurs.

Like @noImplicitCopy, I feel this should not be part of a function’s interface at all. The ability to overrule these attributes means no promises are actually made, and not having Implicit in the name would bury that reality even further.

3 Likes

As a completely distinct set of features, there could be compiler-enforced guarantees that allocations or locks do not occur. This would need to have no exceptions, including calls to other functions.

Anything more lenient than that would just be a false friend, worse than nothing. If there must be exceptions, they should be treated in the same manner as Swift.unsafeBitCast(_:to:).

I’m not convinced this would actually be useful. Maybe for an RTOS?

What about subclasses and protocol conformances? Shall it be like so:

 class Base {
    @noAllocations func foo() {}
    func bar() {}
}

class Derived: Base {
    override func foo() {} // Error
    @noLocks override func foo() {} // probably ok
    @noAllocations override func foo() {} // ok
    @noLocks override func bar() {} // ok
}

protocol Proto {
    @noAllocations func foo()
    func bar()
}

struct S: Proto {
    func foo() {} // Error
    @noLocks func foo() {} // probably ok
    @noAllocations func foo() {} // ok
    @noLocks func bar() {} // ok
}

I've seen some discussion around the unsafePerformance escape hatch but I want to ask, why do we need this? The "Escape hatch" section doesn't have rationale other than "to silence the performance diagnostics".

As a consumer of a library that exposes an API marked with @noAllocations, I would be very surprised to find allocations happening, because the compiler promised me there wouldn't be any.

If a method needs to allocate and "it's okay that the called function allocates or locks" then why mark it with @noAllocations?

4 Likes

The most common need for unsafePerformance, I would guess, is to label code that isn't or can't be annotated with the new attributes, either because it's imported from C, or from an existing Swift package that hasn't yet adopted these attributes, but which is known to no-allocations or no-locks-safe. It could also be used to selectively call into APIs that aren't no-allocations safe in full generality, but which are known to be safe in specific circumstances, or in code that is mostly expected to be no-allocations-safe but in which allocations may be allowed locally in limited circumstances.

3 Likes

Perhaps the semantics would be greatly clarified if we called it, by analogy to withoutActuallyEscaping, withoutActuallyAllocating and withoutActuallyLocking?

6 Likes

Riffing on what you are suggesting, I imagine we want it to be a catchall term like withoutActuallyViolatingPerformanceAssertions? Maybe that is too verbose, I don't know.

I also think that this is preferable. IMO, security is an important enough concern that it should have the monopoly on things called "unsafe" in the language. Locking or allocating can cause performance issues, but in the general case, taking (and later dropping) a lock or allocating don't cause security issues.

I think that withoutActually{PerformanceCharacteristic} is my preferred wording too.

3 Likes

Is the "it's okay that the called function allocates or locks" case feature requirement? If not, I'm also +1 on the withoutActually... naming(s), with the caveat that they should be clarified to only be appropriate when the callee is known to not allocate/lock. IMO it should not be a supported use case to call something that does actually allocate/lock from within the corresponding withoutActually... construct.

(I agree with @Karl that the "it's okay" situation is undesirable anyway—if it's okay that the callee allocates/locks, then the caller is probably an inappropriate place to apply the performance annotation in question.)

1 Like

Ideally API's that do allocations in some cases should be split into separate, say, "alloc" + "safe" + "dealloc" functions, otherwise checks would never be truthful, like advisory locks on unix. Until this ideal future comes, there indeed might be some leeways, like temporary exceptions.

Shall we require some explicit opt-in permission (e.g. compiler setting?) to call "maybe ok" functions in contexts that require strict noLocks/noAllocations? I also think it would be wonderful if there's some diagnostic facility (similar to thread sanitizer) that can verify claims made about noLocks/noAllocations in runtime and complain in case of mismatch.

Re the withoutActuallyLocking name - it sounds a bit misleading as it is not what "actually" happening, if locks are happen to be taken inside.

Let me explore the space a bit, hope it is self explanatory:

@performance(locks=no)
func callbackNoLocks() {}

@performance(locks=yes)
func callbackHasLocks() {}

@performance(locks=maybe)
func callbackMaybeLocks() {}

func realtimeProc(_ execute: @performance(locks=no) () -> Void) { ... }

func test() {
    realtimeProc(callbackNoLocks)    // ok
    realtimeProc(callbackHasLocks)   // error
    realtimeProc(callbackMaybeLocks) // allowed but shall need compiler opt-in
}

@performance(locks=maybe)
func foo() {
    // the following would allow putting @performance(locks=no) on function signature if there is nothing else worse
    @performance(locks=no) {
        someUnknownStuff()     // ok but needs compiler opt-in. runtime diagnostic checks may fire
        callbackNoLocks()      // ok. runtime diagnostic checks may fire
        callbackMaybeLocks()   // compilation error
        callbackHasLocks()     // compilation error
    }
    // the following would allow putting @performance(locks=unknown) on function signature if there is nothing else worse
    @performance(locks=maybe) {
        someUnknownStuff()     // ok but needs compiler opt-in. runtime diagnostic checks may fire
        callbackNoLocks()      // ok. runtime diagnostic checks may fire
        callbackMaybeLocks()   // ok. runtime diagnostic checks may fire
        callbackHasLocks()     // compilation error
    }
}

I think the best way to do this would be to wrap the function with different entry-points, where the @noAllocations version sets the appropriate parameters to ensure the wrapped function really does not allocate.

Otherwise, yeah - a function which selectively allocates shouldn't be tagged at the function level as not allocating. Because it can. And taking advantage of this feature may require some additional work in order to provide the guarantee; that's fine IMO.

7 Likes

I concur: all Swift code is expected to be as efficient as possible with regards to allocation. Unless you’re making a specific guarantee, there’s no point in explicitly saying so.

As I said earlier, I think there might be an argument for an “unsafe” variant that skips checks, since the compiler won’t be able to confirm that for everything. But there should be no room for tolerating violations of that, in the same way that the possibility of incorrectly using Swift.unsafeBitCast(_:to:) is beyond consideration.

Indeed, that'll take some time until compiler can notice all unsafe places and until all calls are annotated (*). OTOH, runtimes checks are relatively easy to implement, in leaf functions like malloc / pthread_mutex_lock, etc (perhaps a few places like these for starters to cover 95% of usual cases, add another dozen of those to cover 99%, refine later to cover the rest):

@noLocks func foo()
    someUnknownCall() // error, annotate or put in "assumingNoLocks" brackets.
    assumingNoLocks {
        someUnknownCall() // ok to compile, happens to be ok in runtime.
        someOtherUnknownCall() // ok to compile, crash boom bang at runtime, as it actually locks.
    }
}

No, that should be banned. If you aren’t dead-certain, don’t say it won’t lock.

And yes, this does mean most functions won’t be able to use it. That is for the best.

It seems to me that there are two issues here: (1) preventing implicit/unexpected allocations/locks within the body of a function; (2) preventing any allocations/locks as a form of API (e.g. so that someone attempting #1 can perform their task well).

As someone who spends an unfortunate amount of time writing audio processing code (both realtime + offline), I’m much more interested in #1, though I can definitely understand the desire for #2 from those involved in writing libraries & exposing stable APIs.

I would expect the documentation of @noLocks functions to specify (in big, bold font) if there are actually situations in which the function may lock/allocate. Since the function calls are spelled out in source code in my function, that’s satisfying enough for me. So for my use case, I don’t mind if a function may actually allocate under some specific situations as long as that fits my needs. An example:

/// Allocates the first time it’s called, then never again. @noAllocations
/// is only used to enforce the writer’s expectations in the body of the function.
@noAllocations
func doProcessing(on batch: Batch) {
	if !setUpCompleted { 
		unsafePerformance { doAllocatingSetup() }
	}
	// do work fast, absolutely no locks or allocations
}

func processOffline() {
	for batch in data.batched() {
		// I don’t care that this allocates the first time it’s run, I just care that the work
		// happens without allocating on *most* iterations
		doProcessing(on: batch) 
	}
}

So in summary I like the proposal as-is. It seems to optimize for issue #1 as opposed to issue #2. The spelling of the annotations does seem to suggest they do #2, but I think it’s still a reasonable spelling & is learnable. The name unsafePerformance is imperfect, but roughly gets the point across. The name withoutActuallyAllocating implies to me a rule (also suggested above) that allocating inside it is strictly banned — I don’t like this rule & would prefer that it be explicitly allowed to allocate inside of the escape hatch.

With regard to C interop, has there been any thought to how C functions will import? Will they be @noLocks or not?

If they import as plain functions (lacking @noLocks), it would be nice to have a header attribute which would import the function as noLocks or noAllocations? In my imagination, this attribute wouldn’t do any checking of the body of the function (seems hard) — just be a promise that this function obeys the rules. I ask because I can imagine a situation like so:

/// There are no situations where locking inside of `bar` is acceptable.
@noLocks
func bar() {
	unsafePerformance {
		// I need to call this C function which doesn’t lock, but it’s not @noLocks
		// because it’s been imported. I mean only to say that calling 
		// `cFunctionThatDoesNotLock` can be ignored by performance diagnostics,
		// not that the possible implicit lock on `globalVariable` can be ignored.
		cFunctionThatDoesNotLock(globalVariable)
	}
}

If we had a header annotation that meant “I promise there are no locks in here” then I could avoid the whole unsafePerformance block in bar and be prevented from an accidental implicit lock when accessing globalVariable.

This example could also apply to un-annotated Swift functions imported from a library, but at least there's a way to spell the annotation in Swift. I care about the C-interop specifically because today's Swift programs which need have critical sections requiring these performance annotations basically must use C interop to achieve them. So the transition from C libraries today to Swift libraries tomorrow requires a gradual Swift rewrite, using lots of interop along the way.

I guess something like

__attribute__ ((noLocks)) void memmove(void*, const void*, size_t)

will do. Totally ignored in C-land (initially), just for Swift interop.

The existing syntax for this is __attribute__((swift_attr("@noLocks"))). You can of course wrap that in a preprocessor macro that’s only defined when the SWIFT preprocessor macro is defined.

2 Likes