Performance annotations

It seems to me that there are two issues here: (1) preventing implicit/unexpected allocations/locks within the body of a function; (2) preventing any allocations/locks as a form of API (e.g. so that someone attempting #1 can perform their task well).

As someone who spends an unfortunate amount of time writing audio processing code (both realtime + offline), I’m much more interested in #1, though I can definitely understand the desire for #2 from those involved in writing libraries & exposing stable APIs.

I would expect the documentation of @noLocks functions to specify (in big, bold font) if there are actually situations in which the function may lock/allocate. Since the function calls are spelled out in source code in my function, that’s satisfying enough for me. So for my use case, I don’t mind if a function may actually allocate under some specific situations as long as that fits my needs. An example:

/// Allocates the first time it’s called, then never again. @noAllocations
/// is only used to enforce the writer’s expectations in the body of the function.
@noAllocations
func doProcessing(on batch: Batch) {
	if !setUpCompleted { 
		unsafePerformance { doAllocatingSetup() }
	}
	// do work fast, absolutely no locks or allocations
}

func processOffline() {
	for batch in data.batched() {
		// I don’t care that this allocates the first time it’s run, I just care that the work
		// happens without allocating on *most* iterations
		doProcessing(on: batch) 
	}
}

So in summary I like the proposal as-is. It seems to optimize for issue #1 as opposed to issue #2. The spelling of the annotations does seem to suggest they do #2, but I think it’s still a reasonable spelling & is learnable. The name unsafePerformance is imperfect, but roughly gets the point across. The name withoutActuallyAllocating implies to me a rule (also suggested above) that allocating inside it is strictly banned — I don’t like this rule & would prefer that it be explicitly allowed to allocate inside of the escape hatch.

With regard to C interop, has there been any thought to how C functions will import? Will they be @noLocks or not?

If they import as plain functions (lacking @noLocks), it would be nice to have a header attribute which would import the function as noLocks or noAllocations? In my imagination, this attribute wouldn’t do any checking of the body of the function (seems hard) — just be a promise that this function obeys the rules. I ask because I can imagine a situation like so:

/// There are no situations where locking inside of `bar` is acceptable.
@noLocks
func bar() {
	unsafePerformance {
		// I need to call this C function which doesn’t lock, but it’s not @noLocks
		// because it’s been imported. I mean only to say that calling 
		// `cFunctionThatDoesNotLock` can be ignored by performance diagnostics,
		// not that the possible implicit lock on `globalVariable` can be ignored.
		cFunctionThatDoesNotLock(globalVariable)
	}
}

If we had a header annotation that meant “I promise there are no locks in here” then I could avoid the whole unsafePerformance block in bar and be prevented from an accidental implicit lock when accessing globalVariable.

This example could also apply to un-annotated Swift functions imported from a library, but at least there's a way to spell the annotation in Swift. I care about the C-interop specifically because today's Swift programs which need have critical sections requiring these performance annotations basically must use C interop to achieve them. So the transition from C libraries today to Swift libraries tomorrow requires a gradual Swift rewrite, using lots of interop along the way.

I guess something like

__attribute__ ((noLocks)) void memmove(void*, const void*, size_t)

will do. Totally ignored in C-land (initially), just for Swift interop.

The existing syntax for this is __attribute__((swift_attr("@noLocks"))). You can of course wrap that in a preprocessor macro that’s only defined when the SWIFT preprocessor macro is defined.

2 Likes

Casting the resurrection spell on this thread. :magic_wand:

I guess Swift Atomics would need to add the annotations?

And can I use it at all with generics?

Also seems like I should be able to use it on a computed variable:

1 Like

It's quite correct about (unspecialized) generics. You can see what it's talking about in a profile as calls to swift_getGenericMetadata and the like.

Oh I meant to say, "can I use generics at all? (without potential locking)" Here's my little fixed size vector:

/// Fixed size vector.
class Vec<T> {

    private var storage: UnsafeMutableBufferPointer<T>

    init(count: Int, _ f: () -> T) {
        storage = UnsafeMutableBufferPointer<T>.allocate(capacity: count)
        _ = storage.initialize(from: (0..<count).map { _ in f() })
    }

    init(_ array: [T]) {
        storage = UnsafeMutableBufferPointer<T>.allocate(capacity: array.count)
        _ = storage.initialize(from: array)
    }

    deinit {
        storage.baseAddress?.deinitialize(count: count)
        storage.deallocate()
    }

    var count: Int { storage.count }

    subscript(index:Int) -> T {
        get {
            return storage[index]
        }
        set(newElm) {
            storage[index] = newElm
        }
    }
}

Making an IntVec I get:

1 Like

I don't think it applies to all generics, generic structs seem to work:

/// Fixed size vector.
struct Vec<T> {
    private var storage: UnsafeMutableBufferPointer<T>

    init(count: Int, _ f: () -> T) {
        storage = UnsafeMutableBufferPointer<T>.allocate(capacity: count)
        _ = storage.initialize(from: (0..<count).map { _ in f() })
    }

    init(_ array: [T]) {
        storage = UnsafeMutableBufferPointer<T>.allocate(capacity: array.count)
        _ = storage.initialize(from: array)
    }

    var count: Int { storage.count }

    subscript(index:Int) -> T {
        get {
            return storage[index]
        }
        set(newElm) {
            storage[index] = newElm
        }
    }
}

@_noLocks
@_noAllocation
func byteManipulation(vector: inout Vec<Int>) {
    guard vector.count > 0 else { return }

    vector[0] = 42
}
3 Likes

Hmm... why would class vs struct make a difference?

I used a class because I wanted a deinit.

The current implementation requires generics to be fully specialized, which isn't currently possible to achieve for overridable methods or deinitializers for classes. It should be possible in principle to use final class methods as long as doing so doesn't trigger any retain/release traffic, though.

2 Likes

I get this on a struct even w/o generics:

1 Like

I'm getting similar behaviour.

Using a typed buffer:

public struct IntVec64 {
  
  private let count = 64
  private let ptr: UnsafeMutableBufferPointer<Int>

  public init() {
    let ptr = UnsafeMutableBufferPointer<Int>.allocate(capacity: 64)
    self.ptr = ptr
  }

  func setValue(_ value: Int, forIndex index: Int) {
    // ERROR: Co-routine calls can cause an allocation
    ptr[index] = value
  }

  func value(forIndex index: Int) -> Int {
    ptr[index]
  }
}

@_noLocks
@_noAllocation
func someMethod(_ vec: inout IntVec64) {
  vec.setValue(42, forIndex: 0)
}

Using a raw pointer:

public struct IntVec64 {

  let count = 64
  let stride = MemoryLayout<Int>.stride
  private let ptr: UnsafeMutableRawPointer

  public init() {
    let alignment = MemoryLayout<Int>.alignment
    let byteCount = stride * count
    let ptr = UnsafeMutableRawPointer.allocate(byteCount: byteCount, alignment: alignment)
    self.ptr = ptr
  }

  func setValue(_ value: Int, forIndex index: Int) {
    // ERROR: Using type 'Int' can cause metadata allocation or locks
    ptr.advanced(by: stride * index).storeBytes(of: value, as: Int.self)
  }

  func value(forIndex index: Int) -> Int {
    ptr.load(fromByteOffset: stride * index, as: Int.self)
  }
}

@_noLocks
@_noAllocation
func someMethod(_ vec: inout IntVec64) {
  vec.setValue(42, forIndex: 0)
}

I guess this is a bug in 5.7, if you're using that version. Not reproducible in 5.8 snapshot from 2022-12-29 for me.

1 Like

Ah ha! Yes, indeed. Works with the 5.8 toolchain. Let's see how far I get with this :slight_smile:

Definitely getting a little further here, but hitting a wall when importing Swift Atomics:

import Atomics

public struct WrappedInt {
  
  private let wrappedInt = UnsafeAtomic<Int>.create(0)
  
  @_noLocks
  @_noAllocation
  public func value() -> Int {
    // ERROR: Called function is not available in this module and can have unpredictable performance
    wrappedInt.load(ordering: .relaxed)
  }
}

The error is "Called function is not available in this module and can have unpredictable performance".

@audulus mentions the imported library requiring annotations.

From my reading above though, my understanding was that this feature would work without called code needing to be annotated.

Any clue how we could get this working with Swift Atomics?

Just re-read, looks like Swift Atomics will require annotations.

Any "function effect" could be enforced as a performance assertion.

At a granular level, we should have, for example:
trap, synchronize, read, write, retain, release/deinit, escape

But we should not confuse these two concepts. A function effect affects codegen by giving the optimizer more information. A performance assert only affects diagnostics.

Crucially, an effect annotation can be "unsafe", while a performance assert cannot be unsafe (the escape hatch should be called uncheckedPerformance).

Note, for example, that a "retain effect" is not the same as an "implicit copy" performance assertion. In fact, @noImplicitCopy is the only additional performance assert I can think of at the moment.

The advantage of a top-level @performance annotation is that it more clearly communicates the meaning of the annotations. It's important for programmers to be aware of the distinction between an effect, which changes -O behavior, and an assert which provides diagnostics. Maybe that's solvable with documentation.

1 Like

This is reminiscent of how the early "nonnull argument" annotation in GCC was mostly unusable because the optimizer would take advantage of it to delete null checks.

1 Like

Using _unsafePerformance also fails to work with generics in Swift 5.8:

public struct Wrapped<T: AtomicValue> {
  
  private let wrapped: UnsafeAtomic<T>
  
  init(_ value: T) {
    self.wrapped = .create(value)
  }
  
  @_noLocks
  @_noAllocation
  public func value() -> T {
    // ERROR: Generic closures or local functions can cause metadata allocation or locks
    _unsafePerformance { wrapped.load(ordering: .relaxed) }
  }
}

You might need to put the _unsafePerformance there around the caller to value(), since value itself is still generic, and to the compiler it looks like it's returning an opaque generic value inside its implementation. Maybe we ought to defer checking the performance annotations on generics to only check them on specializations, though.

Thanks for the idea, but I think then the utility of the annotations would begin to get diminished as I'd end up making an exception for large blocks of code which I really want checked.

For example, here's a function for a queue I'm working on:


  
  @_noLocks
  @_noAllocation
  public func enqueue(_ element: Element) -> Bool {
    repeat {
      let pos = enqueuePos.load(ordering: .relaxed)
      let cellPtr = elementsPtr.advanced(by: pos & elementsMask)
      let seq = cellPtr.pointee.sequence.load(ordering: .acquiring)
      let delta = seq - pos
      switch delta {
      case 0:
        let (success, _) = enqueuePos.weakCompareExchange(
          expected: pos,
          desired: pos + 1,
          successOrdering: .relaxed,
          failureOrdering: .relaxed
        )
        if success {
          cellPtr.pointee.element = element
          cellPtr.pointee.sequence.store(pos + 1, ordering: .releasing)
          return true
        }
      case ..<0:
        return false
      default:
        continue
      }
    } while true
  }

Ideally I'd have this all verified as real time 'safe' by the compiler. If I were to wrap enqueue(_:) with _unsafePerformance at the call site, wouldn't I find myself silencing errors that I really wanted to be informed of?

1 Like