Issue with performance cost of runtime exclusivity enforcement

Scenario

We build a framework for use with OData (lets call it TheFramework).

  • TheFramework defines about 400 open classes and 350 internal classes. (OData has highly structured data and metadata, requiring mutable object graphs, so we use classes extensively).

  • TheFramework defines only one struct type (XSUTF16V), which is a wrapper of String.UTF16View.Index that we use as a helper for UTF-16 based string processing. This struct has 2 mutating methods. Instances of this struct type are only ever held in local variables; they are never stored as class properties, struct properties or global variables.

  • TheFramework defines only 4 methods with inout parameters. These methods are used to simulate increment (“++”) and decrement (“—”) operators in some transpiled code. The values which are passed into these methods are always from local variables. These methods only have one inout parameter and don’t access any captured/external state, so it could presumably be reasoned that they cannot introduce an exclusivity violation. An example of one of these methods is here:

        public static func prefixDecrement(_ value: inout Int) -> Int
        {
            let newValue = value - 1
            value = newValue
            return newValue
        }

A typical workload: an app using TheFramework downloads a large set of data (e.g. one million rows) from a backend system as a stream of JSON, converts the parsed JSON to class instances, and stores the objects in a local database (e.g. SQLite).

Issue

Runtime exclusivity enforcement is taking 60% of CPU time (e.g. increasing the execution time of a one million row download from 35 seconds to 85 seconds). This was determined by changing the Build Settings for Exclusive Access to Memory to disable Run-Time Checks, which results in the faster execution (but notably: no violations of exclusivity are detected).

Profiling indicates that the relevant swift_beginAccess enforcement calls that cause the performance degradation appear to be related to ensuring exclusive access to class properties. The XSUTF16V struct and increment/decrement methods are not currently implicated by the profiling results.

Discussion

  • Per Memory Safety — The Swift Programming Language (Swift 5.7), all our memory accesses would appear to be “instantaneous”. (Except for the above-mentioned XSUTF16V mutating methods and the increment/decrement methods, which don’t appear to be relevant to the performance degradation).

  • Per Swift.org - Swift 5 Exclusivity Enforcement (“As a general guideline, avoid performing class property access within the most performance critical loops, particularly on different objects in each loop iteration. If that isn’t possible, making the class properties private or internal can help the compiler prove that no other code accesses the same property inside the loop”)

We believe that we are following those guidelines as best we can, given the nature of our workloads and the complex object graph parsing and processing that is required.

Questions

  1. Can the Memory Safety document be updated to discuss (perhaps with some examples) conflicts that are possible when using class properties but without the use of inout parameters or mutating struct methods?

  2. How can the Swift compiler and runtime be improved to avoid 60% of application’s CPU consumption being eaten up by exclusivity checking for class properties that aren’t used as inout parameters?

  3. A relative jokingly suggested that a 60% performance degradation would help to encourage people to upgrade their phones, but that does make one wonder if the runtime checking can be so costly, how its enablement by default in release mode is justifiable?

  4. Per Swift.org - Swift 5 Exclusivity Enforcement (“The impact should be small in most cases; if you see a measurable performance regression, please file a bug so we know what we need to improve.”)
    What is considered to be a reasonable maximum performance impact?

  5. Per Swift.org - Swift 5 Exclusivity Enforcement (“Disabling run-time checks in Release builds is strongly discouraged because, if the program violates exclusivity, then it could exhibit unpredictable behavior, including crashes or memory corruption. Even if the program appears to function correctly today, future release of Swift could cause additional unpredictable behavior to surface, and security exploits may be exposed.”)
    Does this mean that the exclusivity rules (definition of a “conflict”) might be changed, or just that the current runtime checking might be missing some conflicts and that an updated runtime might detect formerly undetected conflicts?

  6. The definition of a conflict in Memory Safety — The Swift Programming Language (Swift 5.7) (“At least one is a write access or a nonatomic access”) implies that nonatomic read accesses can conflict with one another, but how does one obtain a long-term read access that could conflict with another long-term read access? Which Swift language features can result in long-term read access (the document doesn’t mention long-term read access)?

  7. The doc also states (section: Understanding Conflicting Access to Memory) - “However, the conflicting access discussed here can happen on a single thread and doesn’t involve concurrent or multithreaded code.” Why then are conflicts described with reference to nonatomic operations, which implicitly refers to atomicity, which involves concurrent/multithreaded code?

  8. Per the definition of a conflict in Memory Safety — The Swift Programming Language (Swift 5.7) (section: Characteristics of Memory Access), why doesn’t the following produce a conflict (as the durations of the calls to swap and swap2 overlap and they both read and write the location of x.value and the location of y.value)? (Does the document need updating to clarify?)

let x = MyClass()
let y = MyClass()

func exclusivityTest() {
    swap(&x.value, &y.value)
}

func swap(_ a: inout Int, _ b: inout Int) {
    let temp = a
    a = b
    b = temp
    swap2(&a, &b)
}

func swap2(_ a: inout Int, _ b: inout Int) {
    let temp = a
    a = b
    b = temp
}

class MyClass {
    var value: Int = 0
}
2 Likes

I can't answer the performance questions right now, but as far as the concept of exclusivity is concerned:

  1. What this means is that the law of exclusivity is part of the language model. Whether or not you keep the runtime checks enabled, the compiler will assume that all code abides by it.

    If your code requires runtime exclusivity enforcement but you have disabled those checks, and it does actually violate the law of exclusivity, your code will exhibit undefined behaviour. Whether or not it happens to produce the correct result today is immaterial; an updated compiler or runtime library might expose that violation in unpredictable ways.

  2. This is poorly worded: two read accesses cannot be in conflict (whether they are atomic or not). What is meant is that there is a conflict if: at least one is a write access and at least one is non-atomic (not "or"). These should probably be separate bullet-points.

    In other words, a non-atomic write together with a non-atomic read/write, or mixed atomic/non-atomic operations.

  3. Atomic operations are for mutable state which is shared between threads, and it is expected that they will operate on the same memory locations at the same time. They are exempt from the law of exclusivity. That's all this means.

  4. The calls to swap and swap2 do not overlap - the call to swap2 is nested within swap, but that is not the same as an overlap.

    exclusivityTest is the code which performs the inout access of the class' instance members, so that it where exclusivity is enforced - at the line which calls swap. The same happens within swap at the line which calls swap2, but this time, swap already knows that exclusivity is guaranteed, so swap2 can be proven safe just with local reasoning.

EDIT: phrasing improvements =)

2 Likes

To further expand on @karl's response, you have misunderstood what the definition of "access" is. As @karl rightly says, the access to the class property begins (and ends) in the function exclusivityTest. Essentially, the operation is:

begin mutable access to x.value
begin mutable access to y.value
call swap with x.value and y.value
end mutable access to y.value
end mutable access to x.value

When a function accepts an inout variable, it does not begin a new access within that function. Instead, it extends an existing access throughout the lifetime of the function with the inout parameter. During the time of that access, the inout parameters are "dead".

This is also how you can have single-threaded exclusivity violations. The following code is an exclusivity violation in one thread:

let x = MyClass()

func add(_ a: inout Int) {
    a += x.value
}

add(&x.value)

We begin a mutable access to x.value before the call to add, and then within add we attempt to perform a read access to x.value. This violates the law of exclusivity.

Importantly, non-instantaneous accesses are not only caused by inout variables. They are also caused by mutating functions. mutating is essentially a spelling for inout self. Any calls to mutating functions will need to perform exclusivity checks.

Note also that exclusivity checks still need to be performed for instantaneous accesses, because those accesses may themselves conflict with a non-instantaneous one.

2 Likes

Thanks folks for the clarifications.

Somewhat of a false alarm regarding the performance.

On retesting, with a microbenchmark, I can observe a 10% overhead with exclusivity testing enabled (due to swift_beginAccess calls).

With full benchmark, as described above, the "slow" test results had app in background (on Mac) which seemed to give it considerably fewer CPU resources. When ensuring the full benchmark app ran in foreground, the degradation was about 7%. Still somewhat concerning but not nearly as bad as I had feared.

2 Likes