Explicit Copies Mode

I’ve been prototyping a new feature aimed at performance-conscious programmers that removes the mystery around when copies (such as retains) of values occur in their code. The idea is that you can ask the compiler to force you to acknowledge every use of a variable binding that requires a copy of it. Let’s take a look at an example:

class Data {
  var value: Int = 0
}

func accumulate(_ xs: [Data]) -> Data {
  let total = Data()
  for x in xs {
    total.value += x.value
  }
  return total
}

If this accumulate function were on an ultra-hot path of code, I’d want to know exactly where all copies might be happening, so that I can tweak the code to minimize them. When I turn on explicit copies mode for this function, I’d get diagnostics such as this first one:

forum-examples.swift:7:12: error: explicit 'copy' required here
 5 | func accumulate(_ xs: [Data]) -> Data {
 6 |   let total = Data()
 7 |   for x in xs {
   |            `- error: explicit 'copy' required here
 8 |     total.value += x.value
 9 |   }

Here, it’s pointing out that when iterating over my array xs, a copy of it is made. As a programmer, I have two options: (1) acknowledge the copy by writing for x in copy xs , or (2) I could push that copy out of this function and into callers by taking xs as a consuming parameter (in hopes that it’s the last-use within callers to elide the copy). If you’re curious why there’s a copy when iterating over a collection at all, it’s because Collection’s makeIterator is a consuming method, and that’s implicitly invoked at the start of this for-in loop to begin the iteration.

The next place where there’s a copy is when returning the result:

forum-examples.swift:10:10: error: explicit 'copy' required here
 8 |     total.value += x.value
 9 |   }
10 |   return total
   |          `- error: explicit 'copy' required here
11 | }
12 |

In terms of language semantics, the return of an lvalue expression like total does make a copy of it. But in this particular case, the copy is automatically elided, even in -Onone (aka no optimizations).

Sometimes, copies are not always elided in all optimization modes. For example, this copy of x is not elided in -Onone:

func elision(_ cond: Bool) -> Data {
  let x = Data()
  let y = copy x // <- not elided in -Onone
  if cond {
    return consume y
  }
  return consume x
}

So, there are a few areas of the design that’s yet to be hammered out, such as:

  • What kinds of copies should be acknowledged, and how (warnings, errors)?
    • I personally don’t think your program’s diagnostics should change based on the optimization level (e.g., Debug vs Release), so all copies that are not guaranteed to eliminated must be acknowledged.
  • How should the boundaries of this explicit copy mode be defined?
    • It’s currently prototyped as a function-level attribute that is non-viral: it doesn’t require callers or callees to acknowledge copies in their respective functions. It’s likely people will want file-level and/or module-level application of this mode. But there’s functions that are marked @_transparent that are always inlined, such as synthesized getters, which blurs that boundary.

Happy to hear people’s thoughts on this!

35 Likes

I like this idea in theory for copyable types and having it as an opt-in "mode" makes sense to me. I would like manual overrides when copies become a compiler warning or error on a project, module, struct/class, and function basis.

However, an opposing thought is to just use noncopyable types, but I do see why this "mode" could be used (when noncopyable types aren't applicable).

1 Like

This seems interesting for sure, opt-in by method sounds good as well. And I’m glad you acknowledged that the copies one needs to annotate cannot differ between debug/release mode, that’d be a nightmare to to keep in a codebase since one usually compiles the same code in both…

I do wonder though if compiling in release mode should offer some kind of hint that “this copy was actually removed” or something… since the copy end up being misleading perhaps if a function is littered with them, but many of them end up being optimized away – what is some way we could make this more managable and understandable? :thinking:

5 Likes

I think this would be super helpful for us for certain pieces of code - it could be interesting with a viral variant too at times, we would definitely make use of this!

Edit: To clarify: viral ”downwards” to functions that are being called by an annotated function.

5 Likes

Agree it would be very useful to understand which copies are elided. One other option would be to just produce a warning in debug mode while error in release mode builds - then one could have the same source but choose to hide it in debug builds output. Not super clean perhaps, but at least for us, we’d be happy to annotate and just want to know that we eliminate the copies for the release build really. Bit tricky to consolidate the two.

It is implicit in your write-up, but at the risk of being entirely obvious I'd like to confirm that anything written with this feature turned on will also compile (without warnings or errors) and exhibit the same behavior when this feature is turned off (i.e., is valid "vanilla Swift"), making this feature an opt-in warning/error group.

This would be in distinction from what we've called language modes or default isolation modes where the same code has different semantic meaning and generates different diagnostics depending on the mode.

1 Like

This is in practice no different than what @ktoso was saying we should avoid by having different behavior for annotating copies itself in debug vs. release builds, because it means one kind of build fails and one doesn't. Unless the user chooses to upgrade that warning to an error in their debug builds, in which case, it begs the question why the compiler didn't just do that to begin with.

In general, diagnostics that differ based on optimization mode are generally a bad idea because many workflows build release builds far less frequently than debug builds, meaning that errors wouldn't get caught until much later. Also important is that we're talking about a language/compiler change and the language/compiler has no concept of "debug vs. release builds"; there are just various flags that control debug info and optimization level and they can be combined in whatever manner the user wants.

8 Likes

I’m going back and forth if this is a good idea or not in debug builds. The first example shows really well why I’m on the fence. As you have mentioned, both places where the copy operator would be required are actually removed with -O: Compiler Explorer
There is actually one retain/release pair in the loop for x which today doesn’t get removed. Assuming this get’s diagnosed properly it would be the only copy that will remain today in a release build:

class Data {
  var value: Int = 0
}

func accumulate(_ xs: [Data]) -> Data {
  let total = Data()
  for x in copy xs { // optimized away
    total.value += (copy x).value // retain/release pair 
  }
  return copy total // optimized away
}

However, I don’t even think the retain/release of x is really required and with more OSSA improvements this could get optimized away as well.

Either way, we end up with at least 2 out of 3 false positives. Is this really helping me understand where copies are problematic? The signal to noise ratio of real copies vs potential copies can be very high. I don’t think this replaces the need to look at the assembly or IR or use @_assemblyVision or profile the code to actually know which copies actually end up in a release build and matter. I have tried explicit borrowing and consuming ownership modifiers with Copyable types for exactly this purpose but they end up not being that useful for performance sensitive code because of that.

I also don’t think we can practically make this annotation emit errors. Currently explicit borrowing and consuming ownership modifiers and the implicit conventions are not considered API for Copyable types. They can be freely changed without breaking source. They are only part of the ABI. With this feature (and errors) we would make ownership modifiers part of the API as changing them could break anyone who would use this feature.

With that being said, I would very much appreciate a tool that can easily show me all the real copies and retain/release pairs inline. I’m currently either profiling, using a disassembler to look at the generated code or reduce my code enough so that I can use godbolt to either look at the assembly or IR. Sometimes also @_assemblyVision but that doesn’t seem to always emit remarks.

Release builds are always necessary for any of those workflows so I wouldn’t mind if this would be required for this as well and then require all real copies to be marked as such.

As I have mentioned above, I don’t think errors are practical. Warnings or remarks might be a good start. I think what would be ideal is if this could be somehow integrated with swift-testing and instead of a build error we would get a failed test (cc @grynspan). This would be much more manageable. (A bit of topic but I would generally like more perf related testing infrastructure e.g. to check the generated instructions like swift-mmio, similar to the compiler tests)

Last but not least, I’m usually not worried about copies of Int’s and other smallish BitwiseCopyable types. Adding explicit copies for those wouldn’t be useful to me. I still think scoping this to a function (or maybe a type or even a complete file) would be very useful. We could say that if the type is BitwiseCopyable and smaller than x bytes (e.g. 32 bytes) we don’t require explicit copies. Maybe we could even make this configurable.

Edit: Generic functions are also interesting. The optimizer can usually do a much better job at eliminating copies if the code get’s specialized. Sometimes I’m interested in the performance of unspecialized code but I’m also very interested in how the code performs if specialized. I think we need something that makes this configurable and diagnose only copies that remain given a concrete type (or multiple) that could be specified similar to @specialize.

12 Likes

Uh okay, even today explicit ownership modifiers, which requires explicit copy's, emit an error and not a warning if a copy is missing. So the ownership modifiers are strictly speaking already part of the API and e.g. changing a method from borrowing to consuming can be source breaking e.g.:

class Data {
  var value: Int = 0
}

func accumulate(_ xs: borrowing [Data]) -> Data {
  let total = Data()
  for x in xs { // error: 'xs' is borrowed and cannot be consumed
    total.value += x.value
  }
  return total
}

Not sure if it was intentional to make ownership modifiers part of the API for Copyable types. I thought this would only emit warnings.

What is currently unfortunate as well is that adding the explicit ownership modifier and then the required copy we end up with a retain of xs which we don’t without explicit borrowing and copy.

2 Likes

@Joe_Groff was the forced copy in for … in supposed to be addressed by the part of SE-0432 about for case?

Correct.

I believe it would be feasible for it to work such that, for when explicit copies mode is enabled in a function, and you add a copy somewhere you thought needed it, but it’s elided or there never would’ve been a copy there anyway, that the compiler emits a warning (or remark).

I could set things up such that the explicit copies mode always defaults to warnings, and you can add an extra compiler flag to upgrade those diagnostics into errors. This would effectively let you upgrade-to-errors for your Release build.

It’s worth noting that this explicit-copies checking needs to happen earlier in the optimization pipeline, just to guarantee stability across different versions of the compiler even for the same optimization level. It’s always possible that the optimizer gets smarter (or not) between minor point-releases and it would be annoying to see the diagnostics change.

Right. The compiler only knows about optimization flags & modes, and there’s at least three major ones like -Onone, -O, and -Osize , plus some profile-guided optimization options that can be layered on top, etc. I’d prefer to keep the language disconnected from optimization flags, as it lets people experiment with squeezing out performance without source code changes. The same kind of people who would want explicit-copies mode are likelty to also tweak optimization flags.

2 Likes

Based on what I can see, the first one, copy xs, doesn’t get eliminated until after quite a bit of optimization: after the generic function Collection<>.makeIterator() becomes specialized, and then inlined, and then some other optimization realizes that there’s no need to construct an IndexingIterator<Array<Data>> at all.

So, I wouldn’t expect this copy to always get eliminated, as semantically you are copying the array (cheaply) in Swift when you iterate over it, to make mutation of it safe. In this case, there’s no mutation and the copy is eliminated.

Thanks for looking into this. It happens to be quite interesting, because it varies depending on the target platform!

There is no retain-release at all in this function when I compile the code for Darwin with -O . I believe it’s because on Linux, the call to Data.value's getter somehow doesn’t get inlined, which is what enables the elimination of the retain of x prior to calling the method. More reason to not have this mode depend on optimization level!

Edit: It seems I’m wrong. My local build of the compiler, branched from main at some point, is eliding the copies because it’s decided to inline. I haven’t determined the reason why it’s inlining it when it doesn’t on Darwin in Swift 6.2

I don’t think it’s required either, because x is a local variable and there’s no exclusive-access worries here. My prototype patch does eliminate these retains in a heavy-handed way. I hope to land a separate fix that is safe eventually, as it’s one of those surprising things you see immediately when playing around with this new mode.

3 Likes

We need to figure out a workable model for the usual developer workflow:

  • I’m building my library, iterating on it in debug builds, or debugging it
  • It’s actually released and here i care about performance and the copies…

So… if I have a piece of code

copy bla

and that gets optimized away in release mode… that is what I care about in production, so I’d like the sources to reflect “aha, there’s no copy here, NICE”.

I would not want to have to keep around this copy in the sources because it is misleading me into thinking “oh man, there’s a copy here, I better fix that” – but there isn’t, it’s optimized in release builds.

So; perhaps this mode should only issue warnings and checks in release builds, or better, allow customization?

Therefore debug builds would be silent and we don’t care about them…? Or should this @explicitCopy(release,debug) be annotated with where we want the checks to be performed? This way I’d keep this in @explicitCopy(release) (spelling TBD), and I won’t be confused in debug builds by spurious copy lines that don’t matter in the released product.

5 Likes

This is a great point and I think will depend on the implementation. I don’t know yet how high the signal-to-noise ratio will be!

My initial plan was to ensure that, early on in the optimization pipeline, we have a canonical, minimal view of what local copies are required within a function. Not those that are eliminated after inlining or specialization. This should help increase the signal-to-noise ratio in my first example, as the copy for a return is often eliminated. Same goes for simple renames of let bindings.

Here’s a simple example where with local-reasoning, the copy y would be required to be written:

func unusedParameterInCall() {
  let y = Data()
  for _ in 0..<10 {
    hello(copy y, false)
  }
}

func hello(_ d: consuming Data, _ b: Bool) {
  if b {
    print("Hello \(d.value)")
  }
}

Within the full optimization pipeline, all of the code within unusedParameterInCall is superfluous! It gets optimized into a function with one instruction that returns immediately. So it’s not just the copy expression that’s unneeded.

We can’t run the full optimization pipeline in every build, so acknowledged copies that end up eliminated after quite a bit of optimization magic, may end up being superfluous. Discovering those might be possible, but it also might be quite difficult. I’m open to hearing more opinions on this, because finding the right balance here is important!

The end-goal here is that explicit-copies mode is a tool that reduces the need for someone to become a compiler expert to optimize their code. Instead, with knowledge of Swift’s ownership features, plus the explicit-copies mode to help draw their attention to copies, they should be able to rewrite their code to minimize them. This feature is meant to work in tandem with those ownership modifiers and if they aren’t working as expected to avoid copies, then they need to be fixed.

5 Likes

The way I think something like this should work is that we should guarantee that copies will be elided in some core set of situations and then allow programmers to opt in to being explicit about their desire to copy everywhere else. That core set of situations could grow over time, but never shrink, so it would have to be shaped by what copies we think we can guarantee to eliminate. At minimum, it would start with things like "a conservative copy of a local var passed as a borrowed parameter will always be eliminated if the var does not escape and is not mutated during the call"; that would be semantically compatible with the current language. We could also experiment with a mode that simply doesn't do conservative copies at all, but I'm wary of adding more hard-fork modes like that.

Anyway, these guarantees would have to be real guarantees, and the compiler would be responsible for doing them even in debug builds. It would be very short-sighted to define the set of no-explicit-copy-required situations by what -O is able to eliminate in a particular release, because that cannot realistically be assumed to be stable. As nice as it would be to say that the optimizer should never regress by failing to eliminate a copy that it eliminated in a previous release, in practice that's essentially a "never rewrite an optimizer pass again" level of restriction.

16 Likes

I suspect this isn't specific to explicit copy mode proposed in this thread. For example:

func test(_ x: consuming String) {
    let y = copy x
    let z = x
    print(y)
    print(z)
}

I don't have the knowledge to view SIL or assembly code to verify if the copy is optimized away in above code, but I guess it might be. If so, I think how to figure out a copy is optimized away should probably be considered as a separate feature.

PS: I think the proposed feature will be a great tool to help average users understand how ownership works. IMO while many users don't use ownership explicitly, they need to understand it.

1 Like

Full disclosure, I'm coming at this as just a casual user of swift. Feel free to correct any of my misconceptions.

From what i can see, this would effectively introduce a dialect (or at the very least, a variant) of Swift, a sort of rusty-swift with explicit ownership semantics.

Reading though this, I understand that “vanilla swift” would be able to compile rusty-swift, but wouldn't this eventually result in projects, sample code, and libraries becoming a patchwork of sometimes-ownership-annotated swift? Imo that isn't a desirable outcome, especially for newcomers encountering inconsistent syntax.

I do see the appeal of explicit annotation. However, I'd much rather this be an editor-based annotation (I know editor integrations are out of scope for SE, but still) instead that performance-minded developers can enable. I know android studio does this for kotlin function argument labels, so it's not completely unprecedented.

Open to feedback :D

1 Like

It’s not really that much of a dialect. Explicit copies are already required for function parameters that have explicitly declared ownership. This just lets the coder opt-in to this behavior in more places than is currently possible.

There’s a basic design question here about how aggressively the mode implicitly performs borrows. If it only performs borrows in situations where it can prove that an exclusivity conflict is impossible, then it’s still semantically equivalent to today’s Swift for code that it does’t emit a diagnostic for. This makes it essentially an opt-in warning that users can decide to map as an error. If it performs borrows more aggressively than that, then it’s truly a new mode the same way that default isolation is.

Yeah, I think "mode" might be the wrong word for this, since it seems like something that shouldn't affect visible language behavior, and "mode" sounds like a deeper dialect change.

One question in my mind is what sorts of types we should actually warn about copies for. To me, it seems like it would rarely be useful to complain about copying an Int or other small trivial type. But at the same time, I would be interested in knowing when the compiler wants to copy a [1024 of Int] which, despite still being trivial, is still large enough for a copy to be expensive. It seems like there's a reasonable threshold below which copies can still be considered cheap enough to ignore even in a "no copies" environment.

8 Likes