Add shared storage to property wrappers

Evolution Pitches
1

(Add shared storage to property wrappers pitch on github)

Introduction

Property Wrappers have empowered users to abstract common property implementation details into expressive components. This proposal aims to make property wrappers more flexible and efficient by allowing them to opt-in to a shared storage.

Motivation

Property Wrappers are responsible for wrapping common getting and setting boilerplate and also for storing any auxiliary helper properties. Often, these helpers are constant across different instances of the wrapper, not changing after initialization. Thus, having to store these properties in each individual wrapper instance should be avoided. In the following Clamped example, every wrapped instance will store its own range — even though there isn't a way for this range to change across different Hud initializations.

@propertyWrapper
struct Clamped<Value: Comparable> {
  private var value: Value
  let range: ClosedRange<Value>
  
  init(wrappedValue: Value, _ range: ClosedRange<Value>) {
    self.value = range.clamping(wrappedValue) 
    self.range = range
  }
  
  var wrappedValue: Value {
    get { value }
    set { value = range.clamping(newValue) }
  }
}

struct Hud {
  @Clamped(0...100) var value = 100
}

// the `range` property is constant and has the same value
// on both `hud1` and `hud2` 
let hud1 = Hud()
let hud2 = Hud()

API-level property wrappers

Another motivation for this feature is mentioned in the Static property-wrapper attribute arguments Future Direction's section of SE-0293. To achieve consistency across the multiple initialization kinds API-level property wrappers are not allowed to have arguments in their wrapper attribute.

Proposed solution

We propose introducing storage that is shared per property wrapper instance. The storage is immutable, initialized once, and stored outside of the instance scope. It's a tool for property wrapper authors to optimize their wrapper abstractions and avoid repeated unnecessary storage.

@propertyWrapper
struct Clamped<Value: Comparable> {
  shared let storage: RangeStorage<Value>
  
  var wrappedValue: Value { ... }
  
  // a plain struct, with properties that will be shared across wrappers
  struct RangeStorage<Value: Comparable> { ... }
}

Detailed design

The storage is declared using the new shared property attribute inside a Property Wrapper declaration. This property will be initialized and stored globally by the compiler while remaining accessible to the property wrapper instance like any other private property defined in the wrapper type.

In the next example, the RangeStorage struct will be used for the Clamped wrapper.

struct RangeStorage<Value: Comparable> {
  let range: ClosedRange<Value>
  init(_ range: ClosedRange<Value>) { 
    self.range = range
  } 
  
  func clamp(_ value: Value) -> Value {}
}

@propertyWrapper
struct Clamped<Value: Comparable> {
  shared let storage: RangeStorage<Value>
  private var value: Value
  
  init(wrappedValue: Value, @shared: RangeStorage<Value>) {
    self.value = shared.clamp(wrappedValue)
  }
  
  var wrappedValue: Value {
    get { value }
    set {
      // `storage` is available like any other property
      value = storage.clamp(newValue) 
    }
  }
}

And at the point of use, the compiler will make sure RangeStorage is initialized once for each wrapper application and stored at a scope outside of the instances. Later on, when multiple instances of Hud are initialized, they'll all be given access to the same $shared properties. Suggestions on how to name the $shared property would be much appreciated. :slightly_smiling_face:

struct Hud {
  @Clamped(@shared: RangeStorage(0...14)) var bar = 5
  @Clamped(@shared: RangeStorage(1...9)) var foo = 1
}

var hud1 = Hud()
var hud2 = Hud()

// desugars to

struct Hud {
  static let bar$shared = RangeStorage(0...14)
  static let foo$shared = RangeStorage(1...9)
  
  var bar = Clamped(wrappedValue: 5, @shared: bar$shared)
  var foo = Clamped(wrappedValue: 1, @shared: foo$shared)
}

// both Hud's get access to the same $shared properties.
var hud1 = Hud() 
var hud2 = Hud()

Initialization

Inside the wrapper's initializer, assigning the shared value to the shared property is handled by the compiler, so there's no need to explicitly do it.

shared let storage: RangeStorage<Value>

init(wrappedValue: Value, @shared: RangeStorage<Value>) {
  self.value = shared.clamp(wrappedValue)
}

The initialization of the storage value itself follows the same principles as static variables: it can't instance variables or methods that depend on self being initialized. Though literals and other type variables can be used.

struct RangeStorage {
  init(_ range: String) { ... } 
}

struct Container { 
  @Clamped(@shared: RangeStorage(1...7)) var weekday = 3
}

// not okay
struct ContainerB {
  var minDay: Int
  var maxDay: Int
  @Clamped(@shared: RangeStorage(minDay...maxDay)) var weekday = 3
}

Property wrappers can be initialized in multiple ways (through a wrappedValue initializer, a projectedValue, or default inits). For property wrappers passed as function arguments, which initializer is called depends on the value that is passed to the function. For those reasons, property wrappers that declare a dependence of a shared storage will need to include it on all initializers.

@propertyWrapper
struct Wrapper<Value> {
  var wrappedValue: Value
  var projectedValue: Wrapper
  shared let storage: SomeStorage
  
  init(wrappedValue: Value, @shared: SomeStorage) { // }
  
  init(projectedValue: Wrapper, @shared: SomeStorage) { //	}
  
  init(@shared: SomeStorage = SomeStorage()) { // }
}

// ...

@Wrapper(@shared: SomeStorage()) var value = ""

The initialization of the shared storage must be resolved and stored at the call site. So providing a default value on the initializer argument is allowed but initializing it inside the wrapper declaration is not.

@propertyWrapper 
struct Wrapper {
  // ...
  shared let storage = SomeStorage() // * error
}

Since the goal of this feature is to allow instances of the type containing a property wrapped property to share the same storage instance, injecting the shared storage into the Container is also a violation.

class Container {
  @Wrapper var someProperty: String 
  
  // this way instances of `Container` could have different `storage` values 
  init(value: String, @shared storage: SomeStorage) {
    self._someProperty = Wrapper(wrappedValue: value, shared: storage)  // error
  }
}

Access control

The shared property is accessible anywhere in the Wrapper scope, like any other property. However, unlike other generated property wrapper properties, it's not directly visible to the container type. It can only be accessed through the backing storage property (unless it was declared private).

class Container { 
  // shared let someProperty$shared = SomeStorage("hi") 
  @Wrapper(@shared: SomeStorage("hi")) var someProperty = ""
  
  func accessStorage() {
    print(someProperty$shared) // not allowed
    print(_someProperty.storage) // okay
  }
}

Lifecycle

There are a few important aspects of the lifecycle of the shared storage. About its initialization, it happens only once. And then it's reused for subsequent instances that need it.

class Container { 
  @Wrapper(@shared: SomeStorage()) var someProperty = ""
}

let firstContainer = Container() // `shared let someProperty$shared` initialized

// the following `Container` instances use the `someProperty$shared` 
let secondContainer = Container()
let anotherContainer = Container()

The shared storage can be declared with classes, structs, and enums. Multiple containers will end up using the storage, potentially at the same time, so it should be read-only. Since the storage is immutable and Swift uses copy on write to avoid needlessly copying values, only one instance of the storage will be alive in the memory regardless of how many container instances use it.

The storage lifecycle is not tied to the "original" instance that caused it to be initialized in the first place. Instead, it follows the rules of other Type properties: it must be given a default value, and it is lazily initialized.

API-level Property Wrappers on function parameters

Implementing this feature also unlocks the possibility for API-level wrappers to pass arguments to the shared storage when passed as function parameters. And unlike the strategy mentioned in the Future Directions section of the SE-0293, using the shared storage won't require accessing wrapped and projected values through subscripts.

struct Layout {} 

struct SharedStorage { 
  let layout: Layout
  
  static func italic() -> Layout {}
} 

@propertyWrapper
struct Style {
  shared let storage: SharedStorage
  var wrappedValue: UIView 
  var projectedValue: Style { self }
  
  init(wrappedValue: UIView) {}
  
  init(projectedValue: Style, @shared: SharedStorage) { // }  
}

func emphasized(@Style(@shared: .italic()) label: UILabel) {}

Composition of Property Wrappers

When a property declares multiple property wrappers, they get composed and their effects are combined through a composition chain. For wrappers with a shared storage dependency, the same can be applied.

Take for example the following composition chain, where one of the wrappers has shared storage and the other does not.

@WithShared(@shared: .init()) @Without var foo: Bool = false

The composition chain will be resolved by nesting the inner wrapper into the outer wrapper type and initializing the shared property as needed. The same logic applies to the reversed order of application ( @Without @WithShared var foo).

shared let foo$shared = Shared()
var foo: WithShared<Without<Bool>> = WithShared(wrappedValue: Without(wrappedValue: false), foo$shared)

In the case of a property with multiple applications of the same wrapper with shared storage, the composition chain would be resolved in the same way. Each wrapper gets its own shared storage property regardless.

@WithShared(@shared: .init()) @WithShared(@shared: .init()) var foo: Bool = false 

shared let baz$shared = Shared()
shared let baz2$shared = Shared()

var baz: WithShared<WithShared<Bool>> = WithShared(wrappedValue: WithShared(wrappedValue: false, baz$shared), baz2$shared)

Impact on existing code

This is an additive feature, and it shouldn't impact existing source code.

Backward compatibility

However, from a library evolution standpoint, making an existing property wrapper opt-in into the shared storage model can be a non-resilient change for ABI-public property wrappers.

Consider a type that exposes a property with a property wrapper to its public API.

@propertyWrapper
public struct Wrapper<Value> {
  var wrappedValue: Value { ... }
  var projectedValue: Wrapper { ... } 
}

public struct Container {
  @Wrapper public var someValue: String
}

// -------
// the generated interface
public struct Container {
  public var someValue: String 
  public var $someValue: Wrapper 
}

@propertyWrapper
public struct Wrapper<Value> { ... }

Suppose that on a later version, the author of this property wrapper decides to change it by adding shared storage. Even if the shared storage is given a default argument in the property wrapper initializer, this is a non-resilient change. The same would be true for the opposite scenario: removing the shared storage from an ABI-public wrapper.

The example shows an API-level property wrapper, but the same would apply to an ABI-public implementation detail wrapper.

Alternatives considered

Static shared storage

Instead of introducing a new attribute, we could store the generated storage property as a normal static variable in its enclosing instance.

@propertyWrapper(shared: Storage)
struct Clamped {
  private var value: Value
  
  var wrappedValue: Value { fatalError("use the subscript!") }
  
  // wrappedValue would be accessed through a subscript
  subscript(shared storage: Storage) -> Value {
    get { value }
    set { value = storage.range.clamping(newValue) }
  }
  
  struct Storage {
    let range: ClosedRange<Value>
    init(range: ClosedRange<Value>) { // ... } 
  }
} 

// .... using it

struct Hud {
  @Clamped(range: 0...100) var value = 100
}

// Desugared version:
struct Hud {
  private var _value: Clamped<Int> = .init(wrappedValue: 100)
  private static let _value$shared: Clamped<Int>.Storage = .init(range: 0...100)
  var value: Int {
    get { _value[shared: Hud._value$shared] }
    set { _value[shared: Hud._value$shared] = newValue }
  }
}

Readability is one of the main disadvantages of this approach, as it would require passing the storage around through subscripts.

Related Work

The Future Directions sections on both SE-0258 and SE-0293, and the threads discussing this feature, especially this thread, were essential to this proposal.

13 Likes
2

Thank you for pushing this topic forward. I haven‘t read the proposal yet, but I plan to do that tomorrow or so. That said, I only quickly scanned the syntax parts and I think there will be some pushback because the proposal aims to introduce more exclusive syntax which will increase the cognitive load for every swift developer.

A few things at a quick glance:

  • Small nit-pick: shared is a property keyword, not an attribute in this proposal. An attribute starts with @.
  • I find it very confusing to see parameters / arguments / labels being merged with the attribute syntax (e.g. init(foo:@shared:)).
  • I will try to think more about the foo$bar syntax in this proposal. In the original PW threads I used $ as a chain replacement for the dot which would have introduced a different calling scope of some sort, but that ship sailed.

Again I only scanned the syntax parts and will try to share a more detailed opinion asap.

2 Likes
3

I don't think this feature can exist without new syntax. Earlier designs of this feature introduced new syntax in the form of a new property wrapper attribute and a new kind of wrapped/projected value subscript, and IIRC it also added some new "requirements" to the ad-hoc property wrapper protocol. In my opinion, this is more cognitive burden than a new kind of shared property.

Regardless, there needs to be some way to denote what the shared storage for the property wrapper is. Of course, bike-shedding is welcome :slight_smile:

FWIW the compiler implementation still classifies some of these keywords as "declaration attributes". I believe the right term here is "declaration modifier".

Yeah, I don't think it's necessary to write @shared: at the call-site. It should use the regular argument label, and the compiler will discover which argument initializes the shared storage through argument-to-parameter matching.

2 Likes
4

Thanks for working on this! This is an important missing part of the property wrappers design, and it'll unlock a lot of capabilities we've been waiting for to have this.

I honestly had the opposite reaction; when I looked at the subscript definition, I felt like I could work out what the parts all do based on how the types line up and what I already know about how subscripts and property wrappers work. With the new shared modifier, I had to read through the proposal to understand what that was trying to do and how it's different from static. That could be my own bias from being overly familiar with the implementation, but the subscript approach feels like a reasonably elegant way of expressing the idea using existing concepts (and I was pleasantly surprised by how compact you all managed to make the subscript example turn out—my own thoughts of how it would have had to look were far uglier!)

16 Likes
5

To be fair, I'm probably also biased because I very much dislike the existing subscript used for enclosing self access :slight_smile:

Part of the reason why I dislike the "shared storage as a subscript argument" approach is because it falls apart as soon as you want to do anything with the property wrapper aside from using the synthesized computed properties, e.g. passing the property wrapper as an argument to some function. The property wrapper on its own is useless, so you also need pass this shared storage along, and then accessing the wrapped value manually is pretty cumbersome. It's also pretty cumbersome to pass around the shared storage if you want to use it in helper methods inside the property wrapper implementation. Also, there's really nothing stopping the programmer from manually passing in some other random value of the same type as the shared storage instead of the one the property wrapper was declared with. I think fundamentally what the subscript approach is trying to model is a special kind of property.

5 Likes
6

I don’t really understand how shared works. What are its semantics? How does it work in a regular struct or class?

5 Likes
7

I'll admit I'm definitely more focused on the "using the synthesized computed properties" part and hadn't really thought about uses for the wrapper type beyond that. However, it seems to me that, in order for any wrapper type to be able to have access to its shared properties while passed around, that would mean that the wrapper type with shared storage has to carry around an extra, possibly-refcounted, pointer to the shared instance, and that would in turn make it impossible for a wrapper with shared properties to be fully "zero-cost" in per-instance storage. However, with the subscript approach, I think you could allow wrappers to opt into this when they need it, by having them capture the shared state as part of their initialization, e.g.:

@propertyWrapper(shared: Storage)
struct Clamped {
  private var value: Value
  private var shared: Box<Storage>

  init(shared: Storage) { self.shared = Box(value: shared) }
  
  // Since we captured the shared storage during initialization,
  // we can use the simpler wrappedValue interface for defining
  // the property behavior
  var wrappedValue: Value {
    set { value = shared.value.range.clamping(newValue)
  }
}

so you could build the shared behavior into your wrapper when you want. Maybe it turns out that most wrappers with shared storage want to have that inline backreference, and in that case, I could see why it's valuable to streamline it, but I think there are at least some wrapper implementation that could utilize shared storage but for which the added per-instance overhead of tracking that shared storage wouldn't be tolerable, such as if we were to supersede lazy with a wrapper, since lazy today does not pay that cost.

4 Likes
8

Have generic value parameters been considered as an alternative?

6 Likes
9

My understanding is that that subscript is not an official thing. It's not mentioned in the language guide, and the proposal only describes it under "future directions".

1 Like
10

Is there anything about the compiler generated shared storage that limits it to property wrappers? Sounds like it could be a useful feature for other types. For example, I could use a shared downloaded images cache across views.

1 Like
11

Implementation-wise, @John_McCall had a neat idea to break the property wrapper into two different types -- one with the instance storage and a "composite" type with the instance and shared storage -- and only piece together the composite type when needed, so most of the time you don't need to pay the cost of that pointer.

This might be a good alternative solution for the Clamped case shown in the pitch, but not for other cases where the shared value isn't a compile-time constant. This was discussed a bit on the other thread that is linked in the pitch:

12

That's right, and part of the reason why it's not an official thing yet is because there may be a better design out there.

13

I wonder how this would affect unit tests for a type that uses property wrappers with shared storage. How could the developer control side effects of other tests the type is involved in or other environmental interference?

1 Like
14

Right now it's being proposed for the property wrappers only, but it could be a future direction to expand the scope of use if we have use cases for it.

Swift allows folks to learn the language features as they need to use it to solve a problem (or by curiosity) so I disagree that it's a burden to every developer.

You can refer to the desugared example somewhere in the middle of the pitch to have an idea, but I think of it as "a static variable on the outer type containing the wrapper" -- which makes it shared for the wrapper instances. And you can use structs, classes, enums as the Storage type. However, the use of shared is pitched for property wrappers only.

3 Likes
15

Oh, thanks for asking this! Since shared properties follow the same rules as static properties, it has the potential to side effects. It can be tricky since the shared behavior is desired for the feature but it can get in the way of testing. I think a few aspects attenuate this though:

  1. The storage is immutable. So it prevents the scenario of one test case mutating it and catching other tests off guard because of the order they were executed.
  2. The storage is only accessible through the wrapper type, so when testing the Wrapper itself you could still inject different values and test it out.
16

I think it would be useful for testing - as well as other purposes - to be able to define a context or scope for the shared storage, somewhat like the environment concept in SwiftUI. That of course raises questions about how the scope is defined and accessed, and I'm really not sure what the answer should be, but I just wanted to put that out there.

2 Likes
17

Does this mean the following code is an error, since Swift doesn't have function-local static variables?

func f(_ lo: Int, _ hi: Int) {
    @Clamped(@shared: RangeStorage(lo...hi)) var x = 0
}
2 Likes
18

Right. The idea is that you write this:

@propertyWrapper
struct Clamped<Value: Comparable> {
  @shared private var bounds: Range<Value>
  private var value: Value

  init(wrappedValue: Value, @shared _: Range<Value>) {
    // There must be a @shared initializer that's at least as
    // accessible as this which has exactly the same parameters
    // as this initializer's @shared parameters.
    // This body can refer to @shared properties, but not directly
    // to its @shared parameters.
    ...
  }

  @shared init(_ bounds: Range<Value>) {
    // This body must initialize the @shared stored properties
    // and cannot use any non-@shared members.
    self.bounds = bounds
  }

  var wrappedValue: Value {
    get { value }
    set { value = bounds.clamping(newValue) }
  }
}

That definition gets rewritten/lowered to something functionally like this:

extension Clamped {
  struct Shared {
    // @shared instance properties from the original type go here.
    private var bounds: Range<Value>

    // @shared initializers from the original type go here.
    init(_ bounds: Range<Value>) {
      self.bounds = bounds
    }
  }

  struct Instance {
    // Non-@shared instance properties from the original type go here.
    private var value: Value

    // Non-@shared initializers from the original type go here.
    // @shared parameters are removed and replaced with an
    // implicit _shared parameter.  References to @shared
    // properties in the body are rewritten to use the implicit
    // _shared parameter.
    init(wrappedValue: Value, _shared: UnsafePointer<Shared>) { ... }

    // No other methods go here.
  }

  struct Composite {
    // Static stored properties from the original type go here.
    // The instance properties are always exactly as follows:
    var _shared: UnsafePointer<Shared>
    var _instance: UnsafeMutablePointer<Instance>

    // Methods from the original type go here.  References to
    // shared properties are rewritten to use _shared.
    // References to instance properties are rewritten to use
    // _instance.
    var wrappedValue: Value {
      get { _instance.pointee.value }
      set { _instance.pointee.value = _shared.pointee.bounds.clamping(newValue) }
    }
  }
}

Then if you have a use site:

@Clamped(1..<1024) var count: Int

This would get lowered like so:

static var count$shared = Clamped<Int>.Shared(1..<1024)
var count$instance: Clamped<Int>.Instance
var count: Clamped<Int>.Composite {
  _read {
    yield Clamped.Composite(_shared: &count$shared,
                            _instance: &count$instance)
  }
  _modify {
    var temp = Clamped.Composite(_shared: &count$shared,
                                 _instance: &count$instance)
    yield &temp
  }
}
1 Like
19

That feels like a pretty elaborate transformation to me. And if there's an ABI boundary at the wrapper implementation, how would we avoid composing and decomposing the aggregate form? Although that may avoid the instance storage, it might introduce a lot of unnecessary copying and moving of the wrapper data around every access if it can't be optimized through.

20

It's definitely an elaborate transformation. Some of that complexity is inevitable in any approach; for example, you have to figure out what arguments end up in shared storage and how to initialize that storage. The transformation approach at least has fairly straightforward resilience properties, avoids the need for widespread implicit arguments, and works with composition.

I'm not sure what you're saying about unnecessary copying and moving; all accesses are in place. On the other hand, the composite type probably needs to be move-only for safety. (This doesn't infect the containing type, which just stores the Instance storage.)

A small amount of indirection could be eliminated by making the composite type @frozen, or baking knowledge of it more generally into the ABI so that non-mutating operations took borrowed values to the two self components and mutating operations took a borrowed/inout pair.

1 Like