Compositional Initialization

Evolution Pitches
1

Compositional Initalization

Introduction

This proposal introduces an opt-in protocol, PropertyInitializable, which provides two new init methods:

  • failable init from a collections of extensible, typesafe keypath-value Property objects
  • non-failable init to clone from another instance, mutating select properties

The name “compositional init” means that this proposal allows the state of an object or struct to be assembled compositionally from sets of properties (including another instance). Compositional initialization allows mutations to be encapsulated in a clear, type-safe way.

This proposal addresses the problems that motivated Matthew Johnson's excellent proposal, SE-0018, but in a different way. Hopefully compositional init can serve as the implementation of SE-0018, which unfortunately got tabled due lack of ABI impact.

I initially wrote this proposal in 2018 based on Swift 4. I have reviewed the changes and proposals since then and it does not seem like there has been anything that would make this proposal unnecessary, please correct me however if I missed something.

From that review of past proposals, I find that this proposal may also address the desires expressed in the Swift Evolution discussion thread "Record initialization and destructuring syntax". I do not see any follow-up proposal for that, so hopefully this might help with that too.

I have a mostly working implementation made in the Swift 5 Playground here.

Motivation

Immutability carries many benefits including thread safety, simplification of state management, and improved code comprehensibility. It’s what makes functional programming so functional, which helps create unit-testable code.

However, in Swift 5, the benefits of immutability include neither:

(1) - "ease of making a copy that differs from the original in a subset of its properties — without lots of boilerplate,” nor (more generally),

(2) - "initializing an object from a collection or set of collections of its properties in one line of code."

The desire for no boilerplate and greater flexibility when initializing and cloning immutable types motivated this proposal.

Use Cases

A growing movement exists to use immutable models to handle data responses from the web in Swift applications.

This proposal was inspired by mocking a web service in Swift, where we had to simulate the server’s responses by changing a few properties of an immutable data model object from a canonical example. However, in addition to satisfying that one need, I found that this style of initialization allows for many other useful patterns that simplify code and improve clarity.

Detail of the Motivation

Chris Lattner’s SE-0018 proposal notes that in current solutions for (1), “initialization scales with M x N complexity (M members, N initializers).” I call this the “boilerplate cost” — measured on the “big B” scale. Indeed, one advised workaround for (1) involves going from B(M*N) down to B(2M), where M is the number of different properties to be supported for mutation during cloning.

Compositional Init, on the other hand, provides B(0) for copying an immutable object while changing some arbitrary selection of its properties, and also provides a B(0) solution for problem (2).

How Other Languages Have Solved (1)

Other languages have good solutions for (1) that are similar to compositional init, but none seem to solve (2) the way CI can.

Some examples of static, typesafe, functional languages that allow the initialization of a new instance via cloning with property overrides:

Dynamic languages have been slower to embrace immutability, but support for this concept is growing:

Proposed solution

Compositional init solves both (1) and (2) by adding simple, clear, Swifty syntax for the initializing an immutable instance from a set of typesafe properties and an optional clone argument. Because this proposal is based purely upon Swift 4/5’s wonderful KeyPath and Mirror types, we get all the type safety and access restriction guarantees that they already carry.

Traditional memberwise init:

let fool: Foo = Foo(bar: “one”, baz: 1.0, quux: nil)

Compositional init cloning fool and mutating its quux property:

let food: Foo = Foo(clone: fool, mutating: \.quux => 42)

Compositional init failably initializing foom from an array of properties:

let properties: [PartialProperty<Foo>] = 
[
   \.bar  => “two”, 
   \.baz  =>  2.0, 
   \.quux =>  nil
]

let foom: Foo? = Foo(properties)

Compositional init failably initializing foom from variadic property arguments:

let foom: Foo? = Foo(\.bar => “two”, \.baz => 2.0, \.quux => nil)

As a result of being based on WritableKeyPath<Root, Value>, the declaration \Foo.bar => “two” will fail to compile if the property Foo.bar is any of the following:

  • not accessible in the current scope
  • not writable in the current scope
  • not the same type as the value being paired with it
  • non-existent

Detailed design

This proposal introduces the following protocols:

  • AnyPropertyProtocol
  • PartialPropertyProtocol
  • PropertyProtocol
  • PropertyInitializable

Accompanying these, we introduce implementations:

  • AnyProperty
  • PartialProperty
  • Property<Root, Value>

This proposal introduces the “partially type-erasing lazy assignment operator” =>, which returns a PartialProperty<Root, Value> from a WritableKeyPath<Root,Value> on the left side, and on the right, an @autoclosure @escaping that can accept either:

  • a Value object, or
  • a function returning Value that will be lazily executed only at init.

Source compatibility

Aside from any naming collisions (sorry), this proposal should have zero effect on source code compatibility.

Effect on ABI stability

The initial PR for this proposal should not impact ABI stability, as far as I can tell.

Effect on API resilience

Compositional init should play nice, but I will leave it to the experts.

Alternatives considered

One alternative is to simply avoid immutable “set once at init” style properties, and instead use var for any properties you might need to change. The pattern is then to “immutable-ize” the root type by using let at instantiation, as in:

struct Foo {
    var bar: String 
    var baz: Double
    var quux: Int?
}

let fool = Foo(bar: “one”, baz: 1.0, quux: nil) // hah! can’t change me now!

var food = fool 
food.quux = 42
let foom = food // immutable once again.

That workaround is not great because there is nothing to prevent mistakes or abuses.

As well there was discussion on the aforementioned thread on destructuring, to which you may refer.

Discussion

I created a mostly working playground implementation of this under Swift 5, which can be found on github here.

There are some limitations in the way Swift 5 handles WritableKeyPath. They cannot be used to write to a property before it is initialized, sadly. If they could be, that would really improve the flexibility of how this can be used. As it stands, this limitation means that any variable which you want to be able to initialize from a keypath-value pair (a Property) must be a var and must have a default value or an initializer that gives it one.

Ideally, WritableKeyPaths could be used to initialize let variable or a priave(set) var, since the type adopting PropertyInitializable inherits an init method that, it stands to reason, should have access to do so. However, the way the compiler decides if a keypath can be considered "Writable" does not consider which scope the keypath will actually be used in when the writing finally takes place, or when it will take place (e.g. at init time, at which point a let variable could still be writable, technically speaking).

Any tips from the compiler gurus on how this could be enhanced would be appreciated. Perhaps we would need something like an "InitializableKeyPath" indicating that the variable can be set only at init time, by an init function within the type, using such a KeyPath. (Note: WritableKeyPath did work with an uninitialized private (set) var in Swift 4, but not in Swift 5.)

However if WritableKeyPaths to let variables cannot feasibly be used even at init time, then we could consider that since compositional init is opt-in at the type level anyway, we might as well consider it opt-in at the property level, too, by making it a var.

I look forward to everyone's thoughts on this. Thanks

7 Likes
2

Just FYI, this proposal was written by @anandabits. Chris was the review manager.

1 Like
3

Ah, thank you, I'll update the main post.

4 
struct Foo {
  let bar: String 
  let baz: Double
  let quux: Int?
} // hah! can't change me now!

let fool = Foo(bar: "none", baz: 1.0, quux: nil)
let foom = Foo(bar: fool.bar, baz: fool.baz, quux: 42) // immutable once again.

Personally I think that having let properties in structs might be necessary only when the type doesn't follow value semantics, as you might want to force changes to go through the init again, or other mutating methods.

I think that the highlighted use-cases can be mostly solved with the famous extension Any { func with ... and, if even needed, some syntax for easier currying (for the inits here)

5

You can get pretty close to the stated goals with just a global function:

func with<T>(_ value: T, update: (inout T) -> Void) -> T {
  var newValue = value
  update(&newValue)
  return newValue
}

let fool: Foo = Foo(bar: “one”, baz: 1.0, quux: nil)

let food = with(fool) { $0.quux = 42 }

Value semantics gives you all the same stated goals of immutability while still allowing for efficient local state updates.

16 Likes
6

Well, the global function is very nice, however it does not allow for the core idea of this proposal, which is initialization via an array composed of properties, e.g.:

let properties: [PartialProperty<Foo>] = 
[
   \.bar  => “two”, 
   \.baz  =>  2.0, 
   \.quux =>  nil
]

let foom: Foo? = Foo(properties)

Being able to assemble a set of properties for an object in a type-safe way, then being able to use that set of properties to create a new instance or clone an existing instance, is the key idea here. To me it seemed like a nice evolution from keypaths. Whether that's seen as worthy of inclusion to the language, that's the question.

7

Having a way to assemble a value from a collection of keypath-value pairs would be interesting, I agree. I think that belongs in the same general bucket of reflection mechanisms as things like StoredPropertyIterable.

4 Likes
8

@Joe_Groff wrote:

Having a way to assemble a value from a collection of keypath-value pairs would be interesting, I agree. I think that belongs in the same general bucket of reflection mechanisms as things like StoredPropertyIterable .

You're right, that proposal is definitely in the same "reflection" bucket. My playground implementation linked above relies on the reflection features from Swift 4, but having some enhanced capabilities like those mentioned in StoredPropertyIterable would be great.

9

Another option that (I think?) accomplishes the same goal would be a generated clone function with the current properties as default parameters. Something similar to how you do a copy in Kotlin

e.g. in Swift it could look like this:

struct ClonableType: Clonable {
    var aString: String
    var anInt: Int

    // could/should be generated
    func clone(aString: String = aString, anInt: Int = anInt) -> Self {
        return Self.init(aString: aString, anInt: anInt)
    }
}

let initial = ClonableType(aString: "Indexed", anInt: 0)
let second = initial.clone(anInt: 1)

This feels much more natural to me as it mirrors the init rather than introducing something new, and skips the extra steps necessary for Key Paths.. It also would follow the existing rules for initializers, which means it could also be added to classes without avoiding expected init behavior.

There are of course several issues, the biggest of which is that clone function isn't currently possible in Swift since default parameters have to be defined at compile time, though my guess is this specific case wouldn't violate the likely reasons for that restriction. It's also unclear what should be generated when there are multiple initializers.

6 Likes
10

In case anyone is interested, the latest draft of my thinking on Memberwise initialization can be found here: Explicit Memberwise Initializers.

11

Not taking away from the content of your proposal, but the Elm example should fall under "static, typesafe, functional languages", not "Dynamic languages".

2 Likes
12

Fixed, thanks.

13

That's a fine approach to cloning.

However, I'm not offering this proposal as an alternative to something like that, nor is this meant as an alternative to the great Explicit Memberwise Initializers proposal of Matthew Johnson.

Rather, the point of Compositional Initialization is to allow an object to be composed before it is initialized from a set of type-safe keypath-value pairs.

In the current form of this proposal, these pairings take the form of Property<Root,Value> objects.

You can look at this proposal as building upon the foundation laid by KeyPath.

See, in Swift 4 we got KeyPath, but we did not get any easy way to use a KeyPath to initialize an object or to make a clone mutated at some property—nor did we get a type like Property allowing the developer to pair a KeyPath with a Value.

Given that it seems natural to pair KeyPaths and Values, once you have a set of these pairs, it seems natural then to make an object from them.

So this proposal offers a simple way to do that.

Here is a working example from the Swift 5 Playground linked in the top post:

struct Customer: PropertyInitializable {    
    var name: String = ""
    var zipcode: Int = 0
    var addressLine1: String = ""
    var addressLine2: String = ""
    var addressLine3: String = ""
    
    /// Necessary because of the fact WritableKeyPath cannot currently be used
    /// to actually initialize a variable. It must have a value first. 
    internal static var _blank: Customer {
        get { return Customer() }
    }
}

let dataFromTestWebResponse = [PartialProperty<Customer>]() 
    + (\.name, "Steve Jobs") 
    + (\.zipcode, 97202) 
    + (\.addressLine1, "Reed College")
    + (\.addressLine2, "3203 SE Woodstock Blvd")
    + (\.addressLine3, "Box #121")

if let customer: Customer = Customer(dataFromTestWebResponse) {
    print(customer)
} else {
    print("Data is missing.")
}

As this example shows, we can create a type-safe array of properties for an object using nothing but tuples. (This relies on a custom infix function + defined on an extension to Array; please see the linked Playground page for the full source.) We can then try to initialize the object from that array.

The failable init method is defined in PropertyInitializable protocol as:

init?(_ properties: [PartialProperty<Self>])

The elements of the dataFromTestWebResponse array are of the type PartialProperty<Customer>.

At first glance, that array might not seem to be fully type-safe.

However, the underlying objects are type-safe Property<Customer,Value> objects that have been partially type-erased so they can exist within the same array. This is necessary because they differ as to the type of Value—zipcode is an Int but the rest are String. (Note: this naming convention follows exactly from Swift's existing PartialKeyPath and KeyPath.)

In this way, you can "compose" the properties of an object in a completely type-safe way prior to initialization, and then initialize the object from those properties with minimal boilerplate.

For example we might do:

let reedieData = [PartialProperty<Customer>]()
    + (\.zipcode, 97202) 
    + (\.addressLine1, "Reed College")
    + (\.addressLine2, "3203 SE Woodstock Blvd")

let studentBoxAssignments: [(String, Int)] = [("Steve Jobs", 121), ("The Woz", 314)]

var customers = [Customer]()

for (name, box) in studentBoxAssignments {
    let data = reedieData + (\.name, name) + (\.addressLine3, "Box #\(box)")
    if let customer: Customer = Customer(data) {
        customers += [customer]
    } else {
        print("Data is missing.")
    }
}

print(customers)

By saying this is completely type-safe, I mean, if we put "97202" as a string then we'll get a compiler error, since the proper type is inferred from the KeyPath.

Now, let me address the reasoning behind the => syntax, from the example in the original post:

let properties: [PartialProperty<Foo>] = 
[
   \.bar  => “two”, 
   \.baz  =>  2.0, 
   \.quux =>  nil
]

Frankly, this is just a simple alternative to using the tuples syntax from the examples above. By defining => as a custom infix operator, we can do away with the parentheses around each key-value pair.

If => seems un-Swifty, I'd love to hear a better suggestion. Swift's custom operator syntax does not permit the use of : when defining a custom infix operator here; that would make this a Dictionary.

Speaking of Dictionaries, I originally tried using Dictionary instead of Array, since it seemed like a more natural place to store keypath-value pairs. However, I found using a Dictionary for this purpose to be frought with problems. Let me know if you want the details on why that did not seem feasible.

1 Like
14

One unfortunate side-effect of the proposed design is that by partially type-erasing the property values in your wrappers, and then incurring a heap allocation by storing them in an array, the creation of this set of properties is probably going to have significantly worse performance than would simply creating the instance of the type itself, at least for many value types.

Bouncing off from @Joe_Groff's idea above, could something like this be achieved by sequencing closures with an inout argument instead? Instead of inventing entirely new concepts around keypath/value pairs, your example above could also be expressed as:

let propertySetter = { (inout Foo) -> () in
  $0.bar = "two"
  $0.baz = 2.0
  $0.quux = nil
}

And the addition of property arrays you were doing could be achieved by applying multiple closures in sequence. This would still allow you to "compose an object before it is initialized"; it's just composed as a sequence of closures instead of as "data". But I think that's more natural Swift. And since there has already been some work done to bridge keypaths and closures, perhaps that could even be extended further to make this more elegant.

1 Like
15

Could PropertyInitializable.init(_: ) be generalized to take a Sequence of PartialProperty?

16

The issue with this is that to use these closures you need an already existing Foo.
If there were something like an init modifier for arguments that says that it’s not an initialised object, so you can only set properties but not read them, then it would work. But you’d still have the issue that — since it’s composable — you don’t have a good way to know when an object is completely initialised. Or even worse with classes to know if super.init was called...
And it also can probably live only in the same module as the type unless you call another init of the type

1 Like
17

I'm aware of that, and it's a limitation of the original poster's own proposal, too:

I'm not suggesting that closures alone would solve the problem of compositional initialization, but nor does an array of keypath/value pairs, because the latter is just a more restricted form of the former. My only point is that if that problem is going to be tackled, I think a solution that composes closures that somehow can operate on a partially-initialized value is a better approach than creating entirely new concepts and collections of type-erased keypath/value pairs.

2 Likes
19

I missed that bit, didn’t know that you can’t use key paths for initialising! Though that could be something that changes as part of this proposal, or a spin-off of it

20

Unless I’m missing something, this sounds rather like an [unconventional] implementation of the builder pattern. The traditional ‘downside’ of builders is that they’re not built into the language - it’s just an additional class you manually define. That can be a lot of tedious work, though. Optimising away that manual labour is appealing. But maybe there’s a more direct if ‘boring’ way to do so… in the spirit of just exploring alternatives for due diligence if nothing else, how about something like a Buildable protocol which can be used like:

struct Example: Buildable {
  let foo: String
  let bar: Int = 0
  let baz: URL?
}

// The above, through the magic of compiler-synthesised methods, becomes essentially:

struct Example: Buildable {
  let foo: String
  let bar: Int = 0
  let baz: URL?

  func asTemplate() -> Builder {
    return Builder(basedOn: self)
  }

  struct Builder {  // Or ‘Template’, maybe?
    var foo: String
    var bar: Int
    var baz: URL?

    internal init(basedOn template: Example) {
      self.foo = template.foo
      self.bar = template.bar
      self.baz = template.baz
    }

    func setFoo(foo: String) -> Self {
       self.foo = foo
       return self
    }

    … etc …

    func init() -> Example {
      return Example(foo: self.foo, bar: self.bar, baz: self.baz)
    }
  }
}

// In use:
let standard: Example = …
let variant = standard.asTemplate().setBar(11).init()

It’s possible to expand on that to also support the traditional build-from-scratch builder use (though I’m not sure how one can write it in a way that avoids the annoying need to use try somewhere during construction, within the capabilities of Swift 5.1).

I’m not saying it’s better, just throwing it out there as an alternative approach.

It does seem to me that this would provide essentially the same functionality - the difference seems to be essentially just the subjective aspects - aesthetics, ergonomics, etc. It seems like it’d make simple cases a bit more verbose, but for complex cases - e.g. when you’re performing the build across significant code and/or time - it seems like it’d be equivalent in brevity.

It has the benefit of being more familiar to a large number of people (e.g. those coming from Java, C++, etc) and of relying less on complex language features (e.g. KeyPaths).

While it’s merely as type-safe as this pitch’s approach, it is IMO a bit safer in practical terms because you don’t have KeyPaths floating around detached from things.

Re. mutability, it avoids any irony from otherwise having to keep around a potentially mutable ‘template’ instance, since you’d only need to keep around the Builder instance itself, and as a distinct type it can’t be accidentally used by code unrelated to the building process itself.

…

That all said, I’m not sure any of this is the right approach… for value types, the existing var and let work just fine IMHO for ensuring that your immutable values truly are immutable (i.e. no-one else can have a var reference to them behind your back - that’s the whole point of value types). And I assume the compiler makes the make-mutable-copy-mutate-make-immutable-copy into the optimal, literally just a copy & subsequent assignments under the cover.

For reference types, I wonder if (a) this all wouldn’t be better addressed through something like Rust-style exclusive mutability control, to get essentially the same guarantees as for value types, but also (b) whether you really can have a generic way to ‘clone’ objects like this, given the potentially complex semantics of reference types. e.g. not every element of an object’s state is necessarily exposed to the type’s users, so would presumably have to be implicitly cloned exactly, but reference types frequently can’t support that because they can have uniqueness constraints on some or all that internal state… thus why I feel that, if nothing else, this functionality has to be both opt-in on a per-type basis and overriddable if the default implementation is insufficient.

2 Likes
21

Nitpick, albeit a worthy one IMO: => looks like a rightward-pointing arrow, so it’s going the wrong direction, since assignment is from rvalues to lvalues. <= would be the correct direction, but is of course already taken for the ‘less than or equal to’ operator. So you’d have to consider variants like <- or <~ etc, but probably no matter what you pick you’ll earn the wrath of at least some library authors that have already claimed those character sequences for their own custom operators. :stuck_out_tongue:

1 Like