Compositional Initialization

[mpangburn (Michael Pangburn)]

A generic Partial type is readily possible by erasing assignments with concrete types into inout closures, as @allevato describes above. Key path member lookup makes this abstraction syntactically nice:

@dynamicMemberLookup
struct Partial<Whole> {
    private var assignments: [PartialKeyPath<Whole>: (value: Any, update: (inout Whole) -> Void)] = [:]

    subscript<Value>(dynamicMember keyPath: WritableKeyPath<Whole, Value>) -> Value? {
        get {
            assignments[keyPath]?.value as? Value
        }
        set {
            guard let newValue = newValue else {
                assignments.removeValue(forKey: keyPath)
                return
            }
            assignments[keyPath] = (value: newValue, update: { whole in whole[keyPath: keyPath] = newValue })
        }
    }

    func applied(to initial: Whole) -> Whole {
        assignments.values.lazy
            .map { value, update in update }
            .reduce(into: initial) { whole, update in update(&whole) }
    }
}

The Partial type above encodes pieces of an arbitrary instance of a type, which can be accumulated incrementally in the absence of a concrete instance. These changes can then be applied at will:

struct Person {
    var name: String
    var age: Int
}

var anArbitraryTwentyOneYearOld = Partial<Person>()
anArbitraryTwentyOneYearOld.age = 21

let aRealTwentyYearOld = Person(name: "Zoro", age: 20)
let aRealTwentyOneYearOld = anArbitraryTwentyOneYearOld.applied(to: aRealTwentyYearOld)
// aRealTwentyOneYearOld == Person(name: "Zoro", age: 21)

[allevato (Tony Allevato)]

That's a really cool application of dynamic keypath member lookup to build that up!

[karim (Karim Nassar)]

That's really beautiful. Adding a quick protocol on top of your Partial nets us pretty much the syntax suggested by @allevato:

protocol CompositionalInitialization {
}

extension CompositionalInitialization {

    func compositeInit(_ composition: (inout Partial<Self>) -> Void) -> Self {
        var partial = Partial<Self>()
        composition(&partial)
        return partial.applied(to: self)
    }

}

extension Person: CompositionalInitialization {}

let zoroNextYear = aRealTwentyOneYearOld.compositeInit { $0.age = 22 }

print(zoroNextYear.name) // "Zoro"
print(zoroNextYear.age)  // 22

[jawbroken]

@vitamin

However, in Swift 5, the benefits of immutability include neither:

(1) - "ease of making a copy that differs from the original in a subset of its properties — without lots of boilerplate,” nor (more generally),

(2) - "initializing an object from a collection or set of collections of its properties in one line of code."

The answer to (1) for most people is to just make a mutable struct. Immutability is valuable for reference types but there is not much point in making almost any value type immutable. There is semantically no difference between mutating a value type directly and making a copy with a subset of properties changed. And as for immutable reference types, in Swift these should almost always be value types instead. So when I see examples like:

@vitamin

One alternative is to simply avoid immutable “set once at init” style properties, and instead use var for any properties you might need to change. The pattern is then to “immutable-ize” the root type by using let at instantiation, as in:

struct Foo {
    var bar: String 
    var baz: Double
    var quux: Int?
}

let fool = Foo(bar: “one”, baz: 1.0, quux: nil) // hah! can’t change me now!

var food = fool 
food.quux = 42
let foom = food // immutable once again.

That workaround is not great because there is nothing to prevent mistakes or abuses.

I don't really understand what the possible mistake or abuse is, or how changing this example to make a copy of fool with quux set to 42 would help avoid it.

[gahms (Nicolai Henriksen)]

I am not sure if I misunderstand you @jawbroken but we often have all our struct members to be let to ensure all members are initialised during instantiation.

For example:

struct Foo {
  let bar: String
  let baz: String?
  let quux: String
}

will ensure that I cannot instantiate Foo without providing a value for all 3 members.

The following will not compile:

let foo = Foo(bar: "bar", quux: "quux")

but with var members

struct Foo {
  var bar: String
  var baz: String?
  var quux: String
}

the above will compile.

With the proposed implementations/workarounds we cannot use let as they require var members to work.

[jawbroken]

@gahms

I am not sure if I misunderstand you @jawbroken but we often have all our struct members to be let to ensure all members are initialised during instantiation.

This is a very blunt hammer to apply to work around the unfortunate default initialisation of optionals to .none. You can instead write an initialiser that takes all three parameters, which will disable the default member-wise initialiser.

Edit: And I suppose the ship has sailed on this now, but I'm surprised that works in the first place. It seems to me it would have been nicer if the synthesised initialiser would only have a default value of nil if the variable declaration was explicitly var middleName: String? = nil, rather than implicitly.

[gahms (Nicolai Henriksen)]

@jawbroken

This is a very blunt hammer to apply to work around the unfortunate default initialisation of optionals to .none . You can instead write an initialiser that takes all three parameters, which will disable the default member-wise initialiser.

Yes I could write initialisers myself but then I loose the compile time check that everything is initialised. If somebody add a new property and forget to change the initialiser it will compile and run but without forcing initialisation of the new property.

[DeFrenZ (Davide De Franceschi)]

@gahms

If somebody add a new property and forget to change the initialiser it will compile and run but without forcing initialisation of the new property.

Only if the new property has a default value, so everything is initialised either way.

[gahms (Nicolai Henriksen)]

@DeFrenZ

Only if the new property has a default value, so everything is initialised either way.

...or is Optional which has a default value by design apparently – as you pointed out yourself.

If we never use optionals then I could hand-code all the initialisers, yes. The convenience of the syntesized initialiers is lost then.

I am arguing that it would be really nice to have a language feature that would allow for initialising a copy of a struct with a few properties changed and still be allowed to use let.

[DeFrenZ (Davide De Franceschi)]

@gahms

I am arguing that it would be really nice to have a language feature that would allow for initialising a copy of a struct with a few properties changed and still be allowed to use let .

But this is exactly what a var is :) Your issue doesn't seem to be with the let vs var but with the automatically generated init which now handles defaulted properties.
I think that something that could solve your problem is to make a way to have Optional vars to not be defaulted to .none somehow, but I'm not sure which would be a nice way to do it...

[jawbroken]

@gahms

Yes I could write initialisers myself but then I loose the compile time check that everything is initialised. If somebody add a new property and forget to change the initialiser it will compile and run but without forcing initialisation of the new property.

Sure, but you can lose this anyway if anyone adds an initialiser (e.g. to maintain source compatibility with call sites after they've added a new property). It's hard for the language to maintain any guarantees within a module where anyone can edit any code, and the synthesised initialiser isn't public so it is no help across modules. And I guess I also don't see how this is a very valuable invariant anyway. People can freely explicitly initialise the optional property to nil using the default initialiser, so it's hard to see it causing much chaos if they implicitly do that. And if nil is significantly dangerous, such that they have to be wary of accidentally using it, then perhaps the instance variable should be non-optional or use a custom enum with suitably scary case names.

[gahms (Nicolai Henriksen)]

@jawbroken

Sure, but you can lose this anyway if anyone adds an initialiser (e.g. to maintain source compatibility with call sites after they've added a new property). It's hard for the language to maintain any guarantees within a module where anyone can edit any code, and the synthesised initialiser isn't public so it is no help across modules. And I guess I also don't see how this is a very valuable invariant anyway. People can freely explicitly initialise the optional property to nil using the default initialiser, so it's hard to see it causing much chaos if they implicitly do that. And if nil is significantly dangerous, such that they have to be wary of accidentally using it, then perhaps the instance variable should be non-optional or use a custom enum with suitably scary case names.

I guess we live in different worlds I see your point but the problems I face are different.

We never use public for anything. We make apps and everything lives in the same module. We never just add an initialiser for compability. We want compile errors and then fix at the call site to be sure the new property is delibrately taken into account, with nil or with a value. It is not a matter of wether nil is dangerous or not. It is a matter of being deliberate about the initialised value. Same way I do not want an Int to default to 0, I want to deliberately assign 0 if that is the correct initial value.

This provides high confidence in refactorings. Swift generally provides mechanisms that provide a lot of help when we do refactorings. Often you can just make the change and look at the compile errors and when they are fixed you are done.

[vitamin (Jon Gilbert)]

@allevato

One unfortunate side-effect of the proposed design is that by partially type-erasing the property values in your wrappers, and then incurring a heap allocation by storing them in an array, the creation of this set of properties is probably going to have significantly worse performance than would simply creating the instance of the type itself, at least for many value types.

To be fair, the proposal follows exactly the type erasure used in the KeyPath types, introduced in Swift 4, namely PartialKeyPath and AnyKeyPath.

More importantly though, this proposal is not about replacing the standard way of initializing objects directly. Rather, this proposal is about replacing less efficient or flexible or safe ways of representing sets of data that will be used to initialize an object, particularly when writing tests.

Whether the performance profile is detrimental or an improvement, depends on what code you can replace. If you can replace a bunch of test code that relied on parsing lots of JSON files, this represents a performance improvement. If this lets you establish a set of properties once, and then reuse them via composition, you could see a performance improvement.

Bouncing off from @Joe_Groff's idea above, could something like this be achieved by sequencing closures with an inout argument instead? Instead of inventing entirely new concepts around keypath/value pairs, your example above could also be expressed as:

let propertySetter = { (inout Foo) -> () in
  $0.bar = "two"
  $0.baz = 2.0
  $0.quux = nil
}

And the addition of property arrays you were doing could be achieved by applying multiple closures in sequence. This would still allow you to "compose an object before it is initialized"; it's just composed as a sequence of closures instead of as "data". But I think that's more natural Swift. And since there has already been some work done to bridge keypaths and closures, perhaps that could even be extended further to make this more elegant.

Great question. While I like the use of chain-able propertySetter closures, and while that might even be a better approach in some situations, I think it's missing some key functionality that this proposal suggests we could add to Swift (as Davide De Franceschi pointed out a few days ago).

Let me go over those key features below. I hope you will correct me on this, if I am overlooking something from your suggestion. I'm very grateful to everyone who has taken the time to read and comment on this.

1. A closure (e.g. propertySetter above) can't be used to failably initialize an object in a way that depends on reflection. (Correct me if I'm wrong please, but I'm not sure it can.)

Given a closure or chain of closures that takes an object as input, it doesn't know if that object has been around for awhile or if it's just been initialized.

As well, we don't seem to have any way to check whether or not all the properties that are required to be set at init time have been set by the closure(s).

One issue appears to be that we need an existing instance of Foo before we can use any propertySetter closures. So, this is not the same thing as initializing a Foo using closures; we're just using the closures to mutate an existing Foo.

2. We can't compare two closures to make sure one doesn't overwrite the values set by another, without invoking them.

Additionally, there is not (as far as I'm aware) any way to introspect such closures prior to invoking them, which means we cannot compare two closures to make sure that one of them is not over-writing changes made by the other one.

Assuming this is correct, then using these kinds of closures does not seem like a full alternative to this proposal, since the result of such closures will always be an initialized object, whether or not the closures initialized all the required values.

More explanation on how failable initialization is used here:

This proposal relies on the existing Swift concept of failable initialization.

The proposed PropertyInitializable protocol's required init(_ properties:) function does not return an instance if the supplied properties array lacks a required property. To check this, we use reflection, extending the Mirror type with a new property nonOptionalChildren. This gives a way to ensure that all non-optional variables get set by the Properties array. We cannot do that with closures if we can't see what's in a closure—however we can see what's in a collection an iterate through those items until all the required variables are initialized (or not).

Note: due to the current limitations of Swift, my proof of concept (in the playground page linked above) relies on a hack—namely, initialization from properties involves:

• initializing a _blank instance within the original type according to a method defined by the adopter
• one by one setting the properties on _blank from the supplied properties array
• checking if any nonOptionalChildren remain unset (if so, the init(properties:) function returns nil)

Since the _blank gets deallocated if it turns out that the supplied properties were insufficient to initialize all its required variables, it suffices for a working proof of concept, with the unfortunate drawback that public let variables cannot be initialized using properties.

I consider this to be a temporary hack is in lieu of (hopefully) deeper improvements that might allow let variables to be written to by WritableKeyPath references inside an init method on the same type as the references (such as the init method provided by the PropertyInitializable protocol in this proposal).

On the subject of initializing let values with keypaths:

I have realized that one key aspect of this proposal (unless it should be its own) should probably be adding a capability for KeyPath to initialize a let value.

For instance, I might propose that Swift add a new type, something like InitializableKeyPath—it's is less than a WritableKeyPath in the sense that it's only writable within an init method on the Root type of the keypath. Of course access control would govern the visibility of Initializable just like it does Writable, and in that regard it does seem like an evolutionary and additive progression from existing Swift (not to mention, the inevitable dove-tailing of this proposal, like all others, into the territory of access control).

If that's how we want to go, then I'd need to make some additional changes to the proposal—namely, adding InitializableProperty and InitializableKeyPath types, defined as those that can be used to write to a variable only once and only at initalization. Following from this, an InitializableProperty would be an InitializableKeyPath<Root,Value> - <Value> pair, only usable for writing within an init method on the Root type.

This of course begs the question of whether it's at all feasible for a future version of Swift to support using a WritableKeyPath (or the hypothetical InitializableKeyPath) within an init method. The point would be to enable a type of keypath to be able to be used to initialize a let variable when initializing an instance. I'd sure like to know if that's way too much to ever hope for, especially as it concerns ABI impact.

[wadetregaskis]

@jawbroken

This is a very blunt hammer to apply to work around the unfortunate default initialisation of optionals to .none. You can instead write an initialiser that takes all three parameters, which will disable the default member-wise initialiser.

Edit: And I suppose the ship has sailed on this now, but I'm surprised that works in the first place. It seems to me it would have been nicer if the synthesised initialiser would only have a default value of nil if the variable declaration was explicitly var middleName: String? = nil, rather than implicitly.

That would mean, in order to be consistent and not complicate the language, that all Optionals would have to require default initialisation. While I recognise the downside to the behaviour in this specific case, I don’t think it justifies changing the default behaviour.

It does seem that the optimal solution would indeed be simply some way to indicate than the synthesised initialiser should not allow implicit Optional initialisation, either on a per-variable basis or for the struct overall (the latter being sufficient for the use case of this thread).

Perhaps:

struct Foo {
  var bar: String
  var baz: String? = .some
  var quux: String
}

The intent being to say that it has an initialisation value of something, but the specification of that something is obviously not fully provided at variable declaration, and so must be completed with an actual value in the initialiser(s).

This would also have the advantage of working broadly; explicit initialisers would also have to explicitly initialise it, which sounds like it’d be what @gahms is looking for even moreso…?

That said, a downside is that it’s a little wonky to explicitly initialise it to .none (nil) after the declaration suggests it has to be a .some of somekind. Maybe a simple but arbitrary …? = nodefault keyword would be better…

[jawbroken]

@wadetregaskis

That would mean, in order to be consistent and not complicate the language, that all Optionals would have to require default initialisation.

I don't see how that follows. It would just be a change to how the default initialiser was generated, and wouldn't be particularly inconsistent or complicated. In some senses it would be more consistent, e.g. only properties written var name: Type = default would receive the defaulted parameter name: Type = default in the initialiser, so all initialiser parameters would match their variable declarations. I had to check while writing my previous reply that it didn't already behave like that, because I had presumed that it would.

And while we're speaking hypothetically, I would have preferred that optionals didn't default to nil anywhere, because it's an unnecessary special case in the language that is of little value. But, as I admitted above, it's likely too late for either of those things.

[vitamin (Jon Gilbert)]

@CTMacUser

Could PropertyInitializable.init(_: ) be generalized to take a Sequence of PartialProperty ?

Absolutely. Perhaps it should just be an unordered Set.

[jawbroken]

Have you thought about how to handle repeated properties in the sequence? Would it be an error, first wins, last wins, etc?

[vitamin (Jon Gilbert)]

@jawbroken

Have you thought about how to handle repeated properties in the sequence? Would it be an error, first wins, last wins, etc?

Good question. In current proof-of-concept implementation, it's "last wins".

Now that I think about it, that's a good reason why not to use an unordered collection. In light of that, clearly I was wrong to suggest above that perhaps Set could be used. (Fixed.)

You can play around with it, just copy-paste this into a Swift 5 playground page in XCode.

[wadetregaskis]

@jawbroken

I don't see how that follows. It would just be a change to how the default initialiser was generated, and wouldn't be particularly inconsistent or complicated. In some senses it would be more consistent, e.g. only properties written var name: Type = default would receive the defaulted parameter name: Type = default in the initialiser, so all initialiser parameters would match their variable declarations. I had to check while writing my previous reply that it didn't already behave like that, because I had presumed that it would.

And while we're speaking hypothetically, I would have preferred that optionals didn't default to nil anywhere, because it's an unnecessary special case in the language that is of little value. But, as I admitted above, it's likely too late for either of those things.

Having to remember that Optionals in one context (e.g. a code block) have different default initialisation behaviour than in others (e.g. member variables) adds cognitive complexity. Granted not much since Swift irrespectively ensures no use before initialisation, but still. I don’t feel strongly about it, but I’d like to see other options explored & ruled out before I’d accept it happily.

On the other hand, changing the behaviour everywhere wouldn’t introduce any new complexity, it would just need to be justified. I thought I was quite against that, but thinking about it more, I’m not so sure… preventing use before initialisation is the more important goal, and that can still happen even if Optionals don’t default to nil - because they’d then just default to uninitialised, like any other type of variable, and having to ensure they’re explicitly set to nil if not to some value isn’t a big burden in the handful of cases I’ve just mentally reviewed.

Nonetheless, there must have been a lot of thought put into this to begin with, and the current behaviour chosen with good reason. So an argument for changing it must revisit that decision process and clearly show why it was mistaken and/or that the situation has changed.

Further nonetheless, though, I still feel like this is a leading contender for solving the motivating problems of this pitch, and impressively clever as Partial et al are, I’d still prefer to render them unnecessary, at least for most uses, by simply using the existing, super simple let vs var language feature.

[jawbroken]

@wadetregaskis

Having to remember that Optionals in one context (e.g. a code block) have different default initialisation behaviour than in others (e.g. member variables) adds cognitive complexity.

I'm not sure if we're talking past each other or not, but I was saying that the default initialisation behaviour would still hold for member variables, so no different behaviour or cognitive complexity. e.g. if you write your own initialiser and don't set the optional member variable then it would still be nil. Only the argument list of the synthesised memberwise initialiser would be different, not the member variable behaviour.