Existentials and custom constraints

@Douglas_Gregor, @John_McCall
Although swift is rely heavily on composition and interfaces, there was a lack and need of thing called existential container, which means a variable that can hold a value of a type, specified by interface constraint. If you have been is swift long enough, you probably faced this problem.

protocol MyProtocol {} //empty protocol, aka empty inteface
var someVar: MyProtocol // this actually works, but hold on.

It works indeed, but thing get more complicated,when protocol contains an assocciatedtype, which means, that types must be derived from context.

protocol OpaqueProtocol { assocciatedtype Value ; var someVar: Value }
struct SomeStruct: OpaqueProtocol { var someVar = "Hello" }
var instance = SomeStruct() // type of Value is inferred to be String

var opaqueVar: OpaqueProtocol = SomeStruct() //error: protocol cannot be used here

I was very disappointed by fighting that issue, but I tried to make a workaround and made a sort of satisfieble solution.

struct CustomStorage<Value> {
    private(set) var value: Value?
    private(set) var setter: Functor<Value, Void>

    init(constraints: [(Value)->Bool] = []) {
        self.setter = Functor.init(preconditions: constraints){$0}
    }
    init(constraints: [(Value)->Bool] = [], value: Value) {
        self.setter = Functor.init(preconditions: constraints){$0}
        do {
            try self.setter.invoke(with: value)
        } catch let predicate {
            fatalError("\(value) does not satisfy \(predicate)")
        }
        self.value = value
    }
    /*mutating func turnIntoUnboundType() {
        self.value = value as! Any
    }
    mutating func reboundTo<P>(type: P.Type){
        self.value = value as! P
    }
    mutating func rebound() {
        self.value = value as! value.Type
    }*/
    mutating func addConstraints(_ list: (Value)->Bool...){
        self.setter.preconditions += list
    }
    mutating func addConstraintsAndCheck(_ list: (Value)->Bool...){
        self.setter.preconditions += list
        if self.value != nil {
            if (try? self.setter.invoke(with: self.value!)) == nil {
                fatalError("Current value in \(self) does not satisfy new constraints")
            }
        }
    }    
    func get<T>(_ transformer: (Value) -> T) -> T? {
        if self.value != nil {
            return transformer(self.value!)
        }
        else { return nil }
    }

    mutating func set(to newValue: Value) throws {
        do {
            try setter.invoke(with: newValue)
        } catch let error {
            throw error
        }
        self.value = newValue
    }
}
struct Functor<Input, Output> {
    //assuming only fuctions with one argument
    let function: (Input) -> Output
    let inputType: Input.Type
    let outputType: Output.Type
    var preconditions: [(Input)->Bool]
    
    init(preconditions: [(Input)->Bool] = [], function: @escaping (Input) -> Output) {
        self.function = function
        self.preconditions = preconditions
        self.inputType = Input.self
        self.outputType = Output.self
    }
    enum PredicateError: Error {
        case preconditionError((Input)->Bool)
        case postconditionError((Output)->Bool)
    }
    func invoke(with a: Input) throws -> Output {
        for f in preconditions {
            if f(a) == false { 
                throw PredicateError.preconditionError(f); break
            }
        }
        return self.function(a)
    }
}

It works in quite useful way, like this:

var nonZeroNumber = CustomStorage.init(constraints: [{$0 > 0}, {Float($0) != nil}], value: 5)
// can hold any value greater than zero, that can be converted to Float

The reason I am showing you this is to demonstrate a concept of a customizible variable, that can not only hold some trivial constraint like var a: Numeric = 0, but more flexible and even modifiable at run-time! Like this, potentially:

var onlyNumbers = CustomStorage
    .init(constraints: [{$0 is Numeric}])
onlyNumbers.addConstraints({$0 < 1024}, {($0 * $0) > $0 + $0})
onlyNumbers =? 0 //assignments fails, cuz constraints are violated
onlyNumbers =! 3 //assignment succeeds
//I also made some syntactic sugar in form of this operators, 
//so it is possible to 
if a =? 10 { print(a.get{$0}!) }  

A variable onlyNumbers can contain any type that conform to numeric, or any other wild constraints, which you can create. Turing-completeness in help!

The most noticeable downfall here is that it doesn't provide wanted flexibility. And strict typing and messy type system are those to be blamed.

First of all, the potential way to do this is to explicitly initialize CustomStorage with type Any, but because swift cannot guarantee, that traits of a given type satisfy closures in preconditions, is halts compilation with error, whenever these closures contains anything type-specific. For example:

var anyStringRepresentable = CustomStorage<Any>
    .init(constraints: [{$ != ""}, {$0 is StringRepresentable}]) //doesnt not compile

One thing to pay attention to is that ,even though anyStringRepresentable is not declared with concrete type(e.g. String), at any given moment after initialization and assignment that Any type in fact represent some valid concrete type(compiler checks all conformances at compile-time). So at any that time it is some what allowed to do the following, if we omit the fact that Any can be non-nominal type:

value = value as! value.Type //impossible to do now. Compiler doesn't know what i want
//or
var a: Any = 5
a = a as! a.Type
type(of: a) //Int

//consider following code as well
var c = CustomStorage<Any>.init(constraints: [{$0 is Int}]) //Only Ints
c =! 4
var d = CustomStorage<Any>.init(constraints: [{$0 is Int}]) //Only Ints
d =! 5
type(of: c.get{$0}!) //Int. Concrete type
type(of: d.get{$0}!) //Int. Concrete type

//If we need to abstract away concrete types, we can define it so all lookups happen at runtime from what i know
func + <T, U>(lhs: CustomStorage<T>, rhs: CustomStorage<U>) throws
    -> Int where T: BinaryInteger, U: BinaryInteger { ... }

let someVar = c + d //Nope! Compilation error

But it fails at compilation, beacuse binary operator '+' cannot be applied to two 'CustomStorage<Any>' operands. So swift actually doesn't perform lookup, i believe, because it cannot know that these types are actually nominal, hence it faults.
Potential solution to this is to split Any into subtypes to have some guaranties, so type cheker can enforce correctness:

AnyType - can store anything that exist in swift. No safe way to lookup at runtime.
AnyNominal - can only store types, that conform to protocols(structs, classes, enums), can be look-uped for metadata in runtime
AnyMetatype - can store type, that describe other types

If we assume the idea above, than doing conformance check is always 100% safe.

extension AnyNominal { func isConforming(to protocol: Protocol.Type) -> Bool {...} }
"String".isConforming(to: Equatable.self) //true
//or
var a = CustomStorage<AnyNominal>
    .init(constraints: [{$0.isConforming(to: Numeric.self)}])
  // means it is safe to compare against

//or from previous example above
var c = CustomStorage<AnyNominal>.init(constraints: [{$0 is Int}]) //Only Ints
c =! 4
var d = CustomStorage<AnyNominal>.init(constraints: [{$0 is Int}]) //Only Ints
d =! 5

func + <T, U>(lhs: CustomStorage<T>, rhs: CustomStorage<U>) throws
    -> Int where T: BinaryInteger, U: BinaryInteger { ... }

let p = c + d // look up at runtime, crush or propagate error, if constraints aren't met 

I assume as of now conformance check can be done by some introspection capabilities with involvement of spawning mirrors and comparing structures to know if an object is nominal, meta-type or anything else, but it would be much less clunky and faster to embed this functionality in lower level.

I have been on swift forum for a while, and noticed folks around were asking for simillar functionality, like in these threads: [Generalized Type Erasure (Existential Types)], [Pitch: Introduce custom attributes], [Extensions for existential types. Custom existential containers], [Enhanced existential types proposal discussion], [Status of subclass existentials]

So what was this all about?
Well,

  1. What is the best way to declare variable with multiple constraints on storable value ?
  2. I want to know what's the state of existential storage, and why it is so delayed
  3. It is already possible to make something similar with current language facilities as I showed, but to have same value as the concept we keep dream of, language should extend type casting system at least. Is it actually the case or am I missing something?
  4. Given all those points , I wonder, whether it would be better to implement opaque storage at language level.
  5. Will swift acquire any rich functionality, if it posses Any split as shown?

Here is some complete code to play around, if you wish. [Opaque Storage.swift - Google Drive]

What are you trying to do? It says existential container, but seems to switch to duck-typing mid-way through.

And all your example uses is Int as a constraint, so (IMO) it'd make more sense to do CustomStorage<Int>.

1 Like

This is somewhat pitched before, though not as generic existential in Allow for Compile-Time Checked Intervals for Parameters Expecting Literal Values (and other threads in the first few replies).

I don't immediately see any benefit. The only demonstration in the pitch is this line.

It doesn't get better since the problem lies in Equatable, not String.
Otherwise, on homogenous protocol you can already do

"String" is SomeHomogenousProtocol

What do you want the opaque storage to be able to do more than Any?

Oh, sorry for aberration. I wrote this code because I wanted a possibility to store values that are not represented by some concrete type, but instead as by a set of protocols, but also to have more freedom in defining restrictions on variables so it's not just protocol constraint, but rather a finer grained control. My main application to this is unit testing for now , but eventually I found out that it is not possible due to swift mostly static nature and inability to perform functions in runtime. While my initial idea was about testing it does not mean that it may not find application in other area, at least I see some good potential. ¯_(ツ)/¯. Perhaps , the solution is actually exist, and we could have something like custom storage in swift, not necessarily in standard library, but as some standalone package. My dream is Dynamic Swift with runtime dispatch and related error handling on top of a current swift. I have a clue that reflection is where to be headed, so I look there. I would like to ask by the way, whether you know if it is possible to get list of all protocol conformances in swift. ¯_༼ᴼل͜ᴼ༽

Anything like this:

var a = CustomStorage.init(constraints: [{$0 is Collection && $0.elementType == Numeric}])
//Any collection that contains numeric types
a.addConstraints({$0.reduce(0, +) > a.reduce(0, +)})
// Now this variable can only hold collections , which' summ of elements is greater than one of previous value
//You also can crush with description of errors
var someText = CustomStorage.init(constraints: [{
    if $0.contains(explicitLexic) {print("\($0) contains explicit lexic")}; return true}])
//Any imaginable constraints!

Well, you're right that is not possible to implement using directly CustomStorage, with something like

CustomStorage<Content: Collection> where Content.Element: Numeric

because that requires generalized existential storage.

But you can still work around the limitation like this right?

CustomStorage for Numeric Collection
struct CustomStorage<Value> {
    private(set) var value: Value?
    private(set) var setter: Functor<Value, Void>

    init(constraints: [(Value)->Bool] = []) {
        self.setter = Functor.init(preconditions: constraints){ _ in }
    }
    init(constraints: [(Value)->Bool] = [], value: Value) {
        self.setter = Functor.init(preconditions: constraints){ _ in }
        do {
            try self.setter.invoke(with: value)
        } catch let predicate {
            fatalError("\(value) does not satisfy \(predicate)")
        }
        self.value = value
    }

    mutating func addConstraints(_ list: [(Value) -> Bool]){
        self.setter.preconditions += list
    }
    mutating func addConstraintsAndCheck(_ list: [(Value) -> Bool]){
        self.setter.preconditions += list
        if self.value != nil {
            if (try? self.setter.invoke(with: self.value!)) == nil {
                fatalError("Current value in \(self) does not satisfy new constraints")
            }
        }
    }
    func get<T>(_ transformer: (Value) -> T) -> T? {
        if self.value != nil {
            return transformer(self.value!)
        }
        else { return nil }
    }

    mutating func set(to newValue: Value) throws {
        do {
            try setter.invoke(with: newValue)
        } catch let error {
            throw error
        }
        self.value = newValue
    }
}
struct Functor<Input, Output> {
    //assuming only fuctions with one argument
    let function: (Input) -> Output
    let inputType: Input.Type
    let outputType: Output.Type
    var preconditions: [(Input)->Bool]

    init(preconditions: [(Input)->Bool] = [], function: @escaping (Input) -> Output) {
        self.function = function
        self.preconditions = preconditions
        self.inputType = Input.self
        self.outputType = Output.self
    }
    enum PredicateError: Error {
        case preconditionError((Input)->Bool)
        case postconditionError((Output)->Bool)
    }
    func invoke(with a: Input) throws -> Output {
        for f in preconditions {
            if f(a) == false {
                throw PredicateError.preconditionError(f)
            }
        }
        return self.function(a)
    }
}

struct NumericCollectionStorage<Content: Collection> where Content.Element: Numeric {
    private var storage: CustomStorage<Content>

    init(value: Content) {
        self.storage = CustomStorage<Content>(value: value)
    }

    subscript<T>(dynamicMember keyPath: KeyPath<CustomStorage<Content>, T>) -> T {
        return storage[keyPath: keyPath]
    }

    mutating func addConstraints(_ f: [(Content) -> Bool]) {
        self.storage.addConstraints(f)
    }

    mutating func addConstraintsAndCheck(_ list: [(Content) -> Bool]){
        self.storage.addConstraintsAndCheck(list)
    }

    func get<T>(_ transformer: (Content) -> T) -> T? {
        return storage.get(transformer)
    }

    mutating func set(to newValue: Content) throws {
        try storage.set(to: newValue)
    }
}

The downside is of course having to write one such container struct for each combination of protocols that we want to use... which kinda defies the purpose of code reutilization :upside_down_face:

1 Like

Yeap , it is exactly that. Having a unified way to express such intents without bloating code and creating syntactic noise is what what would be the best. :slightly_smiling_face: