Generalization of Implicit Conversions

Evolution Discussion
1

Since the topic of generalized implicit conversions came up during multiple recent pitch discussions, I’d like to create a space where folks who are interested can discuss whether such a feature fits into the language and gather potential design ideas. This thread could be easily referenced in the future instead of having to search of bits and pieces accumulated across multiple pitch threads.

Swift language supports a variety of implicit conversions: value to optional promotion, optional conversions, collection upcasts, Double/CGFloat conversions, conversions involving pointers etc. What follows is a list of conversions with their traits, note that one trait could be shared by multiple conversions.

Note on notation - <: denotes a subtype relationship, ! → Implicitly Unwrapped Optional, ? - Optional.

  • Value to optional promotion:
    • T → T?
  • Optional to optional conversion (T and U are types and T <: U)
    • T? → U?
    • T! → U!
    • T! → U?
  • Bi-directional conversions:
    • Double ↔ CGFloat
    • CF type ↔ Objective-C type
    • named tuple ↔ un-named type (when element types match)
  • Controlled lifetime conversions (cannot contract intermediate/temporary objects)
    • Array → pointer
    • String → pointer
  • Non-type and controlled lifetime conversions
    • inout → pointer
  • Pointer → Pointer conversions, for example:
    • UnsafeMutableRawPointer → UnsafeRawPointer
    • UnsafePointer → UnsafeRawPointer
  • Pointer conversions for C/ObjC interoperability (currently being pitched on the forums), for example:
    • Unsafe[Mutable]RawPointer → UnsafePointer<[U]Int8>
    • Unsafe[Mutable]Pointer<Int{8, 16, 32, 64}> → Unsafe[Mutable]Pointer<UInt{8, 16, 32, 64}>
  • Upcast and bridging conversions for standard library collection types:
    • Array → Array where T <: U or T bridged to U
    • Set → Set, where T <: U or T bridged to U
    • Dictionary<K1, V1> → Dictionary<K2, V2> where
      • K1 <: K2; or K1 is bridged to K2
      • V1 <: V2; or V1 is bridged to V2

Currently supported conversions are generally composable (e.g. Double → CGFloat → CGFloat?) with some restrictions e.g. Array → inout → pointer, and allowed in AST representable positions (e.g. arguments of calls, subscripts etc.) as well as inside types e.g. Double → CGFloat? requires value to optional promotion with subsequent Double → CGFloat conversion.

Design of the generalized implicit conversions should evaluate which of the existing conversions could be subsumed (implementable via new mechanism) and which should be kept separate e.g. bi-directional, or non-type → pointer conversions, because they would require compiler support. Design should also account for the compiler impact, for example, how proposed mechanism would affect type-check performance of expressions that - use of implicit conversions (does it scale for N conversions?), and ones that don’t require them to be type-checked successfully.

23 Likes
2

when the inner types match there's also
named tuple ↔ unnamed tuple

3 Likes
3

Thanks! I'm going to add that to the list.

2 Likes
4

I’d be interested in:

• User-defined variance of generic parameters. As in, the author of a type should be able to specify that, say, Element is covariant, and thus MyType<Subclass> is a subtype of MyType<Superclass>.

I don’t know how that would work under the hood, but conceptually it would be nice if users could make types that behave like Array in this manner.

• Enum restrictions. As in, given an enum Foo, programmers should be able to declare enum Bar: Foo which redeclares a subset of the cases of Foo.

Bar is then treated as a subtype of Foo, and users get the type-system guarantee that an instance of Bar can only hold a restricted subset of cases, while still being able to pass it to functions that take a Foo parameter.

• • •

Struct subtyping could be useful for C++ interop, but I haven’t yet encountered a need for pure-Swift struct inheritance. Someone else might though.

6 Likes
5

My perspective is that the current ability of Array to perform implicit conversions when its element is convertible was a mistake, and we should not propagate it.

In languages where everything is held via a pointer that conversion might be OK, because it is possible to just reinterpret the buffer as containing pointers to AnyObject instead of a pointers to SpecificObject. That can be done in constant time.

But Swift doesn't just have arrays of pointers like NSArray does. It has arrays of structs. And while SpecificStruct can be implicitly converted to Any or SpecificStruct? or SomeExistential, that does not mean that you can convert a [SpecificStruct] to an [Any] in constant time. Instead, the compiler must perform an implicit form of specificStructs.map { $0 as T }. This is a linear operation, and that is a bad thing to do implicitly.

I understand why we have this feature. To many people used to other languages, it would be completely inexplicable that array not have this conversion. And since Swift has classes, there are some cases where it can be done in constant time. And this kind of conversion was essential for smooth interoperation with Objective-C, especially early on.

As much as I'd like to, I expect it's too source breaking to consider undoing the linear-time versions for Swift 6. But we definitely shouldn't worsen the problem by extending it to other collections. I also suspect that (rightly) not having this ability to generalize collection conversions weakens the case for generalized implicit conversions overall.

13 Likes
6

To more formally pitch a solution I proposed in another thread:

Proposed Solution

I propose that we should allow types to inherit from other arbitrary types at the point of their definition in the same way that we declare subclassing or protocol conformance, and then require that the type provide a func upcast() -> SuperType member function.

To give an example, UnsafeMutablePointer would be declared as something like:

struct UnsafeMutablePointer<Pointee>: @subtypeConvertible UnsafePointer<Pointee> {
    public func upcast() -> UnsafePointer<Pointee> {
        return UnsafePointer(self)
    }
}

The subtype would inherit all protocol conformance requirements from the inherited type, and would have default implementations for the requirements that upcast to the inherited type and call the protocol requirement on that.

Upcasts can be chained, so if C: @subtypeConvertible B and B: @subtypeConvertible A:

let a: A = C()

is implemented as:

let a: A = (C().upcast() as B).upcast() as A

If there's a way to make the runtime casting functionality work, we should also allow conditional downcasts using the as? operator if the subtype provides a init?(downcasting: SuperType) initialiser:

enum GPUResource {
    case texture(GPUTexture)
}

struct GPUTexture : @subtypeConvertible GPUResource {
    public func upcast() -> GPUResource {
        return .texture(self)
    }
 
    public init?(downcasting superType: GPUResource) {
        switch superType { 
        case .texture(let texture):
            self = texture
        default:
            return nil
        }
    }
}

This proposed solution unfortunately doesn't cleanly subsume any of the currently supported implicit conversions. Implicitly, we have for any type T that T: @subtypeConvertible Optional<T>, but it's not reasonable to ask the user to write that out for every type.

For the collection conversions, we could maybe have something like:

struct Array<Element>: @subtypeConvertible Array<Super> where Element: Super {}

To @Ben_Cohen's point above, I'm not sure whether supporting that is a good idea or not.

If we want to support widening integer conversions, those could be written as:

struct UInt16: @subtypeConvertible UInt8 {}

Motivating Use Case

For performance reasons, I've on multiple occasions implemented my own protocol existential equivalent. One example of this: in a library I maintain, Resources are handle types that index into struct-of-array registries. To give a simplified implementation:

enum ResourceType: UInt8 {
    case buffer
    case texture
    case heap
}

struct Resource {
    let handle: UInt64

    var type: ResourceType { 
        return ResourceType(rawValue: UInt8(self.handle.bits(in: 56..<64))) 
    }
}

struct Texture /*: @subtypeConvertible Resource */ {
    let handle: UInt64

    init?(_ resource: Resource) {
        guard resource.type == .texture else { return nil }
        self.handle = resource.handle
    }
}

Currently, I have a parent ResourceProtocol protocol which all conversions have to go through but which I never want to use as an existential; this proposal would allow me to remove that.

In particular, if we supported as? downcasts, it would have prevented a number of bugs where I've inadvertently written:

let resource: Resource = ...
if let texture = resource as? Texture {} // never succeeds
// as opposed to what's currently required:
if let texture = Texture(resource) {}
7

This may be already mentioned across the various pitch threads, but I think is important when talking about a more general implicit conversion to consider experience from similar features in another languages and try to get some insight about the goods and bads of such feature in their context.

Coming from C++ that supports such feature where the rules are simply if a type A have a single parameter user defined constructor that takes a type B (or a conversion operator A -> B), type B is implicitly convertible to A.

class B {};

class A {
public:
  A(B&) {}
  A(const B&) {}
};

int main(int argc, const char *argv[]) {
  const B b;
  A a = b; // implicit A(B&)
}

This is also valid for conversion operators, but since is also user defined the rules are basically the same.

It often common to stumble in best practices talks and blog and see that although that implicit conversion is allowed in the language the use of such mechanism is often discouraged or advised to be used carefully. The most relevant one being that implicit conversion can introduce extra runtime cost depending on how this constructor is defined, that is often hard to debug because user have to be aware that an implicit conversion may be happening. So it is often discouraged in many conference talks and community discussions and the recommendation is to always try to use explicit keyword that make a constructor not available for implicit conversion so the compiler can error and not allow such implicit behavior.
An example can be found here.
Also, additionally to runtime extra cost, an implicit conversion can also introduce unexpected behavior because it depends on how the conversion is defined by the user and which conversion was picked up by the compiler. This is something that can be specific to the way C++ does selection of the implicit constructor, but it can highlight how it can cause unexpected behavior from the programmers perspective or even sometimes the addition of an implicit conversion can introduce ambiguity.

Note that this may be valid only on the C++ context because of the way implicit conversions work on the language, but I think they are still worth mentioned if some similar concept is considered for Swift.

There may be more we could take from C++ community in relation to this feature and from other languages as well and they may or may not be relevant for modeling implicit conversions in Swift, but is worth raising the point.

From personal experience, I do agree with the recommendation of C++ community and as for the whole general implicit conversions feature, although it really useful in some cases a general mechanism has to be used carefully or the feature could be designed in a way that impose some restrictions that make it harder for users to fall into similar problems as the C++ feature.

4 Likes
8

My understanding is that implicitly-unwrapped optionals are now exactly that: optionals with a tag saying that you can use them without writing x! at each usage site.

From SE-0054:

Appending ! to the type of a Swift declaration will give it optional type and annotate the declaration with an attribute stating that it may be implicitly unwrapped when used.

However, the appearance of ! at the end of a property or variable declaration's type no longer indicates that the declaration has IUO type; rather, it indicates that (1) the declaration has optional type, and (2) the declaration has an attribute indicating that its value may be implicitly forced. (No human would ever write or observe this attribute, but we will refer to it as @_autounwrapped .) Such a declaration is referred to henceforth as an IUO declaration.

Likewise, the appearance of ! at the end of the return type of a function indicates that the function has optional return type and its return value may be implicitly unwrapped. The use of init! in an initializer declaration indicates that the initializer is failable and the result of the initializer may be implicitly unwrapped. In both of these cases, the @_autounwrapped attribute is attached to the declaration.

So it seems like it's more of a syntax sugar for this hidden attribute, rather than an implicit conversion between types. I have no idea how the compiler treats it internally. Is it possible that it still treats IUOs as a separate type with implicit conversions for compatibility with pre-Swift-4.2 code, and that it can now be simplified?

1 Like
9

Yes, I think you are right, this case might not be particularly interesting one for the language, although it's indeed implemented as an implicit conversion. T? -> T! is more interesting but it's covered by T! -> U!, so I'm just going to remove it from the list.

3 Likes
10

In an earlier thread @jrose mentioned the current compiler already supports implicit conversion like UnsafePointer<Int8> -> UnsafeRawPointer for function call arguments. I wonder if that should be in the above list also?

11

Sure, I have updated the bullet point that talks about pointers and added some more examples.

12

This is not universal, right? I think this only works when calling a closure with a tuple parameter. Are there other times this works?

13

If I remember correctly also in case matching and other patterns. It used to be any subtype relationship but @Slava_Pestov would know better.

14

This thread is about type conversions that succeed. Therefore I want to carefully bring this phenomenon to your attention and ask if it would be somehow related?!

While the conversions are not implicit, they still succeed.

1 Like
15

Thanks, @DevAndArtist! This discussion was intended to be more about merits and possible solutions for generalization of implicit conversion than just trying to meticulously enumerate all of them :slight_smile:

1 Like
16

I get that, but identifying current implicit conversions and comparing how they work (and how they differ) is an important part of conceptualizing a generalization that would later encompass them all, do you think?

17

I consider some of them like tuple-tuple to be a part of generic type conversion, that’s why I haven’t listed some conversions specifically, other examples include - deep equality, T -> Any or AnyObject, superclass conversion etc.The list in description only contains what compiler handles as implicit conversions, separate from normal conversion and subtyping.

18

What about #file, #fileID and #filePath? They can be used in placed where a String or StaticString is expected. Maybe that is just implemented as an overload and not as an implicit conversion because StaticString -> String or String -> StaticString does not work elsewhere. However, #file and friends are documented to return a String and not a StaticString: Expressions — The Swift Programming Language (Swift 5.7)

19

These are literals. More specifically, I believe #file and friends are a special kind of string literal (except for #line and #column which are special integer literals). They have a default type of String, but the inferred type can be anything that conforms to ExpressibleByStringLiteral.

12 Likes
20

To try to keep this thread moving I’d like to re-state my previous thoughts on generalized conversion, this time in the form of a proposal. Apologies to those who have seen these before but I’ve made minor changes to the design I discussed previously.

Generalization of Implicit Conversions

During the review process, add the following fields as needed:

Introduction

It's proposed that a mechanism be added by a relatively small change to the Swift compiler to express the implementations of conversions from source to destination types which can be automatically applied by the compiler as it type-checks an expression. These conversions should be expressed in the Swift language itself and have as near as possible to zero cost to users who do not make use of the feature.

Related Swift-evolution threads: Automatic Mutable Pointer Conversion Pitch: Implicit Pointer Conversion for C Interoperability Generalization of Implicit Conversions

Motivation

While Swift is capable of perhaps unprecedented expressiveness in its expressions it is very strictly typed and incapable of automatically making some of the smallest conversions between distinct types even when such a conversion would be lossless. This is a source of friction when interacting with data derived from C apis and a source of frustration to new users of Swift who are used to C's less rigid rules. One encounters this when trying to compare an Int32 with and Int for example or trying to pass an Int32 to a function that expects an Int. A means needs to be made available where the compiler can convert a less precise type to a more precise type with the user having to explicitly add a "cast" to match the types using an initialiser.

It is possible to think of many such implicit conversions who's presence Swift would benefit from but the tendency until now has been to express these inside the compiler itself disempowering the average Swift developer and progressively adding complexity to the type checking operation and time spent compiling.

Proposed solution

In it's simplest this conversion could be expressed in the following form as a new initializer (with the attribute @implicit) on the destination type:

extension ToType {
	@implict init(from: FromType) {
		self.init(from)
	}
}

Detailed design

A concrete example of this feature in operation would be adding the following extension to the standard library:

extension Int {
  @implict init(from: Int8) {
    self.init(from)
  }
  @implict init(from: Int16) {
    self.init(from)
  }
  @implict init(from: Int32) {
    self.init(from)
  }
}

In a prototype it has been found it is then possible to pass Int8 values to functions expecting an Int and make comparisons between Int8 and Int values with requiring a "cast" of the less precise type as, operators are implemented as functions in the Swift lanaguge. In using the prototype it was not possible to measure any slowdown of the compiler with these conversions in place with repeated compilations of a reasonably large open source project. This may be due to a feature of the specific implementation which will discussed in detail later.

Another example would be a new potential implementation of the bi-directional conversion between CGFloat and Double.

extension Double {
  @preferred_implict init(from: CGFloat) {
    self.init(from)
  }
}
extension CGFloat {
  @implict init(from: Double) {
    self.init(from)
  }
}

Such a bidirectional conversions can create ambiguities for the compiler however as were one to compare a CGFloat to a Double the compiler does not know whether to convert the CGFloat to a Double and compare Doubles or vici verca. In this very rare (perhaps unique) case a means has to be found to specify that one direction is the preferred conversion perhaps using yet another attribute @preferred_implict as shown in this example.

A final example could be a developer working extensively with C strings in Swift in a particular project might find it useful to add the following conversions:

extension String {
    @implict init(_ from: UnsafeMutablePointer<Int8>) {
        self.init(cString: from)
    }
    @implict init(_ from: UnsafePointer<Int8>) {
        self.init(cString: from)
    }
}

Note it is not possible to express the reverse conversion as an initializer cannot express adequately the lifetime of the resulting pointer. Also note that the initializer is unlabelled in this case as, to reuse the current implementation of the type checker an unlabelled initializer needs to be available. More on this later.

Whether developers can be trusted with this expressivity is an open question which a decision on which is at the core of whether this proposal should proceed at all. I would counter this is a debate which often comes up in in relation to whether the presence of operator overloading enhances a language. I personally am in favour of trusting the developer and empowering them over concerns about potential overuse. Swift does include operator overloading to implement it's operators and despite this I'm not aware of such a powerful feature being a frequent source of incomprehension when approaching another developer's code base due to over use. It is typically, simply not used other than to implement the Swift language itself by library developers as is the intended audience of the implicit conversion feature feature. This reservation might be assuaged by ensuring implicit conversions are properly scoped by applying existing access control mechanisms. One example of this might be that it could be applied is to require that @implicit public init()'s, i.e. those that library developer is exporting only be allowed on a type defined in that module to limit "leakage" of implicit conversions.

If the feature were to be made available, is it possible it could be used to re-implement some of the implicit conversions already built into the Swift compiler? The answer seems to be "yes and no". For example, it may be possible to express the promotion of a non-optional value to an optional in Swift but it would not however be possible to express the conversion of a named tuple to an unnamed tuple in a generalized manner. In my opinion it may not be useful to pursue this anyway as the C++ code is already there and such a refactoring many serve little purpose and may introduce subtle regressions. It is intended the feature should be primarily additive.

Implementation

It is worth reflecting for a moment on the specifics of the implementation of the prototype fleshed out in the PR as this will help refute the other common reservation about adding implicit conversions: that they will result in slower type checking performance. To discuss this, let's break type checking into it's two distinct phases. The first, most time consuming operation is funding solutions for a particular expression in terms of concrete functions to call implemented by specific types. In the Swift compiler, this is performed by a very abstract "constraint solver". After this, a solution may have been found but it may need to be "fixed" or "repaired" by "applying" one or other of the conversions built into the compiler.

For example, for an expression trying to call a function expecting an Int with an Int32 value it seems (I'm not the expert) the constraint solver identifies that an unlabelled initializer for Int taking an Int32 argument is available but doesn't view this solution as being "viable". So, it is already performing the work but giving up at that point and logging an error. The implementation of the prototype simply adds a "fix"/"repair" if an @implicit initialiser is available making the solution viable after the initializer has been "applied".

This led me to conclude that type checking is not being slowed down and indeed it was not possible to measure a slowdown with repeated builds of a fairly substantial open source project with and without the feature being available or in use. Keeping implicit conversions out of the constraint solver also prevents them from cascading leading to an exponential explosion of possibilities to check. It also makes possible bi-directional conversions. For more details on the prototype, consult the provisional PR mentioned above and the summary notes.

Source compatibility

This is an additive feature allowing source that would previously have failed to compile without an explicit conversion to compile in future versions of swift.

Effect on ABI stability

N/A as this is a source level feature and is additive.

Effect on API resilience

N/A as this is a source level feature and is additive.

Alternatives considered

Continuing to add add-hoc conversions to the C++ source of the compiler or not making more conversions available at all. An alternative form where conversions are expressed as an overloaded function on the FromType returning the ToType has been considered in the past and was in fact a feature of the early Swift betas but adding it as an extension to the ToType seems to group conversions better from the point of view of localising them and allows the implementation to re-use the type checker essentially as-is.

Acknowledgments

Swift

Edit: replaced @preferred @implicit with @preferred_implicit so as not to have an independent attribute @preferred.

Edit 2: Added idea of restricting public implicit conversions to be only possible if the module owns the type. This would preclude implementing the CGFloat <-> Double conversion using the new mechanism but we already have an implementation for that.

11 Likes