Constrain protocol usage outside the module

Evolution Pitches
1

Hello Swift community, I know I know, some of our souls are bleeding after the tiring mail-list discussions in dozens of similar threads, but I still feel that we have finally to sort this out. In the end we also moved to a forum not only for discoverability of older topics and better communication, but also to be able to push the language forward without necessarily relying on the help of the core team. In fact this is also implied by the new review rules where we must provide an implementation up front in order to get to a review process.

That said, I'd like to push this topic forward and hopefully find a group of people which share some interest in this topic and also implementors who are willing to tackle this for the next major release.

The topic was spread around different threads on the mailing list, that's why I wanted to start over, or better yet merge the discussion into one thread, so that the process is more organized.

Here are some of the threads I could find:

Yes this is another topic about two access modifiers, but only because access modifiers solves that issue nicely and in a consistent way. During the rush towards Swift 3 we introduced the open access modifier and made public the closed outside own module access modifier which is a great thing. However during the review only a very little of reviewers had some concerns that by introducing open to classes will left protocols out of the game and create (a) an exclusive access modifier for classes and (b) and inconsistent public behavior.

IMHO we as developer should benefit from all sort of possible constraints that are provided by the language for us. If we can say that classes cannot be subclassed outside a module, but still extended, why cannot we say that protocols cannot be conformed outside our module. Furthermore closed protocols should still be extendable and even can be subtyped by the user, this will allow the user to retroactively conform our types with his/her own refinement of the protocols and still forbid the conformance in own project. This also means that such a closed but public protocol can only be used as an interface.

Then open protocol will behave the same way as public protocol behave today. This is a nice alignment to the open/public class behavior.

A quick example:

// MODULE A

public protocol PublicProtoA {
  func doWork()
}

open protocol OpenProtoA {
  func beLazy()
}

open class Human : PublicProtoA {
  ...
} 

// MODULE B

import ModuleA

protocol ProtoB : PublicProtoA {
  func doMoreWork()
}

class Robot {}
extension : ProtoB { } // error cannot conform to ProtoB because `PublicProtoA` is public (closed)
extension : OpenProtoA { ... } // this is okay

extension Human : ProtoB { ... } // Retroactively conform a type to a refinement protocol

let lifeForm: Any = ...
if let human = lifeForm as? PublicProtoA {
  // This is an allowed use as an interface through dynamic type casts
}

To not break anything in Swift 5 mode all current public protocols will become open and public will have a closed behavior like described above.

This will also allow the core team to disallow the conformance to all hidden yet public protocols annotated with an underscore in the standard library.

Here is a list of such public protocols that I could find today:

_MinimalIterator
_HasCustomAnyHashableRepresentation
_ObjectiveCBridgeable
_ExpressibleByBuiltinIntegerLiteral
_ExpressibleByBuiltinFloatLiteral
_ExpressibleByBuiltinBooleanLiteral
_ExpressibleByBuiltinUnicodeScalarLiteral 
_ExpressibleByBuiltinUTF16ExtendedGraphemeClusterLiteral
_ExpressibleByBuiltinExtendedGraphemeClusterLiteral
_ExpressibleByBuiltinStringLiteral
_ExpressibleByBuiltinUTF16StringLiteral
_ExpressibleByBuiltinConstStringLiteral
_ExpressibleByBuiltinConstUTF16StringLiteral
_ExpressibleByStringInterpolation 
_ExpressibleByColorLiteral
_ExpressibleByImageLiteral
_ExpressibleByFileReferenceLiteral
_DestructorSafeContainer
_AppendKeyPath
_DefaultCustomPlaygroundQuickLookable
_SwiftNewtypeWrapper
_Pointer
_BitwiseOperations
_Mirror 
_ShadowProtocol
_NSFastEnumeration
_NSEnumerator
_NSCopying
_NSArrayCore
_NSDictionaryCore
_NSDictionary
_NSSetCore
_NSSet
_NSNumber
_NSArrayCore
_NSDictionaryCore
_NSSetCore
_NSStringCore
_NSStringCore
_UnicodeEncoding
_UnicodeParser
_OpaqueString
_UTFParser 
_AppKitKitNumericRawRepresentable
_AVCaptureDeviceFormatSupportedColorSpaces 
_AVCapturePhotoOutputSwiftNativeTypes
_AVCapturePhotoSettingsSwiftNativeTypes
_CFObject
_CGColorInitTrampoline 
_ObjectiveCBridgeableError
__BridgedNSError
_BridgedNSError
_BridgedStoredNSError
_ErrorCodeProtocol
_KeyValueCodingAndObserving
_INRideOptionMeteredFare
_UIKitNumericRawRepresentable

PS: I don't want this topic to escalate to a debate whether or not we should bring up this discussion or not. That said if you don't share any interest in that topic, please don't put a rock in it's possible way of success. ;)

5 Likes
2

This does not make sense in the context of the larger idea. If you can create protocol Q : P and then conform your own type to Q, then it necessarily conforms to P.

At this stage in Swift, I don't think that this is realistic. I think whatever the exhaustive enum conversation settles on (frozen?) will likely be how future additions to the language such as this would proceed.

These protocols are underscored for varying reasons.
Which brings me to an important question: What are some concrete use cases which motivate you to bring this up?

3

Maybe my wording wasn't quite correct there, but I showed in the example what I really meant. You can refine public-but-closed protocol, but you cannot conform your own types to nether the public-but-closed protocol nor to your refinement, the only conformance you're allowed to make is to a type which already conforms to that protocols - which is a type from where that protocol was defined. You can only conform your types to open protocols.

Why? As long a we do it in respect of source compatibility that should be fine, or are you saying that we cannot evolve the language any further, and we should not fix areas that couldn't be fixed due the lack of engineering time to implement things in time? I think @tkremenek mentioned that in a different topic that we should still be able to evolve the language even if the ABI is closed, but I still get the impression from such replies as yours now that the ship has sailed for the language to evolve any further.

Funny that you ask. Wasn't it you who said long time ago that to solve a particular problem I described we can use enums, to be more precise anonymous enums cases?

// The example was more or less like this
protocol CustomType {
  var instance: IntOrString { get }
}

enum IntOrString {
  case int(Int)
  case string(String) 
}

But now that we know this isn't going to happen at all because we don't want Int | String in the type-system you can no longer argue with that solution against closed protocols. Closed protocols can be used to disallow the user from conforming to a protocol which might be used to hide concrete types from the outside the module for example (isn't Apple also doing this quite often in theirs frameworks? - there are some protocols which you cannot conform to because you cannot instantiate or get all types it requires due to hidden init). I think it's fair to say that the library design is up to developers and should not restricted by language design, especially not when we already have a similar behavior in the language. This will become a generalization of that feature.

One could introduce @closed public protocol for Swift 4 which in Swift 5 mode would mean public protocol and public protocol without @closed from Swift 4 would mean open protocol in Swift 5 mode. Isn't the source compatibility save here?

4

I see what you mean. That's fine, but this is a very confusing way of expressing the idea.

You must be proposing a different thing from what the literal words mean in your post. From what I'm reading, you're saying that public protocol should mean a different thing in a future version of Swift from what it means today, which would be massively source-breaking. I do not think that such a change is going to ever be permitted.

I don't understand your example, and I don't recall seeing it from the past. You've just restated what closed protocols would be as the motivation for why you want them. But why? That is, what concrete real-world problem are you trying to solve which motivates you to prevent users from conforming to one of your protocols?

5

Apple's own frameworks have a handful of these protocols, like NSFetchRequestResult. These are cases where a library wants to handle a finite set of types, but the types aren't otherwise related. Of course you can document "you shouldn't implement this protocol", but it would be nice™ if the compiler could just tell you that.

I wish that we could use public/open for this, but unfortunately I think that ship has sailed. What I'd rather look into is the idea of non-public requirements, which would render a protocol unimplementable from outside the home module anyway. I admit this is somewhat of a language hack, but it's a fully backwards-compatible one.

EDIT: That said, there are still interesting Library Evolution concerns here: if a protocol has non-public requirements, it can never remove them, or at least not all of them (unless we figure out how to say when they were removed); and if a protocol does not have non-public requirements, it cannot add them.

1 Like
6

Of course, a non-frozen enum can also do most of this…but that's a lot heavier, syntactically. (It's interesting how non-frozen enums and non-open protocols can both nearly do the job of the other, but the language doesn't make it easy to do so.)

1 Like
7

I agree that while the ideal design (as I have proposed in the past) is to use public and open to align with classes that ship has sailed. At this point It seems like we would need to use something along the lines of @closed public protocol to indicate that conformances outside the module are not allowed. It's unfortunate that a more elegant design isn't possible anymore but the semantics are far more important than the syntax anyway.

I would really like to see a feature along these lines, especially along with exhaustive switching over non-public and @closed protocols. As @jrose points out, while there are many similarities in capability between enums the language provides friction of various kinds that usually result in one or the other being a superior design. IMO it would be nice to see protocols achieve more parity with with enums in this area. @closed is one desirable step. Exhaustive switch is another. I can even imagine @frozen @closed public protocol although I don't know a concrete use case for that off the top of my head.

@xwu I don't have time to dig through the archives right now but there have been example use cases posted in the past. @DevAndArtist if you want to revive this topic it might be a good idea to spend some time gathering those and putting together motivation for tackling this topic.

2 Likes
8

If I get it right, what you're asking for is something akin to sealed trait in Scala. I believe that the only real "benefit" it has over enums with associated values is that it gives better locality on conformance. i.e. each case does the full conformance instead of having each property implemented by all the cases (often just a big switch)

9

That’d be beautiful. I’ve often wanted enums to be a bit more flexible. This solves the same problem by making protocols a bit more constrained. An Either type (with an arbitrary number of possibilities/cases) would be another solution.

10

Would be interesting if we could make the following possible at some point.

@closed protocol IntOrString where Self == Int || Self == String {}
1 Like
11

Would that be syntactic sugar for:

@closed protocol IntOrString {}
extension Int: IntOrString {}
extension String: IntOrString {}
1 Like
12

I'm not sure, this is all bikeshedding, and I'm confused what @closed for non-public access should mean anyways. The where clause should remain so that you cannot conform other types to the protocol. To me only @closed public does make sense, but the where constraint already forbids that you conform the protocol to other types and there is no need for @closed here.

// `final` so that you cannot extend the protocol?
final protocol IntOrString where Self == Int || Self == String {}
extension Int : IntOrString {}
extension String : IntOrString {}

I think this is exactly why the type system should not support Int | String because if it's done using protocols, the amount of protocols will simply explode.

In the end that particular Example really should be solved using anonymous enum cases, which we probably never will see. :(

@fronzen public enum IntOrString {
  case (Int)
  case (String)
}

@closed has other use cases as well.

13

The part I’m most interested in is that @closed disallows consumers of the module from conforming their own types to a protocol. This, in turn, means that we can generate an exhaustive list of all types that conform to a certain protocol since they’re all within the module. Therefore, we can perform an exhaustive switch over those types.

switch intOrString {
  case let int as Int: print(int*int)
  case let str as String: print(“\(str) squared”)
  // no default necessary & will get a non-exhaustive error if we conform another type to IntOrString in the future
}

EDIT: of course this exhaustive switching will only work within the module, clients still need to be resilient to potential future changes in the module they’re consuming

3 Likes
14

Ah I see, I might have been misunderstood your question.

In theory in Module A:

@closed public protocol IntOrString {}
extension Int: IntOrString {}
extension String: IntOrString {}

would allow to use an exhaustive switch in Module B (which imports Module A).

EDIT: But what happens if in the next major release of Module A you add extension Double : IntOrString {}, because the protocol itself didn't had any other constraints?

15

Yeah, we both edited our posts to clarify that.

Exhaustive switching over a closed protocol defined in module A can only be done within module A, as module B cannot control what module A might do in the future. It’s the same reason we need “non-exhaustive” enums.

16

Right, a protocol would have to be both closed and frozen (with all conformances public) in order to allow exhaustive switching outside the declaring module.

17

Wouldn't that require @frozen to be versioned like @available?

18

I think it would have exactly the same semantics as @frozen will for enums.

19

I have a need for this exact feature. I have an API in a project which needs to handle: Array<Int|String>

Both currently available options are not great:

  • Using an enum to wrap the values hinders the ergonomics of the API to an unacceptable degree
  • Using a protocol with conformance for Int and String requires me handle error scenarios that would be impossible if I could freeze conformance to the protocol outside of my module, and that error handling also impacts the ergonomics of the API.

Would love to see the ability to prevent refinement and retroactive conformance of a protocol outside of the declaring module*. I'm pretty agnostic as to how that feature is spelled.

(* - Or some other solution... I'm not picky about this. If the solution involved generics instead of protocols, that would solve my problem too)

2 Likes
20

Can you give more details about what exactly you're designing that requires an array of either Int or String? Since it's been stated on this list that enums are the intended way of solving this issue, what ergonomics issues are you running into, and shouldn't we be solving those?