Sealed protocols

The syntactic requirements of a protocol have always been expressible in the language itself, but the semantic requirements are not and largely cannot be. It is essential to a proper conformance to "RTFM"--that is not a fallback--and I would say that anything that gives a user the impression that they don't need to "RTFM" to understand the contract is not only a non-goal but an anti-goal.

1 Like

Another thing to consider though is what guarantee the "sealed" invariant gains you as a programmer developing your library. As proposed, "sealed" alone doesn't give the compiler enough information to statically verify that you've exhaustively switched over conforming types, so the only statically total way to interact with a protocol interface would still be through its requirements. Therefore, while you would be getting a guarantee that code outside your module doesn't add conformances, you'd still have to rely on dynamic assertions and/or auditing outside of what the compiler checks to ensure that you're exhaustively handling your own cases in switches or other non-protocol-requirement-dispatch mechanisms. OTOH, if you factor your code such that you always dispatch your type-specific logic through the protocol's requirements, then the "no outside conformances" constraint may not really buy you anythingā€”it may not always be practical for external types to conform, but if they manage to, there's no direct harm in their doing so.

If part of the goal of this proposal is to allow exhaustive enumeration of conformances outside of protocol requirement dispatch, then an alternative design might need to be explored. However, having a requirement like "private and local classes are not allowed to conform if a protocol is sealed" would be really weird. In languages with "case classes" to represent closed sums as class hierarchies, for instance, all of the cases must be defined together with the parent class; maybe a sealed protocol could have a way to enumerate the conformances up front as part of the protocol definition?

There are a lot of semantics expressed in the type system and more semantics implied by the names we choose for our members. The fact that we canā€™t express all semantics directly in the language is not an excuse for supporting the ability to express the semantics that are reasonable to support in the language. I hope we can agree on this point and the debate is focused around what is ā€œreasonableā€.

IMO, there is very little burden on users who donā€™t use this feature, only benefits for users who need it, and it sounds like @Karl has an implementation. The staunch opposition that arose today is rather confusing, surprising and disappointing to me.

I think this would be acceptable. Itā€™s inline with the spirit of most or all of the use cases I know of. One note: I think only the conformance declaration itself should be required to be stated with the protocol declaration while members implementing requirements could still be declared elsewhere. The same mechanism could also be used for internal and fileprivate protocols.

Sure, there are semantics expressed in the type system: that's precisely why protocols aren't just bags of syntax. Conforming to Error expresses semantics. And yes, names can imply semantics, and a name that implies unintended semantics would be rightfully judged a poor name. And yes, names are important. (But the compiler does not enforce the semantics of names. It's a pretty thin definition of "in the language" if the semantics implied in a method name is "in" but the semantics stated immediately above the declaration in a doc comment is "out.")

The opposition, I think, is borne out of the clarity that comes with sufficient discussion. We have learned that some of the proposed compiler benefits are not realizable. We are confronted with the challenge that there's an unclear division of labor between this feature and enums.

Circling back, the question is: what does supporting the expression of these specific semantics in this compiler-enforceable way, at the cost of a new addition to the language, gain users that isn't possible now? I would disagree that this is an end unto itself, and it's not clear to me anymore that there is much gained besides.

This, exactly this. We don't need language solutions to every problem where users (ab)use APIs by doing the wrong thing with them.

-Chris

7 Likes

What if "users" is not just "other teams using some library I developed" (for which I'm of course not responsible), but "my coworkers (some of which might be working on different subsystems) accidentally misusing a part of the code that I wrote as a separate module, and if they do it will affect me too"? I would say that, all else being equal, having the compiler being able to verify properties about your program should be preferable to having to rely on human judgement.

5 Likes

All else are not equal though, there is a cost for every feature added. This post applies equally well (if not more) to Swift:

https://graydon2.dreamwidth.org/263429.html

1 Like

The thread has moved on quite a bit, but it's worth noting that this is only true for protocol requirements without default implementations. It is not true for methods defined in a protocol extension (whether default or not). Protocol extensions provide a lot of power that enums never will. Even if N is relatively small the boilerplate for enums is significant compared to what is required for protocol extensions.

Which is one of the reasons it would be very nice to be able to use them with sealed protocols!

Would any of those opposed to this proposal possibly warm up to it more if we explore the "case class"-inspired direction @Joe_Groff pitched upthread? That would make exhaustive switch a lot more tractable. This would bring a significant new capability and open up new design options.

You can use default or wildcard patterns to cover enum cases in exactly the same way.

Not if you need data or primitive operations that are on the associated values (which can be made available using protocol requirements).

Those intermediate requirements then have to be described somewhere. With a switch you can do that with case .a(let x, let y), .b(let y, _, let x), ...:.

1 Like

Good point, I stand corrected. :slight_smile:

We have to draw the line somewhere. You are right that there are similar analogies in other parts of the language: For example, we require open to distinguish between methods that can be overridden and those that don't, precisely to avoid certain kinds of accidental API lock in due to "abuse". One could use my argument that such problems could be handled through comments, and many did during the discussion about open.

The difference in this case is one of scope and magnitude - there are just a lot more class methods in the world than there are intentionally "sealed" protocols.

OTOH, there are tons of other kinds of abuse (e.g. pre and post conditions) that we have no way to model and express right now, and there is a very general class of problems that can occur from that. I would argue that pre/post conditions are worth solving, because a well done feature could be a great expansion of the expressive capabilities of the languages and enable new classes of safer APIs entirely. Such a feature would bloat swift, but would also be widely applicable to a large range of problems.

My issue with sealed isn't just that it is bloat. It is also that the problem it solves is very very narrow and doesn't seem to cause big enough problems in practice. This is just a cost/benefit tradeoff, and very much MHO.

-Chris

8 Likes

Thanks for elaborating. If your perspective is not one of being opposed to sealed protocols, but more one of not viewing them as a priority I can understand that. Iā€™m not sure whether you have something along the lines of refinement types or more along the lines of design by contract in mind in the prior paragraph, but either way I agree that those would deliver a lot more benefit to a lot more people than sealed protocols (especially if they donā€™t deliver exhaustive switch).

Fair enough. Would you feel differently about the ā€œcase classā€ inspired design Joe mentioned above? That approach opens the feature up to being applicable in a much wider range of use cases because it would be useful for non-public protocols as well.

FWIW, I run into use cases that would best be solved by exactly this feature on a somewhat regular basis. That is why I have advocated strongly for it. There are times when both enums and protocols force a tradeoff that would evaporate if this tool was available. Even if it isnā€™t a priority right now, I do hope Swift will have something along these lines ā€œin the fullness of timeā€.

Firstly, apologies for not being so involved with the discussion. I've been ill and haven't been able to concentrate enough to read and consider all of the points.

Secondly, I welcome the scrutiny. I believe it's important that we thoroughly discuss anything that gets added to the language, and consider as many alternatives as we can think of.

So, even though it is not directly a part of this proposal, the most important thing this will enable in the future is non-public protocol requirements. That is a significant feature which would solve a lot of real-world problems in an elegant and straightforward way. To do that, we at least need a way to communicate across modules that external conformances are not supported.

Whether or not we allow more fine-grained access control (e.g. fileprivate) for protocol requirements is an open question which I don't want to get in to here. It deserves its own proposal and discussion.

I still believe that this proposal would enable optimisations. Perhaps not to the existential layout specifically, but there may be other operations where knowledge of what is (not) inside the box allows the compiler to omit handling certain kinds of structures which it knows it will never encounter. Some information is better than no information.

Yup, another hack. This is definitely an issue for those with advanced models involving value-types.

I'm not sure how anybody can look at this and be entirely satisfied that this is the ideal solution and we shouldn't even try to make it better/more straightforward.

We (the programmers) know that everything which conforms to Syntax should have the things inside _SyntaxBase, but the compiler doesn't know that, and needs to account for the possibility that somebody ignored the documentation. Would we not be able to generate better code by eliminating the dynamic downcasting and the possibility of trapping on every access?

Personally, I don't consider it to be a goal of this proposal. As I said before, if we allow exhaustive downcasting for protocol existentials, there is no reason why it shouldn't be extended to non-publically-subclassable base classes (or existentials involving them). So then we'd need to redesign public/open. I don't think it's worth it at all.

But it's still a good idea to discuss what the goals of this proposal are/are not.

That depends on your design. Many developers are more comfortable with class hierarchies because they learned Object-oriented programming. If your design makes heavy use of value-types or associated types, you might be using protocol-oriented programming.

Unfortunately this can become impractical because the language cannot express protocols like StringProtocol or Syntax which are only used for type erasure. We need to pretend at the language-level that external conformances are allowed, even when they are clearly not supported.

Better pre/post-conditions would be great, but they don't solve the same problems as sealed protocols. It is absolutely not just a bloat feature; please try and keep an open mind and not be overly dismissive. Protocol-oriented design is made significantly weaker than object-oriented design in practice, because of this need to superficially pretend that any public protocol can be conformed to.

2 Likes

I think the broader point though is that if it is an (eventual) goal for sealed protocols it might influence how they appear in the surface of the language, even if exhaustive switch is not part of the initial proposal. I am very happy to see movement on this topic, but would be disappointed if we designed ourselves away from the eventual ability to exhaustively switch over them.

Are you open to exploring the "case class" style surface syntax with that in mind? Or is it your opinion that it must be possible to declare conformances anywhere in the module (keeping in mind that we are only discussing the conformance declaration itself, not the implementation of the requirements)?

It's definitely possible, but then, it would be an even more significant departure from public/open.

I mentioned before that I think non-public requirements would largely obviate the need for that kind of downcasting. If you can think of some examples, I'm happy to re-evaluate.

EDIT: I think that listing the specific conformances would be good for something like @_frozen, though. Again, not part of this proposal.

Off the top of my head, maybe the simplest way to help the compiler the way "case class" style does would be to just say that conformances to sealed protocols must be declared in the same file as the protocol. That would be no additional syntax from what you have proposed and would give the compiler a complete view of conformances after scanning a single file. @Joe_Groff any thoughts on that approach?

I don't see why this proposal is needed to enable internal protocol requirements. If that is the significant feature being sought after, then we should do exactly that.