Introducing role keywords to reduce hard-to-find bugs

Evolution Pitches
81

Right, that's an added bonus for this approach!

I still think default implementations should be able to be written within the protocol definition itself, but that's mostly separable.

Doug

4 Likes
82

I think there are many ways that protocols could be fixed up. I will outline my proposal below, since I understand its details, but I accept there are many other possible solutions.

My suggestion is that in the same file as the protocol you can add extensions much as at present. However if the protocol is implemented in an extension then the override keyword is needed. In other files, regardless of module, you can only add final extensions. Final extensions cannot be overridden.

Common usage of protocols using my proposal is demonstrated below.

// Protocol and extension in same file.
protocol CommonUsage {
    func foo()
    func bar()
}

extension CommonUsage {
    func override foo() { ... } // *Must* add `override`. BREAKING CHANGE.
    func baz() { ... } // Behaves as though `baz` was declared in protocol and then overridden (like `foo`). BREAKING CHANGE.
}

// In seperate or same file (could be `class/struct/enum`).
struct CU: CommonUsage {
    override func foo() { ... } // *Must* add `override`. BREAKING CHANGE.
    override func bar() { ... } // *Must* add `override`. BREAKING CHANGE.
    override func baz() { ... } // *Must* add `override`. BREAKING CHANGE.
}

The use of override is made consistent, if you are implementing something already declared, implemented or not, then you add override. This catches typos and accidentally orphaning an implementation when refactoring.

baz behaves like foo, it is exactly the same as if baz was declared in the protocol. This brings consistency to how protocol methods dispatch, they are always dynamically dispatched. Currently overridden, extension-only methods can dispatch differently (statically), and surprisingly, from those that have a protocol declaration.

Outside of the protocol’s declaration file an extension must be final, which as expected means it cannot be overridden.

// Not in same file as protocol declaration.
extension CommonUsage {
    final func qux() { ... } // Outside of declaration file *must* add `final`. BREAKING CHANGE.
    // Cannot `override` an existing func, e.g. `foo`.
    // Cannot implement `bar`, only add new `final` funcs. BREAKING CHANGE.
}

Since the method, qux, is final and therefore cannot be overridden there is no ambiguity as to which implementation will be called (unlike at present where the declared type determines if it is a static or dynamic dispatch).

Also you cannot implement an existing abstract func with a protocol-extension outside of the file the protocol is declared in. This ensures same method is called regardless of type (dynamic dispatch only).

Retroactive conformance is popular in Swift and is therefore important to support:

// Existing `class/struct/enum`.
struct Existing {
    func foo() { ... } // Note same name, `foo`, as in `CommonUsage` protocol.
}

// Retroactively add `CommonUsage`.
extension Existing: CommonUsage {
    retroactive func foo() // Need to be explicit that you are retroactively adding conformance of a method (cannot have an implementation, calls `Existing.foo`). BREAKING CHANGE.
    override func bar() { ... } // *Must* add `override`. BREAKING CHANGE. 
}

Requiring retroactive catches typos and orphaning due to refactoring.

Thus the above suggestion catches the errors identified by Oliver et al. and ensures that there is clarity about which method is called, regardless of declared type, and still allows retroactive conformance.

As I said above there are other methods of achieving ‘sane’ protocol behaviour and still retaining the power. I think the main thing to achieve is to move away from the existing behaviour which is a ‘bug-factory’.

2 Likes
83

I have used a language that tried to catch near miss typos with a warning, Mathematica. They persisted with refining the warnings over many years by refining the heuristics used. However it was a largely hated system and eventually turned off by default. Hopefully Swift’s version will fare better :).

84

For anyone still following along:

85

The new draft looks much improved to me. It looks like this is heading in the right direction. Thanks for tackling this important topic!

I have a few suggestions to consider (I apologize if any of these have already been discussed):

  • The proposal could also be strengthened by covering other kinds of protocol requirements, especially associated types. Are you proposing to allow the role modifiers on typealiases and nested type declarations? I suspect so but the text of the proposal doesn't make this completely clear.

  • It should be possible to specify a default modifier on the protocol extension itself, just as we can with access control modifiers. This will help to reduce the syntactic weight of the feature while still delivering the same semantics.

  • It would be interesting to consider exploring a name deriving from satisfies rather than default for a couple of reasons. A name in this direction could also be used to members of concrete types that satisfy protocol requirements. We could also explore parameterizing a name derived from satisfies to specify the requirement that is satisfied, i.e. @satisfies(ExampleProtocol.thud(with:)). This would allow us to be explicit in concrete conformances when necessary (as is possible in languages like C#).

  • The interaction of this use of extend with the existing notion of extension seems ripe for confusion to me. I don't have a specific suggestion for improving it but I would like to see some further bike shedding happen before this proposal is ready for review. I think (hope!) we can come up with something better.

86

I would advocate against it because I think these convenient features made the language worse and less extensible in that area. I clearly understand that this feature reduces syntactic weight but it creates just another source of confusion and bugs (I recall the access modifier issue on extensions that we had over and over). Furthermore not every extended member is a default implementation. An access modifier for instance on an extension can be overriden by a different modifier, which IIRC can only be less visible than the one before the extension itself. I think it does not make any sense to allow to override default with extend or the other way around.

If we're going the optional route then I would prefer an optional attribute (like @default) instead of a keyword (default). This simply asks the compiler 'can you check if this extension is a true default implementation of a protocol requirement?'. In that sense there isn't really a need for extend and it also eliminates the idea of prefixing the extension itself with @default.

My personal wish would still be a single mandatory default keyword in extensions for default implementations.

87

@ attributes are for low-level implementation details (like @convention(c), @_fixed_layout, @inline, @_specialize, etc etc)

88

Can you elaborate? I haven‘t seen a rule that specified that. @discardableResult is not low-level for sure.

89

I wouldn't say @noescape or @autoclosure are "low-level". They effect the semantics of things.

2 Likes
90

tru i forgot about that one

there’s still the problem where @discardableResult is required to supress compiler warnings while @default would be absolutely optional. that means that some code may not use it at all leading to dialect fragmentation. but that would also be a problem with default anyway

91

Exactly, that‘s why I’m still in favor of the mandatory single keyword despite the source breakage (but that‘s just my wish thinking).

An optional @default (or what ever this attribute should be called like) reminds me somehow of the optional @Override attribute in Java which is ugly in the first place.

92

The only issue I see with this approach is that it feels redundant if you‘re trying to resolve a naming collision between two different protocols.

  1. You‘re asking the compiler to verify the default implementation.
  2. You‘re telling the compiler the protocol requirment and you still have to provide an alis for the colliding member.
@satisfies
func ExampleProtocol.thud(with: ...) { ... }

@satisfies
func OtherProto.thud(with: ...) { ... }

This is still shorter.

Out of curiosity: is it possible to make default an optional keyword in Swift 4.2 and emit a warning and then in Swift 5 make it a mandatory keyword for default implementations and emit an error if it‘s missing?

In Swift 5.1 we could move default implementations into protocol declarations as pure sugar change for ergonomics.

93

this seems like the best course of action

can someone like explain why this can’t be done right now? it sounds like a purely syntactical thing

94

I think it can be done, it‘s just my personal opinion that it would be safer if we had a required keyword first. Then the compiler would desugar it to the default prefixed member extension and a protocol requirement without default since it only should be part of the default implementation and not the requirement itself. Therefore implementation in the protocol body does not need an extra keyword while a default implementation in an extension would still require the keyword.

95

I find the ability to say private extension to be quite useful. Are you suggesting you would prefer if that were not allowed?

I don't think allowing an explicit modifier to be provided at the site of an individual declaration would be harmful. The semantics would be more straightforward than the semantics of access control in any case.

96

If we were to go with your syntax for specifying the satisfied requirement I don't think we would need the attribute. The reason for my suggestion was that it allows the name of the member to be different than the name of the requirement in the protocol. This might avoid the need to write a forwarding wrapper in some cases. I would be happy either way.

The main point I was making is that it may be a good idea to look for a keyword that can also be used in conformances provided by concrete types. default is clearly not adequate for in that context.

97

Well I apologize that I messed up the proposal to remove the access modifier on extension back then. I completely underestimated my knowledge of that topic in the past and screwed up the proposal. I wish I didn't and that we got rid of access modifiers in extension for Swift 3, but that ship has sailed (or can we theoretically deprecate that functionality in the future?).

Don't get me wrong, I understand your point that for some people it's still a useful convenient feature. In my codebase however it's completely banned with a linter rule.

Let's say we would go for a 'keyword' based solution, and let's still use extend in the following example.

protocol SomeProto {
  var foo: Int { get }
  var bar: Int { get }
  var something: Int { get }
}

default extension SomeProto {
  var foo: Int { ... }
  var bar: Int { ... }
  extend var baz: Int { ... }
  var something: Int { ... }
  extend var somethingElse: Int { ... }
}

extend extension SomeProto {
  default var foo: Int { ... }
  default var bar: Int { ... }
  var baz: Int { ... }
  default var something: Int { ... }
  var somethingElse: Int { ... }
}

I think this is over engineered and just a new source of confusion. Furthermore the imported API won't look like that anyway, because even access modifiers are not being printed before the extension when viewing the API only the original source code will.

Quite some time ago I had some idea about a new attribute/keyword which must be written before the declaration of a type - (however I don't remember what it was anymore, or the context of that idea) - but then there was an issue because of an other convenient keyword indirect in front of an enum. To be honest indirect isn't even used that much in that fashion, but it was included for the same reasons we have access modifier in front of an extension. This convenience made my idea completely impossible and not even worth a pitch.

Another argument against such convenient features is actually an argument that was used against a pitch I posted long time ago to the mailing list.

Back then I pitched something like scoped (named or anonymous) group with access modifiers.

struct Foo {
  private group name {
     var foo: Int { ... }
     ...
  }
}

An argument against it was something similar to this, but is just nit-picking from my perspective:

  • "We would have to scroll up to fine the group to check the access modifier"

In that sense I can say exactly the same against the convenience features on extensions and the indirect keyword, but here for some reason no body would even care.

Long story short, I personally dislike it, but others don't have to.

98

I din't thought about that when I posted my reply. I get the idea behind it, but I think it really isn't a better alternative because choosing name can be hard and it will also force you to choose carefully so that you don't create collisions for your API if you want to extend / maintain it in the future.

Bikeshedding by using : instead of a dot.

// Dealing with the collision
func SomeProto:foo(with bar: ...) { ... }
func OtherProto:foo(with bar: ...) { ... }

// disambiguating
instance:SomeProto:foo(with: ...)
instance:OtherProto:foo(with: ...)

Different syntax:

// Dealing with the collision
func <SomeProto>foo(with bar: ...) { ... }
func <OtherProto>foo(with bar: ...) { ... }

// disambiguating
instance.<SomeProto>foo(with: ...)
instance.<OtherProto>foo(with: ...)

It will only work if it's completely optional right? Because if it were a required keyword than it won't work because of the retroactive conformances.

99

Right, but it could still catch some mistakes. If we're going to have syntax for this we might as well make it available to concrete types as well.

100

Erica, thank you for continuing to revise this proposal and keeping up the discussion of this issue. It's one that's important to me, although I have a different (personal!) opinion on what the solution should look like.

I'd like to recast my earlier comments in this thread as a specific response to your latest proposals, so we have something more concrete to discuss.

Personally, I don't feel that the default modifier is the right solution for clearly separating default implementations from other functionality. Per the proposal, it would be used (e.g.) like this:

public protocol ExampleProtocol {
   func thud(with value: Float)
}

extension ExampleProtocol { 
    public default func thud(with value: Float) { ... }
}

The role of default here is to link the method in the extension with the requirement in the protocol. However, the common case here is that the protocol and its default implementations are defined in the same module, so in that sense there's a lot of boilerplate. The generics manifesto suggests that simple defaults like this could be written within the protocol itself:

public protocol ExampleProtocol {
   func thud(with value: Float) { /* default implementation */... }
}

This approach makes it clear that the default implementation is not an entity that's separate from the requirement, but a way to implement the requirement. It's less verbose than the default solution, and it mirrors the approach already used by associated types, e.g.:

public protocol HasAssoc {
  associatedtype Assoc = Int
}

However, this doesn't cover all of the cases that the proposed default covers. Specifically, one can have a default implementation in another module, or which is conditional on other requirements. The latter case is the interesting one, because it's something like:

import ExampleProtocolModule

protocol MyProtocol {
  func thunk(with value: Float)
}

extension ExampleProtocol where Self: MyProtocol {
  // default implementation of ExampleProtocol's thud(with:) using MyProtocol's thunk(with:)
  func thud(with value: Float) {
    thunk(with: value)
  }
}

This is an important case for a couple reasons. It seems really important to do something here, because there's no way to tell that's a default implementation. However, I don't think the default modifier handles this case well, because it doesn't feel specific enough: presumably it's a default for ExampleProtocol because that's the type we're extending, but what if some other protocol also had a thud(with:)? Would it act as a default for that protocol, too?

The suggestion I made about qualifying member names covers this case in a manner that is more verbose but (IMO!) more clear:

extension ExampleProtocol where Self: MyProtocol {
  func ExampleProtocol.thud(with value: Float) {
    thunk(with: value)
  }
}

I won't rehash the discussion that followed my post, but this approach allows for disambiguation amongst different protocols that have same-named requirements and doubles as a way to express what the proposed required modifier (from your follow-on proposal) describes.

One last comment w.r.t. default. It's not clear how default functions work outside of protocol conformance checking. For example, given this code, is the function f well-formed or is the default function not callable from generic code?

extension ExampleProtocol where Self: MyProtocol {
  default func thud(with value: Float) {
    thunk(with: value)
  }
}

func f<T: ExampleProtocol & MyProtocol>(_ t: T, f: Float) {
  t.thud(with: f)   // can I call a default function?
}

I think I want this code to be ill-formed, because default implementations should be just that---defaults, which are ignored when not needed.

Cheers,
Doug

5 Likes