Pitch: Protocols with private fields

How can it?

Protocol just defines interface. What would be a meaning of an interface defined yet not accessible?

For an instance, this definition means that an adopter must implement foo, but allows adopting type to restrict accessibility just within file. Apparently it will be accessible within the protocol extension implementations.

protocol P {
    var foo: Int { fileprivateGet }
}

But what is the point to restrict access to foo in a conforming type, if anyone who has an access to the protocol definition knows how to access it?

extension P {
    var hijackedFoo: Int {
        foo
    }
}

This is why protocol requirements cannot have lover access level requirements than the protocol itself.


When it comes to common default implementation that needs fileprivate access levels, I'm afraid we are limited to do such implementation with all adopting types in that file.

This is an example how var foo: Int { get fileprivateSet } can be achieved:

protocol P {
    var foo: Int { get }
    func bar()
}

private protocol _P: P {
    var foo: Int { set }
}

extension _P {
    func bar() {
        foo = -1
    }
}

struct S: _P {
    fileprivate(set) var foo: Int
}

Now you can call S(foo: 0).bar() outside of the file and foo will be set to -1 via default implementation.

Perhaps you misunderstood my pitch.

First, let me stress that I agree with your premise:

We disagree on the conclusion:

Currently, a protocol prescribes that all its requirements have at least the same access level as the whole conforming type. You can (although that's probably useless) implement the requirement with a higher access level. So, in fact, requirements have a lower bound defined by that of the protocol's access level.

The heart of my pitch is to let allow protocols to specify other lower bounds on individual requirements.

So, in your example, fileprivate would have a different meaning than in the context of a standard type declaration. It would indicate that conforming types should at least provide a fileprivate implementation of that requirement. Consumers of the protocol would not be allowed to expect the requirement to be visible, unless they are declared in the same file as the protocol.

// in P.swift
public protocol P {
  var foo: Int { get fileprivate(set) }
  var bar: Int { get private(set) }
}

// In S.swift
public struct S: P {
  public internal(set) var foo: Int
  public fileprivate(set) var bar: Int
}

// In main.swift (same module)
let s = S(foo: 1, bar: 2)
print(s.foo) // OK, getter is public
s.foo = 3    // OK, setter is internal
s.bar = 3    // Error, setter is fileprivate

let p: P = s
print(p.foo) // OK, getter must be at least public
p.foo = 3    // Error, getter might be as low as fileprivate
p.bar = 3    // Error, getter might be as low as private

Notice that the conforming type is allowed to choose the access level with which it wants to expose its requirements, as long as they are higher than what the protocol prescribes.

internal protocol Q {
  var foo: Int { get set }
}

// The extension is well-typed because S chose to expose
// foo's setter as internal.
extension S: Q {}

What the conforming type cannot do is to implement the requirement with a lower access level than prescribed, as it would violate the assumptions that consumers of the protocol can make.

struct T: P {
  private var foo: Int // Error, `foo` must be a least fileprivate
}

I believe that we could encapsulate behavior in default implementations using that approach. I provided an example in the original post. I'll add another based on your example as template:

// in Q.swfit
public protocol Q {
  var foo: Int { private(get) private(set) }
}

extension Q {
  public mutating count() -> Int {
    foo += 1 // OK, a type can always access its own
             // properties regardless of their access level
    return foo
  }
}

// In U.swift
public struct U: Q {
  // foo is invisible to the consumers of this type
  private var foo: Int = 0
}

// In main.swift
var q: Q = U()
print(q.foo)     // Error, getter might be as low as fileprivate
print(q.count()) // OK, prints 1
print(q.count()) // OK, prints 2

Oh sorry, I can see it now. Yeah... I think I like it!

Could you accomplish the same things with scoped conformances?

There is definitely some overlap, but I think the goals are a bit different.

IIUC, scope conformance would only allow to provide a conformance that does not need to be exposed outside of an access' boundary. However, I would like to expose the conformance, only without having all internal details exposed (in particular w.r.t. mutation) at the same level.

One way to illustrate is to think of an AST library. Inside the library is defined a visitor protocol whose default implementation just walks an AST and calls one or several methods to interact with the visited nodes. Clearly, consumers of that library may need access to such a protocol, but they might not be interested in the internal shenanigans that the library does to configure the state of the walker.

In that specific example, maybe we can achieve a similar design with scoped conformance using two protocols. One Walker protocol describing the general public API and another _WalkerImpl protocol to deal with the "internal shenanigans". Specific walkers would conform to _WalkerImpl internally and expose their conformance to Walker.

In a more general setting, though, I think that requirement bounds are more flexible. The problem with _WalkerImpl is that it would be internal, preventing consumers from inheriting default implementations defined over there. If it was also exposed, then we would loose the advantage of trying to encapsulate behavior in the first place.

Further, requirement bounds might be slightly simpler. In particular, they would not change Swift's current conformance resolution strategy, AFAICT, and the dynamic example from the generic manifesto would not require the user to "think in scopes" to build their own mental model of how dispatch should behave.

That being said, scoped performance would have one advantage over my approach: the ability to actually scope the conformance itself and the associated benefits of that feature :sweat_smile:.

Sure. I just wonder if the same thing might not be accomplished by composing public protocols, but using one private or internal conformance. What I have in mind is,

// Module A
public protocol X { ... }
public protocol EasyXImpl {}

extension X where Self: EasyXImpl {
  // implementations of X requirements in terms of EasyXImpl requirements
}

// Module B
import A
public struct Y: X private EasyXImpl {
  // EasyXImpl requirements
}

That would have a few advantages:

  1. The protocol system would retain its simplicity
  2. A protocol would have a single, well-understood meaning that doesn't change across access levels
  3. We'd avoid adding another language feature, since I believe we need scoped conformances anyway.
  4. You'd still have the option to create X conformance without EasyXImpl. In fact there might be several EasyXImpl variants for different means of achieving that conformance.
1 Like

No, unfortunately scoped conformances won't help. They allow to limit visibility of a conformance. The problem we are talking about is quite orthogonal to this, we want to limit visibility of a required property but expose it to default implementation.

Actually, I think this use case would be served quite well with scoped conformances, and I agree with @dabrahams that scoped conformances would be a more expressive feature that additionally enables other use cases.

Here, you’d have the public API guaranteed by protocol P, and then the implementation-only property would be a requirement of a distinct protocol Impl: P to which the same type would have a scoped conformance, and the default implementation of P’s public requirements would be implemented in extensions of P where Self: Impl.

1 Like

I don't see it there. Can you please rewrite last example from this post to work with scoped conformances?

It's exactly as @dabrahams has just outlined above:

public protocol Q {
  mutating func count() -> Int
}

protocol QImpl { // Optionally, QImpl may refine Q.
  var foo: Int { get set }
}

extension Q where Self: QImpl {
  public mutating func count() -> Int {
    foo += 1
    return foo
  }
}

public struct U: Q, private QImpl {
  fileprivate var foo: Int = 0
}

Not quite: in the original outline from @dabrahams, his EasyXImpl (your QImpl) is public. This is a notable difference. The pitch is about private implementation details.

Can scoped conformances deal with an EasyXImpl / QImpl protocol which is not public? If not, could they become able to do it?

2 Likes

Exactly. Only if adopting a protocol with scope means also to override the scope for requirements, then scoped conformances can solve this.

Actually I'll side with @dabrahams and @xwu on that matter. Their example is indeed one way to implement my use case. Although QImpl has to be exposed, crucially, the implementation details brought by QImpl are not exposed by the conforming type.

The approach proposed in this pitch is to completely encapsulate an implementation from the outside. From the perspective of a type consumer, though, that's a distinction without a difference. Further, as suggested by @dabrahams, scoped conformance would allow a library to vend multiple implementations for the same protocol.

// In module A
protocol P {}
protocol PImplA: P { ... }
protocol PImplB: P { ... }

// In module B
struct S: P, private: PImplA { ... }
struct T: P, private: PImplB { ... }

Here, both B.S and B.T conform to A.P, but they use two different implementations defined in module A.

I disagree with @dabrahams on a couple of points though:

Scoped conformance have their own subtleties. For instance, dynamic dispatch resolution is a rather significant one. In particular, I am worried about that sentence:

The post ends with a handful of other difficult questions. Generally, it seems like dealing with multiple conformance and conformance inheritance will compel us to come up with new rules.

Although I don't claim such mechanisms cannot be designed or understood, having formalized and partially reimplemented Swift's current method resolution I'd argue that it's definitely not "simple".

My pitch does not challenge that assertion. A protocol would describe a single, well-understood API with the addition be that access levels would be part of that API.

While I adhere wholeheartedly to the principle of "economy of concept", I'd like to point out that the setup for implementing my use case with scoped conformance is more involved and feels a little like a "trick".

I'd be interested to discuss things the other way around. What use case solved by scoped conformance cannot be solved by access lower bounds?

1 Like

I see this as a very useful part of scoped conformances.

But regardless whether the implementation protocol definition is public, its interface is not exposed publicly as part of the type (in this example, U), fulfilling the stated design goal of not exposing implementation details in the type’s public interface. Indeed, if it were possible to declare a private member of a public protocol, the declaring protocol is public anyway.

I thought this pitch is about protocol requirements with lover access level than the protocol itself so a default implementation can access a fulfilment without that fulfilment being exposed. I'm having troubles to follow informal language sentences here. Can we express ideas to solve the original problem in Swift? For instance in the example from @xwu:

Do we know if this fileprivate foo fulfilment in U is legit in private conformance of internal QImpl's requirement? Will this compile? It seems to me that this will decide if scoped conformances solves what this pitch asks for.

1 Like

Allowing the protocol’s requirements not to be exposed publicly is precisely the definition of a scoped conformance. If this example does not compile (as it currently does not), then Swift does not support scoped conformances.

Whoops it looks like I missed these...

(where Wrapper was internal in a previous figure)

and


Initially, I understood that protocol requirements must be fulfilled with protocol's access level (like today), and the scoped conformance just limits visibility of that conformance. Which fortunately seems (based on quotes above) to be incorrect.

I only made it public to demonstrate its use across modules: a module can publish a helper protocol that encapsulates common patterns for implementing its other protocols.

Can scoped conformances deal with an EasyXImpl / QImpl protocol which is not public?

Certainly.

public protocol X { ... }
private protocol EasyXImpl {}

extension X where Self: EasyXImpl {
  // implementations of X requirements in terms of EasyXImpl requirements
}

public struct Y: X private EasyXImpl {
  // EasyXImpl requirements
}

What about that sentence made you worry? We were not wedded to that particular semantics, but it seemed logical enough to us.

The post ends with a handful of other difficult questions.

Well, I still disagree with that assertion; as the proposal says:

Generally, it seems like dealing with multiple conformance and conformance inheritance will compel us to come up with new rules.

I don't think it increases our obligation to come up with rules, because as noted in that thread we have all the same issues today in a slightly different form.

Although I don't claim such mechanisms cannot be designed or understood, having formalized and partially reimplemented Swift's current method resolution I'd argue that it's definitely not "simple".

Ah well, I never said requirement resolution was simple.

My pitch does not challenge that assertion. A protocol would describe a single, well-understood API with the addition be that access levels would be part of that API.

This depends on your point of view I guess. The way I see it, each access level exposes a different API.

That's an interesting question. Would these lower bounds solve Doug's problem or address the metadata and witness table bloat issues for which scoped conformances were originally designed? Another use case: two different modules want a conformance of public type Int to public protocol C.Monoid, but one module wants to use the (+, 0) monoid and the other wants to use the (*, 1) monoid.

I use Protocols and extensions to form part of a refactor, where you want to hide the actual protocol implementation from the outside world. I also use it to enforce implementation requirements on a class.

Ie, a certain parameter needs to exist in the class, but you want its access to the outside to be hidden.

The only way this can be done currently using using class inheritence. But it seems like this is not the way of the future for protocol oriented programming.

I believe that protocols are great for abstractions like lists, but not so useful in the real world when dealing with less abstract concepts like specific view/data transforms and implementations.