Amending SE-0444: Exempt default implementations of protocol requirements from member import visibility rules

Evolution Discussion
1

SE-0444 (Member Import Visibility) addresses the long-standing inconsistency where member declarations from transitively-imported modules are in scope in a source file even though top-level declarations from those same modules are not. The proposal fixes this by requiring that the defining module of any referenced member be directly imported. This behavior is available today behind the MemberImportVisibility upcoming feature flag and will become the default in a future language version.

After the initial implementation and adoption, library owners discovered a source-compatibility issue affecting the interaction between MemberImportVisibility and protocol conformance checking. This post proposes a targeted amendment to SE-0444 to address it.

The Problem

When checking whether a type satisfies a protocol's requirements, the compiler searches for witnesses of those requirements — either an explicit implementation in the conforming type, or a default implementation defined in a protocol extension. Under SE-0444, the compiler now applies MemberImportVisibility restrictions to the name lookups for these witnesses. This turns out to be source-breaking for a fundamental protocol evolution pattern, which is adding a new requirement to an existing protocol at the same time as providing a default implementation for that new requirement.

The issue surfaced concretely in the swift-foundation and swift-crypto packages. FoundationEssentials declares the ContiguousBytes protocol and, in a recent revision, added a withBytes requirement along with a default implementation:

// Module: FoundationEssentials

public protocol ContiguousBytes {
    func withUnsafeBytes<R>(_ body: (UnsafeRawBufferPointer) throws -> R) rethrows -> R
    func withBytes<R, E>(_ body: (RawSpan) throws(E) -> R) throws(E) -> R // New requirement
}

extension ContiguousBytes {
    // Default implementation of new requirement
    public func withBytes<R, E>(_ body: (RawSpan) throws(E) -> R) throws(E) -> R { ... }
}

In swift-crypto, MD5Digest conforms to ContiguousBytes transitively through an internal protocol hierarchy. Crucially, the file declaring MD5Digest does not import FoundationEssentials. Instead, the relevant import lives in a separate file that defines the intermediate protocols:

// File: Digest.swift (imports FoundationEssentials)
import FoundationEssentials

public protocol Digest: ContiguousBytes { ... }
internal protocol DigestPrivate: Digest { ... }

// File: MD5.swift (does not import FoundationEssentials)
struct MD5Digest: DigestPrivate {
    func withUnsafeBytes<R>(_ body: (UnsafeRawBufferPointer) throws -> R) rethrows -> R { ... }
    // withBytes() is satisfied by the default implementation in FoundationEssentials
}

With MemberImportVisibility enabled, the compiler checks whether the default implementation of withBytes is visible in MD5.swift. Since FoundationEssentials is not directly imported there, the conformance is rejected:

error: type 'MD5Digest' does not conform to protocol 'ContiguousBytes'
note: instance method 'withBytes' used to satisfy a requirement of protocol 'ContiguousBytes'
      is not available due to missing import of defining module 'FoundationEssentials'

swift-foundation had to revert the addition of the withBytes requirement from ContiguousBytes entirely to avoid breaking swift-crypto and potentially other clients. This is a significant regression: it should be possible to add protocol requirements with default implementations to a library without requiring source changes in clients, but with the current rules of MemberImportVisibility this cannot be guaranteed.

The goal of SE-0444 is to give developers explicit control over which modules' members are visible in a source file, preventing ambiguities and implicit dependencies from arising due to transitive imports. With that goal in mind, it generally makes sense to restrict protocol requirement witnesses to members that have been directly imported. This prevents conformances from implicitly creating dependencies on transitively imported modules. However, an exception should be made for the default implementations that are conceptually "inherited" by a conformance declaration.

Proposed Amendment

SE-0444 should include an explicit sub-section on conformances and default implementations:

Detailed design

...

Protocol Conformances

Types conforming to a protocol must have a witness for every requirement declared by that protocol. Under the rules of this proposal, a member that is considered to be a viable witness for a protocol requirement must be visible from the source file that declares the conformance. However, an exception must be made for a default implementation of a requirement so long as it is declared in a protocol extension in the same module as the extended protocol. This exception is necessary to give library owners a way to preserve source compatibility when adding new requirements to protocols.

5 Likes
2

FYI the issue link doesn't actually link to anything.

1 Like
3

Thanks, fixed.

4

The exception described in the second quote is (much) wider than the exception described in the first quote. The former seems clearly motivated by and tailored to the problem you describe; I worry the latter will get us back to a place where we might find new regrets and further cycles of amendments...

To be concrete:

// File: Digest.swift (imports FoundationEssentials)
import FoundationEssentials

public protocol Digest: ContiguousBytes { ... } // (1)
internal protocol DigestPrivate: Digest { ... } // (2)

Your first articulation of the exception would suggest that we should have member import visibility treat the default implementation of withBytes "as though" it were implemented at (1)—well, an extension thereof, but I'll simplify the wording—with effective visibility public and implemented at (2) with effective visibility internal. Doing so is consistent with the overall semantics of DigestPrivate refining Digest refining ContiguousBytes.

Your second articulation of the exception would seem to require all default implementations for ContiguousBytes declared in FoundationEssentials to be visible in any file of a package that imports FoundationEssentials (in this case, swift-crypto) regardless of what that package actually uses—even if, say, the package never declares any refinements of ContiguousBytes.

5

My second articulation is an attempt to add necessary precision to the plain language of the first articulation, since I didn't think that "inheritance" was necessarily a well-defined term for this purpose. I'll wordsmith it to whatever is needed to accurately describe the narrow carveout. However, I'm not sure I understand the distinction you're making. The sentence you called out is in the context of a paragraph that is specifically about satisfying protocol requirements. Do you think it needs more verbiage to make that clearer?

6

I've updated the language to try to make my intent clearer, and also opened a pull request for this amendment:

Protocol conformances

Types conforming to a protocol must have a witness for every requirement declared by that protocol. Witnesses to protocol requirements can be members that have been imported from other modules. Under the rules of this proposal, members must be visible in a source file in order to be discovered as viable witnesses of protocol requirements. However, a default implementation that is not visible from a source file may still satisfy a protocol requirement as long as it is defined in the same module as the extended protocol it belongs to. Suitable default implementations are inherited by conformances that do not provide an implementation of the requirement, rather than being looked up from the source file. This nuance is necessary in order to give library owners a way to preserve source compatibility when adding new requirements to protocols:

// External module: A

public protocol P {
  func existingRequirement()
  func newRequirement()
}

extension P {
  public func newRequirement() // default implementation
}

// File: Q.swift (imports A)
import A

internal protocol Q: P { }

// File: S.swift (does not import A)
struct S: Q {
  func existingRequirement() { ... }
  // newRequirement() is satisfied by inherited default implementation from A
}
1 Like