Extensions to private nested types (illustrated with CodingKeys)

Using Swift
1

Hello,

The Codable protocol generates coding and decoding methods when possible, as well as a private CodingKeys enum. This enum contains very precious compiler-generated and safe string information that some applications or libraries would like to use.

But the generated CodingKeys can not be extended (as far as I know):

struct Player: Codable {
    var name: String
    var score: Int
    
    // error: declaration is only valid at file scope
    extension CodingKeys: SomeProtocol {}
}

// error: 'CodingKeys' is inaccessible due to 'private' protection level
extension Player.CodingKeys: SomeProtocol {}

As far as I know, there is only one way out: declare the extension when the private type is defined:

struct Player: Codable {
    var name: String
    var score: Int
    
    // OK
    private enum CodingKeys: CodingKey, SomeProtocol {
        case name, score
    }
}

In an ideal world, I could write:

struct Player: Codable {
    var name: String
    var score: Int
    
    private extension CodingKeys: SomeProtocol {}
}

Did I miss something? Do we plan to let the above sample code compile eventually? Is it worth pitching nested extensions?

2

This makes sense for cases like the following. But I also have the impression this would significantly extend the diagnostics engine. In what way – regular or implying performance issues – is a good question to a knowing person.

class A {}

class Foo {
    fileprivate struct B {} 

    fileprivate extension A {
         func foo() -> B {}
   }
}

I assume you understand that you are declaring a different CodingKeys here, which has nothing to do with the inaccessible eponymous private type: the original CodingKeys is private for your scope, not in your scope. So it can't be extended even if you were able to private extension CodingKeys {.

3

Anthony, reading your answer, I fear that I have failed expressing myself with enough precision. The CodingKeys example brought its own context, and has surely polluted my question. Please let me try again.

A private nested type has to be fully defined in one shot. You currently can't define it gradually with extensions (unless I'm mistaken):

// OK
struct A {
    private struct Inner: P {
        // properties...
        // methods...
        // P adoption...
    }
}

// Not OK
struct A {
    private struct Inner {
        // properties...
    }
    
    extension Inner {
        // methods...
    }
    
    extension Inner: P {
        // P adoption...
    }
}

My question is thus whether it is worth pitching for the above construct (we'd then enter detailed discussions about scopes and access modifiers, for all relevant combinations of open/public/internal/private nested types/extensions/declarations).

Such a pitch would require some motivation.

So far, I could only motivate extensions to private nested types based on syntactic preferences: some people prefer to declare their types gradually, grouping related features in coherent extensions.

Syntactic preferences are a weak motivation.

The CodingKeys example brings a stronger motivation. As all private nested types, it can't be extended in any way, and thus has to be fully defined in one shot. But then we have to provide all coding keys by hand, and lose the advantages of the code generated by the compiler:

struct Player: Codable {
    var name: String
    var score: Int
    
    // OK
    private enum CodingKeys: CodingKey, P {
        case name, score // <- meh
        // methods...
        // P adoption...
    }
}
4

Ah, so CodingKeys is generated in the conformance scope at compile-time. Excuse the misunderstanding.
Syntactic preference is surely not a trifle, and I believe it should stand on par with compiler generated code simply because it is compiler magic.

Apart from nested types, nested extensions of file scope types can also be useful and have a stronger motivation precedent:

Again, it would be good to hear about the possible impact this can have on the type checker and diagnostics engine, which is crucial to whether the proposal has a chance of being accepted.

5

I guess @itaiferber would say that it is not that magic. An explicit hand-crafted CodingKeys nested type, as I showed above, is actually used by the Decodable.init(from:) and Encodable.encode(to:) generated by the compiler.

But I agree that if the generated CodingKeys is the only type that can't be extended, this pitch-of-a-pitch won't go far.

All right. But is it the same topic? Do you think it includes my own little needs as a particular case?

6

Indeed, it's not that magic :slight_smile: There should be no difference here between letting the compiler generate the CodingKeys for you versus writing them yourself manually; when you let the compiler generate the CodingKeys for you, it inserts values into the AST to generate the equivalent of what you would have typed in manually. The following are equivalent:

struct S1 : Codable {
    let var1: String
    let var2: Int
    let var3: Double
}

struct S2 : Codable {
    let var1: String
    let var2: Int
    let var3: Double

    private enum CodingKeys : String, CodingKey {
        case var1, var2, var3
    }
}

(In both of these implementations, the compiler goes on to use the CodingKeys enum [whether manually written or synthesized] to implement init(from:) and encode(to:).)

The only difference is that when you write them yourself, you control the access level, as you originally point out; that being said, this can indeed be an inconvenience. The motivation behind giving CodingKeys a private access level is that they are implementation detail that no one outside of the type should rely on (by default, at least; you're welcome to expose it yourself if you really want to for some reason).

However, private does indeed in this case hermetically seal the type. There was discussion about exactly this issue elsewhere that unfortunately, I can't seem to dig up anywhere (either on the forums or on JIRA), but if I find it I will link to it; since extensions can only come at file scope, there isn't a way to extend this type from the outside.

And while I agree that this case is pretty rare (generally speaking, many folks would either write a private type with all functionality built in [i.e. no `extension`s], or they'd write a fileprivate type in order to extend it), I don't think this should hold back a proposal to address the underlying issue. I think more types could stand to be private rather than fileprivate were this changed.

I think that it is. There's no reason in my mind to distinguish between private and fileprivate types here; in fact, there is more of a motivating case for private over fileprivate, since extensions of fileprivate types can come at the file scope:

struct Outer {
    fileprivate struct Inner {}
}

fileprivate extension Outer.Inner {
    func foo() {
        print("Foo")
    }
}

There are largely two ways to solve the issue at hand here (for CodingKeys types):

  1. Have the compiler synthesize the type at fileprivate access level (and use the existing mechanisms to extend the type)
  2. Allow extensions in inner scopes

(1) is especially problematic because of a use-case that many folks forget about: inheritance. If you write a Codable subclass of a Codable superclass, you should neither need nor want access to the superclass's CodingKeys, which you would get if they were fileprivate and you wrote both classes in the same file. Worse, it would be easier to accidentally inherit the superclass's CodingKeys by forgetting to write your own.

I think that it should be easy enough to find other cases of manually-written fileprivate nested types which have extensions at file scope which could be re-written as totally private types were extensions allowed in inner scopes.

7

Yes, your example is a special case. It might be reasonable to launch a pitch that addresses the problem in a more general manner to gather motivation, but it's entirely up to you of course.