SE-0396: Conform Never to Codable

Evolution Proposal Reviews
41

Ok I think I realized what I’m concretely proposing. Consider the Result type:

enum Result <Success, Failure> {
    case success (Success)
    case failure (Failure)
}

extension Result: Codable
    where Success: Codable,
          Failure: Codable { }

today, the above code does not allow me to do the following:

takesCodableValue(
    Result<Int, Never>
        .success(7)
)

Under the current proposal this would be made to work, because Never will now conform to Codable.

My proposal is that we consider making the function call work not by making Never conform to Codable but rather by changing the way in which the compiler interacts with Never such that when it is considering whether or not Result<Int, Never> deserves the conditional conformance to Codable it arrives at the answer “yes” because the effect of Never is to completely remove the failure case from the enum, meaning that after substituting the concrete type parameters (Int, Never) the compiler sees the enum:

enum Result: Codable {
    case success (Int)
}

which of course we can easily synthesize the Codable conformance for.

If I’m not mistaken, the error that I was suggesting that Result<Int, Never> should throw when attempting to decode itself from the data {“failure”:{“errorCode”:1}} is what would naturally fall out of my new proposal.

Maybe most importantly, this approach would also automatically yield things like Equatable conformance for Result<Int, Never>.

Actually, it would remove the need to ever conform Never to any protocol if all you need is for conditional conformance to work for your enum. (Ok maybe not this, I just had some new thoughts that I think mean I’m wrong here. I think this only applies to the handful of protocols for which the compiler synthesizes a memberwise conformance.)

Lastly, I’ll say that to me this seems to better capture the real reason why Result<Int, Never> is Codable - it’s not really because Nevers are Codable, it’s because there are no Nevers, which I think is exactly the thing provoking the ongoing debate about which error to throw.

42

I'd like to suggest that Never conform to Encodable, but not Decodable. Since you can never instantiate it, you can never find yourself in a position to call Never.encode(to:), but init(from:) conceptually would allow for instantiation (even though it's impossible and always fails.) Rather than allow that initializer to be invoked, we should simply not support it—and therefore, any aggregate type that contains an instance of Never can also not be implicitly Decodable, because it is impossible to instantiate it.

Or, put another way: misuse can and should trigger compile-time errors rather than runtime errors.

1 Like
43

I like the more rigorous correctness. I think though that not having Decodable is a limitation large enough that it defeats the intent of the pitch.

Put a better way:

Result<Int, Never> is a type all of whose values can in theory be losslessly converted to and from Data, but the compiler does not currently treat Result<Int, Never> as Codable, and the purpose of this pitch is to fix that obvious missed opportunity. Any solution under which Result<Int, Never> can’t be decoded is leaving chips on the table.

Wishing for an overview of the opinion space …

Here’s a concrete example from this current thread which relates to the consensus building system that I’ve mentioned:

Regarding the concept of Never conforming to a protocol with initializer or factory method requirements - there’s clearly something to say there, but evidently it’s not “Never should never conform to such a protocol” or we wouldn’t be considering this pitch. I would love to be able to browse exactly how each participant here would phrase their various assertions about the topics at hand (for example that one).

@tera, regarding your question, I don’t have the system thoroughly conceived yet, but I have plenty of ideas which I’m slowly considering how to formulate, at which point I’ll post it and let you know. One key thing that I can tell you concisely is that I believe that the core value comes through the combination of users discretizing their opinions into individual assertions (e.g. “Never should never conform to a protocol with initializer requirements.”) and the system providing the ability to interact in ways that are clearer than “liking”. Specifically I think that some key interactions would include “I agree”, “I confirm” and “I trust” (not all of which are applicable to all types of content, which is an important wrinkle to consider).

1 Like
44

like @Joe_Groff said, part of the power of Never is the way it can compose with generics. so what i like about Never is we can attribute certain error conditions to it, instead of having to invent a new error type for every possible generic type that uses Never.

for some examples of how this is useful in practice:

  • “i don’t expect this field to be present at all”
{
    "keyword": "struct",
    "name": "Dictionary.Keys",
    "lexical_scope": "Dictionary"
},
{
    "keyword": "protocol",
    "name": "Equatable"
}

if  try bson[.keyword].decode() == "protocol"
{
    try bson[.lexicalScope]?.decode(to: Never.self)
}
  • “i expect this field to be present, but always null”
{
    "cLanguageStandard" : null
}

let _:Never? = try bson[.cLanguageStandard].decode()
  • “i expect this array to be present, but always empty”
{
    "options": []
}

let _:[Never] = try bson[.options].decode()
2 Likes
45

This seems to me actively negative, because the error that will end up getting thrown could have contained the relevant information about why an error was thrown but when written this way it won't (the relevant information being that there should not have been a value for "lexicalScope").

Under my proposal Never? would be Codable, so not only would this still work, but the error would actually be improved. Under the current proposal, if there is a value besides null then that value will be passed directly to Never.init(from:) and the error that you'll get out will contain no detail about the particular situation. Under my proposal, Optional<Never> acts like it has just one case, none, and if there is a value besides null then the error that comes out will now reflect that specifically the issue was that null was expected and something other than null was found.

And if you really need to statically specify that an array is expected to be empty then, given how uncommon I think that is, maybe it's fair to at that point tell the user that they can define their own conformance of Never to Codable and then they can make the decision for themselves about what error to throw.

46

I've been convinced. :+1:

However, does a single error encapsulate the concept that creating an instance of an uninhabited type is impossible? And would it be

? I think so, but it depends on what "underlying" means, and it's not defined.

protocol Uninhabited {
  typealias HabitationError = UninhabitedHabitationError<Self>
}

struct UninhabitedHabitationError<Uninhabited: Swift.Uninhabited>: Error { }

extension Never: Uninhabited { }
47

i contest this:

import BSONEncoding

enum OuterCodingKeys:String
{
    case metadata
}
enum InnerCodingKeys:String
{
    case cLanguageStandard
}

let bson:BSON.Document = .init(OuterCodingKeys.self)
{
    $0[.metadata, using: InnerCodingKeys.self]
    {
        $0[.cLanguageStandard] = 1
    }
}

import BSONDecoding

let decoder:BSON.DocumentDecoder<OuterCodingKeys, [UInt8]> = try .init(
    parsing: .init(bson))

try decoder[.metadata].decode(
    as: BSON.DocumentDecoder<InnerCodingKeys, ArraySlice<UInt8>>.self)
{
    _ = try $0[.cLanguageStandard].decode(to: Never?.self)
}

TypecastError<Never>: cannot cast variant of type 'int64' to type 'Never'
Note: while decoding value for field 'cLanguageStandard'
Note: while decoding value for field 'metadata'
Current stack trace:
0    libswiftCore.so                    0x00007fa302eac750 _swift_stdlib_reportFatalErrorInFile + 112
1    libswiftCore.so                    0x00007fa302b9f66f <unavailable> + 1427055
2    libswiftCore.so                    0x00007fa302b9f487 <unavailable> + 1426567
3    libswiftCore.so                    0x00007fa302b9e290 _assertionFailure(_:_:file:line:flags:) + 364
4    libswiftCore.so                    0x00007fa302bfd0f6 <unavailable> + 1810678
5    BSONNever                          0x0000556768c9c2af <unavailable> + 1270447
6    libc.so.6                          0x00007fa30211f050 __libc_start_main + 234
7    BSONNever                          0x0000556768c2e16a <unavailable> + 819562
Illegal instruction (core dumped)
48
Firstly (less importantly), clarifying my claim and walking it back a little bit ...

I said that this code:

would throw an error with "no detail" about the particular situation, which was hyperbole. It would have been much more accurate to say that I see two aspects of your code that would cause the decoding error to lose some detail - one of them is incidental and easily fixable (and probably not even worth commenting on), and the other is what I believe is inherent to the approach of making Never conform to Codable and is what I'm trying to discuss.

You indeed demonstrated in your updated code that the incidental loss of detail was easily fixable (the incidental loss of detail in the original example was the loss of the key cLanguageStandard from the decoding key path).

---
Secondly, and most importantly, your new example still demonstrates the downside to making `Never` conform to `Codable` that I'm talking about ...

This error:

is actually a fundamentally different (and less accurate) error message than what would come out under my proposal. Consider the decoding flow we're discussing in this example:

  1. Decoding of metadata begins.
  2. Decoding of cLanguageStandard begins.
  3. Decoding an instance of Optional<Never> is attempted using the (incorrect) data 1.
  4. Optional<Never> first checks whether the given data is exactly equal to the null token.
  5. It finds that it is not.

It is at this point in the flow that my proposal differs from (and improves upon, I think) the current pitch.

If we claim that Nevers are Codable (the current pitch), then the flow continues like this:

  1. Optional thinks: "It's no problem that the data was not exactly equal to null, because I can also be decoded if my Wrapped type can be decoded." so it passes all of the data to its Wrapped type (Never in this case) to attempt decoding.
  2. Optional<Never> directly rethrows the decoding error thrown by Never.
  3. You get the error message you showed.

Under my proposal, Optional's thought process would instead be:

  1. "I know that my Wrapped type is Never, meaning that I only have one case (.none), meaning that I do not need to delegate to my Wrapped type - seeing that the data is not exactly the null token is enough to authorize me to immediately throw a more descriptive error which states clearly that null was the only valid option and something else was found.

Obviously, the two messages look similar on the surface. I'm arguing that the message that results from my approach is at least more correct if not demonstrably better.

I hope at least that the concrete difference that I'm proposing is clear. I think that the claim that it's more correct is very strongly backed up by the fact that it allows us to skip the quandary caused by trying to implement an initializer requirement for Never. I think the claim that it's "demonstrably better" would be clearer to show using examples with enums that aren't Optional.

49

we can’t really give Optional<Never> a different thought process than all the other Optional<T> where T:BarbieDecodable because this thought process needs to flow from the type parameter. otherwise we have

extension Optional<Never>:BarbieDecodable
{
}
extension Optional:BarbieDecodable where Wrapped:BarbieDecodable
{
}

and they cannot coexist.

you’ve stumbled upon a known issue in swift-mongodb, that Optional.init(bson:) possibly needs to catch the callee’s error and wrap the error in something that remembers itself.

the only reason i didn’t implement this already is because there are a few other generic types (Array<T>, Set<T>, etc.) that also don’t wrap errors, and i didn’t want to create the impression that just because a type doesn’t appear in the decoding trace, that it wasn’t involved in decoding at all. this is fundamentally just a composibility/scalability rationale - i don’t want every generic type conforming to BSONDecodable to have to remember to wrap errors.

1 Like
50

I’m in favor of .typeMismatch(Never.self, context) over .dataCorrupted(context) for a very mundane pragmatic reason - when seeing this error in the logs, I want to see word Never somewhere in the text message.

8 Likes
51

That's a great point, that makes typeMismatch a great pragmatic choice for handling the Never case.

2 Likes
52

Reasonable proposal and see no downsides.

I'm in favour of typeMismatch over dataCorrupted

1 Like
53

A thought… since Never would now have an available initializer, the compiler could no longer assume it was impossible to create an instance of it. Or is the compiler sufficiently well-informed because the type is uninhabited? :thinking:

54

an init is just a function that returns Self. it's no different from the other myriad of Never-returning functions in the wild.

2 Likes
55

It's sufficiently well informed because it's uninhabited. You can actually use any uninhabited type for a function that doesn't return, and the compiler will understand that control flow can't return from it.

Never is only special because it gets extra affordances like built in conformance to common protocols.

4 Likes
56

Good point!

57

@allevato @nnnnnnnn Since I'm tagging you I'm going to be very careful to keep the cognitive burden I'm trying to impose on you right now to a bare minimum.

Regarding the alternative that I have proposed here in this thread:

Do either of the following options match your perspective?

A) You have read what I proposed at least up to its first fatal flaw, and due to that fatal flaw (whatever it or they may be) you do not see my proposal as a genuine alternative.

(maybe unimplementable, incoherent, coherent but doesn't check all of the boxes that interest us, way-out-of-scope, etc...)

B) You have read it and believe it is a genuine but sub-optimal alternative.

(In which case I would suggest that it could be included in the "Alternatives Considered" section)

Just for summary's sake, the essence of my alternative is:

"We try to find a way to allow types that involve Never (like Result<Int, Never>) to be Codable without taking the easy but potentially "harmful" shortcut of simply conforming Never to Codable."

58

You'd need to make a convincing case that it's (a) "harmful"; (b) a shortcut.

59

My assumption is that this is accepted as true on a philosophical level.

I believe that my proposed alternative allows us to properly model this truth, whereas it is impossible to model correctly by simply conforming Never to Codable.

When I say "impossible to model correctly", I mean that it is not possible to throw the "correct" error when attempting to decode Never, because it will always be the enclosing enum that has the context required to properly point the finger at the real problem (e.g. that the "failure" key was found, and that that is fundamentally invalid for Result<Int, Never>).

I'm proposing that rather than giving this impossible task to Never, we put that responsibility where it can be properly handled, which is in the enclosing enum.

Perhaps "harmful" is a stretch. It feels weird to set the precedent of Never conforming to a protocol with initializer requirements and to gloss over what I see as the non-coincidental and philosophically non-trivial problem about which error to throw.

(By the way, am I correct that this will be the first official conformance of Never to a protocol that has initializer requirements?)

I understand that while the problem about which error to throw may be philosophically non-trivial, it is entirely trivial in terms of implementation, which is what makes it so easy to just put the philosophy to rest, implement it so that it serves our immediate need and move on.

I think I have found in my career that attempting to adhere to my philosophical intuition (within practical limits) provides unexpected gifts down the line. I can't promise that this debate I'm raising will ever prove itself worthwhile down the line, but I do think there's a valid point that I'm making, and since I have not gotten that much substantive response I wanted to try one last time to see if I can make my point clear.

If the rationale for taking this simpler route is pragmatism that seems perfectly fine to me. I just want to know if my line of argumentation is convincing - that my proposed alternative would indeed be more "philosophically correct", whether or not "philosophical correctness" is of interest to us in this context. If so, then I think putting it in the "Alternatives Considered" would be fitting.

60

how could this be implemented without banishing all other Result types from conforming to Codable?

1 Like