[Accepted with Modifications] SE-0341: Opaque Parameters

Hi Swift Community,

The review of SE-0341: Opaque Parameters has completed. The Core Team has decided to accept this proposal with modification.

Along with a number of enthusiastic endorsements for the change, the majority of the discussion turned on one issue: that opaque result types are sufficiently different to generic parameters that the some keyword isn't appropriate. The position of the proposal was defended by the authors and other reviewers and the core team agrees that this is an appropriate use of the some keyword.

There was also discussion about how the use of opaque types in the consuming position of function arguments is problematic e.g. func g(fn: (some P) -> Void) { ... } . This is a similar situation as with SE-0328: Structural Opaque Result Types, and the proposal author has put up a PR to subset them out. The core team is including this modification with the acceptance.

Thank you to everyone who participated in the review!

Ben Cohen
Review Manager

19 Likes

Can I understand this acceptance as the acceptance of 'flipping' interpretation?

[EDIT]
I think this is an important point. As many people did so, simple understanding of this proposal is that 'some is syntax sugar of generics in parameter position, and some is syntax sugar of reverse generics in result position. Because 'flipping' interpretation is not equal to this, the proposal should clarify which interpretation was accepted.

Perhaps that would be the minority view point but I'm a bit shocked the way it was handled. The proposal was not defended by the authors and other reviewers from where I am.

The answer to the relevant question (1) was dismissed with a mere "it is accepted as bad practice when designing Swift APIs" which doesn't actually answer the question.

The answer to the secondary question (2) was dismissed with completely inappropriate unrelated examples. To which I can bring my own set of (unrelated?) counter examples, like no "var" in parameter position, or the lack of "goto".

Overall my felling is that the authors rushed away and swept the relevant raised issues under the carpet.

All IMHO.

2 Likes

Swift discourages using the return type to resolve overloads, yet it still allows it to be done (by specifying the return type in the calling statement). I see a parallel here with forbidding the use of such a mechanism for a new(ish) feature.

As for why it's terrible, this is subjective. I see it as bad because it is unclear at the point of use if type specification is necessary for a correct build, or merely a style choice. If you had overloads which returned Any and Int, you would not want the Any overload called just because you decided to remove as Int from the call site.

Wow this was a really quick review.

I've been mentioning the issues with the generics syntax for literally years, beating the drum that it was really our biggest problem, even when the conventional wisdom was that things like existential self-conformance were more important.

I really wish I would have had time to share my thoughts on this. I don't know what they would have been. I saw the issues around function types (which were still apparently an active area of debate as little as 6 days ago), and though "oh, I'll need to dedicate some more time to think about this". But no; exactly 14 days after the proposal went up (which was exactly 7 days after the pitch went up), it has been accepted as part of the language.

I give up my time voluntarily to participate in this community - it's not work. I need to balance my time between work, recreation, and spending it with loved ones, and even a minor illness dramatically cuts in to the time available. With a 14-day review, even a 3-day illness means losing more than 20% of the review time (!). 29% of the review time (a whopping 4 days) was over weekends.

I've been working on a draft of a more detailed critique of the lightweight same-type constraint syntax, but haven't had time to finish it yet. Things like this are really demoralising. It makes me feel that we're just here to be ignored. The decisions have been made and will be pushed through regardless of what we think :frowning:

What exactly is the rush? And why is this language being designed in a rush?

15 Likes

I share a lot of your disappointment, but I really don't think that there is a problem with deadlines here: When you miss them, at least you know what's happening (nobody could consider you feedback, because it wasn't published); it's much more depressing when arguments are ignored silently.

Actually, I'd say there is another issue with timing: Afaics, the norm is that the process takes much longer than scheduled. Maybe it would be better to increase the review time, but then be strict with completion.

The months before WWDC seem to be the time of year when changes to the platform are made very quickly. I remember wondering in 2019 why property wrappers and opaque result types seemed to spring forth full grown in April in two enormous posts by doug and joe that weren't really pitches so much as statements of accomplished fact. I've been really pleased that concurrency (so far) has not followed that model. I could easily be wrong, but this feels like one of the situations where internal needs at apple drive the platform evolution. I just wish that there was a product that absolutely required move-only types, C++ integration, scripting and variadic generics. :)

10 Likes

I think that maybe SwiftUIs ViewBuilder would greatly benefit from Variadic Generics.

I’ve written a bunch of ResultBuilders and ended up writing boilerplate-y repeations of similar buildBlock functions taking various amounts of generic type inputs. I was in the brink of generating them all with GYB (or Sourcery), but it was just below the amount to warrant the increased complexity of meta programming. This repeatition could be removed with Variadic Generics.

So I think SwiftUI has similar situation with ViewBuilder and would benefit greatly from Variadic Generics.

1 Like

And I think swift-crypto, i.e. CryptoKit certainy used by lots of Apple's products would benefit from move-only types. Feels natural to use these in security sensitive context where i.e. one wants control over the sensitive bytes of a private key, and during the operations (functions) where this data is needed, i.e. when performing ECDSA signatures. Apple's swift-crypto contains a SecureBytes struct, maybe this could be either improved by move-only types or possible even replaced? Hard for me to tell.

The project Cryptoswift also contains a SecureBytes type that might benefit as well.

No, it is discussed as a future direction. There is no interpretation accepted as part of this proposal for the meaning of opaque types in the consuming position of function arguments: they are restricted from use. See the linked PR.

2 Likes

I see, thank you. It's a feature that has been discussed from 2019 (three years ago!), so I'm sort of convinced that the proposal was accepted quickly, but it's unfortunate that the interpretation of this point was not clearly decided during the review.

1 Like

AFAIK "about two weeks" has been standard operating procedure for Swift Evolution for quite some time. In fact, the document that lays out the process only promises a single week for reviews.

Yes, sometimes decisions are not posted until a week or more after the conclusion of the review period, but I don't think this review was uniquely fast compared to others, and it's always been the expectation that community members get their feedback in by the conclusion of the review period to ensure that it gets considered properly.

It may be worth having a discussion about whether the default review period should be lengthened if two weeks does not feel like enough time to ensure that we're getting sufficient community input, but IMO it doesn't make sense to single this proposal out.

8 Likes

There’s inherent difficulty for anyone to provide conclusive answers, because the context is ambiguous.

Do you want to use (1) because of some other programming language prefers that or its a typical convention for you? Or is this an academic question without actual real world use cases or need for it? And so on…

For (2), Swift or any other programming language cannot explicitly prohibit all bad code, that would make the compiler way too complex for no good gain. The review thread provided other examples for this, like while(true) {}

Swift emphasizes static type system and known/explicit types are favoured over ambiguous types, so in a case of (1), as written in review thread, providing type as parameter: func foo<T: P>(type: T.Type) -> T {}. So calling foo(type: Int.self) would be the swift way instead of calling foo() as Int or similar. By providing the type as prameter, the compiler can statically know the type and enforce the code to always work, on the other hand, the ”as Int” version is more dynamic and could result in errors in some cases.

2 Likes

I use this feature every now and then. I didn't encounter other languages with this feature (but I don't know many languages). Older languages didn't have this feature when the left hand side variable type influences the type of the right hand side expression. Besides other reasons this was to simplify parsers / compilers (similar reasons older languages often had trouble implementing closures). Swift supports this feature, which I believe was done deliberately, and supporting this feature increased language and compiler complexity. If this feature is deemed to be terrible / bad practice (whether this is the case or not is the thing I am trying to establish in (1)) it would make sense to remove this feature from the language and the compiler to make them both simpler (2).

As an example I mentioned - we do not have goto operator. We could have implemented it into the language, make the language and the compiler more complex and then... start discouraging people from using goto. Would not be wise, would it. And if we did have goto operator today, it would make sense to remove it from the language and the compiler to make them both simpler and make it impossible for people to use goto vs merely treating it a bad practice and discouraging people using it.

I don't think the example at hand is really representative for the ability to distinguish functions by their return type. Instead of involving generics, it is probably more common to have separated functions (that opens more possibilities to write code that makes sense).
However, as the statement was not restricted to generic contexts, I think the question is still worth a proper answer...

I didn't get a chance to reply on the other thread before it was closed, so I just wanted to sincerely thank everyone who replied to my posts and helped me understand the relation between opaque return types and generics. If I could summarize the revelation I had in a sentence it would be:

A function with an opaque return type causes the caller's code to become generic over its output, rather than the function itself being generic internally, or more specifically, de-sugaring to have any type variables within its own signature.

This especially makes sense when you consider the function can be swapped out "under your feet" (via an OS update) without the client code being recompiled.

That along with the intuition from the Rust RFC - that any where you see some, whomever is supplying the value to that location also determine's its type - allows me to apply the same intuition in both the parameter and return positions so I no longer see them as being different concepts.

Not that it matters at this point, but I change my vote to +1 :slight_smile: Thanks again for the detailed replies and helping me understand Swift generics more intimately. There were some surprising things I learned there, specifically about specialization and the performance of generics, but I'll start a new thread for those questions.

17 Likes

I don't think comparing the exclusion of goto to the possibility of overloading on return type is fair. goto is avoided in modern languages because it is unstructured. In its purest form, it can jump anywhere in the codebase, though in practice languages which have it restrict it to the current (function) scope.

Overloading on return type can be a handy trick, but its use is never unstructured. One never looks at a call site and wonders "how did we get here/where are we going?". The reason it's considered bad is because how the call site is annotated can change which function is called, and thus which type is returned. This is non-obvious. It can lead to a situation where tidying up the code (removing seemingly superfluous type annotations) can change the behavior without breaking the build. That's bad.

As far as removing a feature that is now considered harmful, we have to consider source compatibility. Breaking existing code is also bad, and the Core team is extremely reluctant to do so. We can all acknowledge that a practice is bad while also agreeing that we can't pragmatically prevent it. These are not contradictory stances.

  1. But that issue is not unique to the features in question (function overload by return type and its generic analogue), or is it? In the following examples "it can lead to a situation where tidying up the code (removing seemingly superfluous type annotations) can change the behavior without breaking the build. That's bad", right?
    func foo<T: Equatable>(type: T.Type) {
        print(type == Int.self ? "Other behaviour" : "One behaviour")
    }

    var x: Int64 = 0 // before refactoring
    foo(type: type(of: x)) // "One behaviour"
    var x = 0 // after refactoring
    foo(type: type(of: x)) // "Other behaviour"
    
    func bar(_ v: Int8) -> Int { 4 }
    func bar(_ v: Int16) -> Int { 6 }
    func bar(_ v: Int32) -> Int { 11 }
    func bar(_ v: Int) -> Int { 20 }

    var i: Int16 = 32767 // before refactoring
    print(bar(i)) // 6
    var i = 32767 // after refactoring
    print(bar(i)) // 20
  1. And perhaps there are other more compelling killer examples? I'd love to know them. Provided there are, and the features are indeed flawed, surely that flaw would become obvious during the language design phase, right? Any serious peer review would bring a series of questions like, "look, its not in other languages, why exactly do we want this?" and "let's imagine we have it, let's consider a few examples to see the pros and cons."

  2. And if the flaw wasn't spotted during language design phase, surely it would be caught early during initial years of swift evolution, at the same time when we removed, say, cured functions, or ++ operator, no?

  3. And if the flaw was still not caught during those early stages, and if we consider that to be a flaw today - we have an established mechanism how to remove things: we deprecate them for one/few swift versions and then remove in some never swift version. We do deprecations / breaking changes every now and then (@escaping parameters, "String Index Encoded Offsets", etc).

  4. And if there's in fact no flaw in those features, but its one of those "in my personal view that's a terrible idea" cases, perhaps that's subjective and not a universal viewpoint? And people who don't share that viewpoint would legitimately reach out for func foo<T:P>() -> T, and then start naĂŻvely converting between that form and func foo() -> some T on the grounds "hey look at this: func bar(param: some P) -> some T, here "some" means generic parameter, surely it has to mean generic result, no?!"

I believe the function overload by return type and it's generic analogue features were brought to swift to make it better / richer compared to other languages that don't have it (Objective-C including). Otherwise why? It's just incomprehensible to me to have these features listed as "Excellent" on the first page and "Terrible" on the last page of the same book. It's one or another... And it deserves to be resolved which way it is.

There once was a great litmus test: "if we created swift today would be do it"? I believe we dropped that evolution principle too early.

All IMHO above.

PS. as you can see I'm a great proponent of "five whys"

1 Like

There exist arguments that function overloading is always bad, but the ship has long sailed on parameter overloading. How old is C++ now? This is not the place to litigate the differences between argument and return type overloading, but I'll just give my opinion in brief:

It is expected that passing different values to a function will yield different results. It is not that great a leap to understand that, in languages which support function overloading, passing different types may also yield different behavior. However, return-type overloading is rare. I don't know of any other language which supports it.

It is just as common to see let i = 4 as it is to see let i: Int = 4, even though they behave identically. It's not a stretch to see someone changing the latter to the former, if they prefer that style.

It would be very surprising, therefor, for

let i = f()

to behave differently than

let i: Int = f()

Like most subjective judgements, it's a matter of taste and familiarity. Parameter type overloading has been around much longer than return type overloading.

Probably not relevant for the validity of your point, but for the reference: I know that Haskell and Ada both support return-type overloading.

1 Like