[Pitch 2] Light-weight same-type requirement syntax

Is there a reason why (eventually) we couldn't have syntax like

func doSomething(_ items: some Collection<Int> where Index == Int)

(And hopefully drop the requirement to specify some too.)

Obviously, this is a future direction but I don't see the pitch as closing off corners, or making a chasm between simple generics and more complicated expressions. The leading angle brackets syntax would remain the fullest and most capable form, but the kind of sugar being discussed in this pitch can be used very frequently.

Learnability is subjective and I personally can remember when I was a newcomer to the language with 1.0, trying to write generics using syntax just like the proposed sugar ('cos if you can write Array there why not Collection?'). To me at least, the pitch seems more like a stepping stone than a weird magic special case.

Just adding signal: I often push Swift into teams that are extremely well entrenched in C/ObjC/C++. One of the biggest things I struggle while integrating Swift into these teams are "one-off" features that don't appear to be obviously composable*. The Protocol<SpecializedType> syntax appears to be like it will be another one of these features.

I think starting with a method of applying constraints to Protocol associated types generally would a lot easier to teach and explain.


*I know all of the above languages have bespoke non-composable features, but Swift is a lot more complicated than C & ObjC, so being able to easily explain features is extremely valuable.

9 Likes

Remember we already have a composable way to apply constraints to protocols as parameters. We are purely talking about the ability to apply arbitrary constrains to opaque result types. If implemented this would be an extremely niche feature: in this threads’ many posts there has not been a single compelling example. The need is always presupposed.

So regarding how to teach the general syntax if we had it, to a first approximation it should not be taught. My experience working with beginners learning swift, even experienced programmers from other languages, is they are better off avoiding non trivial generics until they have a very solid grounding in the language, because there is a tendency to get overexcited and build some very complex and problematic code. One symptom of this is protocols bristling with associated types and complex constraints, when extensions on concrete types or other simpler strategies would be far better. From this perspective “composable features” are a mixed blessing, and obscure ones of questionable benefit a footgun.

This is not to say that arbitrary constraints on opaque result types aren’t something we want to have. But the challenge is that the possible different syntaxes for them are not great. It is a hard problem to solve well and we should take our time with it. But it is also a feature that is neither urgent nor important. Constraining by a “primary” associated type on the other hand is both urgent and important. The lack of this feature is actually hampering real APIs every day. That is why the in theory “sugared” syntax is being prioritized.

2 Likes

Big +1 to what @Karl , @DevAndArtist and @Max_Desiatov said, i think this proposal needs another iteration with carefully consideration of the alternative ideas and future directions (generic protocols)

i would like to also hear about the ideas of Chris cc @Chris_Lattner3

1 Like

I really did not want to jump in again because at this point I sound like a broken record, however I need to highlight this part of the conversation. To me, a person whose opinion is seemingly not important in this conversation or the entire evolution process of the language, this particular part reads as following. There is again an internal need for a feature that could / would / should improve APIs that heavily build on some DSLs with a lot of opaque result types in the returning position (not pointing any fingers here as I'm capable to understand why it's actually an issue). Stamping the general feature as "not urgent nor important" is just a way of admitting that there is some deadline and a requirement for some solution approaching. So what should we do? Well let's introduce some "sugar" and work our way from back to front this time, I'm sure we'll figure it out somehow. Forgive me for that slightly sarcastic wording.
We've already seen very similar approach with the enormous discussions around the trailing closures in the past. To this day I didn't touched that feature because at least for me personally it simply does not drive anywhere without the ability to optionally use the first trailing label. That's off-topic, but it's an example of a "we need THAT feature", regardless how hardly the community tried to find the middle ground and articulated for an optional first trailing label. Anyways, I'm really sad that in recent years we trap ourself in these passively toxic dilemmas - for both sides. I do not argue that this feature is worthless, it isn't, I just do not want to see it done THIS way which will break my neck, the neck of the language user, not yours, the developer / maintainer of the language and its frameworks.

TLDR; let me bring this quote up again:

And no this quote isn't a pass for a NO in every discussion around sugar code, but there's definitely a good reason for why some people resonate with this opinion.

20 Likes

I'm not suggesting otherwise - although I do hope proposal authors make more effort in future to acknowledge the time and effort volunteered by the community.

But that's all I'll say on it. It's a comment on another comment, not the point of the thread.

This seems very arbitrary and hand-wavey. Unilateral declarations that some things are simply not important (despite plenty of feedback to the contrary in this thread) essentially shut down debate.

At the same time, I don't think this is an advanced feature, at all. It is a completely basic thing for a developer, just starting with their first generic collection function, to attempt to tag a Range<Index> using a dictionary or store indices in a Set. And what will they see?

Also, where was the community survey asking if the lack of constraints on opaque types were hampering our APIs? Because they certainly do. We also create "real APIs", you know ;)

12 Likes

Every time I’ve looked at a problem and asked myself if I can use opaque return types, the answer has been “no, because I’d need to constrain an associated type”. And in most if not all of those cases, it has been a monadic protocol where I’d want to constrain the primary associated type.

There may or may not be an internal need driving this proposal; I have no insight into that. But, based on my experience, adding support for constraining by primary associated type will lift opaque return types from a niche feature that’s barely useful outside SwiftUI to a generally applicable tool that covers the majority of cases.

4 Likes

There is zero doubt it's worth solving this problem. At this point however I stated my position loud and clear that I do not support the approach taken for this particular use case. Using Chris' words from the quote, I think the proposed solution is "mortar", but I would rather support an alternative solution which is probably way more complex, requires more time to develop and nail down correctly however it'd be a "brick" in such analogy.

8 Likes

No, generalized opaque result type constraints are fully modeled in the compiler implementation. The only missing piece is a generalized surface syntax, but the syntax design is a very difficult problem to solve, and despite the comments here, I do not believe there is one obvious syntax for it. I’ve already stated above that I’m writing up all of the idea that I know of — there are at least 4 different options and I’m hoping the community will surface more — for a new discussion thread, because it’s a problem worth solving in addition to what’s proposed here. I hope to post that discussion today or tomorrow.

Remember that this proposal is not just about opaque result types. I find this proposal much more broadly applicable to opaque parameters now that it has been accepted, where this feature is purely sugar for what you can already express today. This feature is not a replacement for a general syntax for opaque result type constraints. I would appreciate feedback on the merits of this proposal with that framing.

On a separate note, there’s a lot of speculation based on false extrapolation of comments going on above, and I find it unhelpful and disrespectful. The general theme of the recent generics proposals is improving the learnability of the generics system, primarily for novice programmers and others who shouldn’t have to dive so deeply into the generics system to write basic generic code. I’d like to ask everybody to please keep your comments on the topic of the technical merits of the proposal.

9 Likes

There is plenty of doubt. The use case for general constraints on opaque result types has not been demonstrated, unlike the very tangible need to constrain the element of a collection. Instead, it is being taken as a priori that there must be a use case. This discussion would benefit greatly from real examples of how opaque result types could be constrained in other ways and the expressive power this would bring. Maybe there are loads of good examples, but we need to see them to know for sure.

Again, this does not mean the feature should not or cannot be added eventually. But there is no urgent need, because it is not clear there is any need.

Putting aside the unnecessarily emotive language: in what way does it "break the neck" of the language user to introduce this feature in this way? You are making a claim that somehow not introducing generalized opaque result constraints hurts specific kinds of users, but do not provide any supporting evidence for this claim. Actual code examples would be really helpful here.

Bear in mind when thinking about those examples that opaque result types is almost entirely a feature for the production of frameworks. Not just ABI stable frameworks but also packages that want to remain source stable, avoiding a semantic version bump, whilst updating implementation details that would otherwise leak out in the form of a concrete type.

When you are purely a user with control of both sides of an interface, the need to use opaque result types is very marginal (it can sometimes be used to loosely couple code but in general brings as much hassle as benefits). So it is certainly true that improving this feature is being improved specifically to allow for better frameworks. But the claim that this is, in turn, hurting users needs substantiation.

There was the example quoted below. Maybe it got an answer, but I could not find it.

There was also a request for clarification about subclasses (does Collection<Base> accept [Derived] or not?), which did not have any reply.

6 Likes

It is difficult to ignore that a very specific phrase is appearing every single time without fail, and it very much suggests that you view this through the narrow lens of opaque types used in result position only. Firstly, that is a strange position for review manager of the recent opaque parameters proposal to take, and secondly, it seems to be directly contradicted by the proposal's author:

It's difficult to know what to make of this. There seem to be mixed signals about its intended utility.

9 Likes

Thank you for pointing this out - I think this is a good observation. I completely agree that evolving a generic signature from the sugared form to the more expressive form is important, and this will become a crucial part of the learning curve for concepts like type parameters and associated types.

The need to transform a signature when adding associated type requirements is an inherent problem introduced by the opaque parameters feature that was just accepted. I don't think that's a bad thing - from my teaching experience, showing novice programmers a "desugaring" transformation is a very illuminating part of the learning curve, because it shows that this seemingly-complicated thing they have to approach now is something they've already been writing in their code and have internalized what it means. To me, the implication here is that this desugaring transformation should be surfaced to the user, and it must be discoverable. My opinion is this should be surfaced through a diagnostic fix-it, and I'm also a big fan of this sort of transformation via the refactoring engine, as Gwendal suggested, in both directions (sugared → desugared, and desugared → sugared when possible).

Further, I think this proposal introduces that cliff at exactly the right point. You can start writing the extremely common constraints using this syntax. I do believe that when you're working with wrapper or container protocols like Collection, the constraint you want to write the vast majority of the time is on the Element type, and most conformances to such protocols are generic over that same associated type, which is why I believe the concept of a "primary" associated type is valuable. This proposal allows expressing same-type, conformance (and layout), and supertype constraints on primary associated types, so all of the fundamental constraint types are covered. The first is the core part of the proposal, and the others fall out of structural opaque types. So, Collection<Int> expresses a same-type constraint, Collection<some Numeric> expresses a conformance constraint (albeit with an extra type parameter, but that only makes a difference for those who care about extra type parameters in their ABI), and Collection<some MyClass> expresses a supertype constraint.

When the programmer wants to constrain a different associated type, that's when they really need to learn more about associated types and how they are different from type parameters. For basic constraints, that difference doesn't really matter, because you can constrain both forms of type parameters in the same way.

3 Likes

My point here was an attempt to not fixate the discussion on opaque result type constraints, because this proposal is also very valuable for opaque parameters, which are already "just sugar". Personally, I think input type parameters are more common and therefore this feature will be more heavily used in parameter position. Perhaps that is just because opaque result types are fairly limited today, but I still think it's more common to abstract generic code into a function rather than impose generic code onto callers. In any case, the intended utility is for any opaque type, whether in parameter or result position.

I think it's a stretch to claim this is a common use case but it's certainly a real one that might come up. But note, you used an example of a paramter not a result type. And so this problem has a solution in the language today, and we are just talking about whether there's a big leap from your first example to your second and whether the sugar should be adjusted to improve the latter at the expense of the former (which we can discuss more, my view is no).

(incidentally, I think this proposal also improves the second example, which could probably be written func doSomething<C>(_ items: C) where C: Collection<Int>, C.Index: Hashable)

But the main contention here is about opaque result types and expressivity. Your example should be about those. And if it was, it would fall apart. Because an API author is not likely to return a collection that is opaque yet with a hashable index. "But what if the user who is returned this opaque result type needs the index to be hashable" you might say. And this is a reasonable thing to ask, but the fact is the API author is not going to speculatively make their collection indexes hashable just on the off chance some user is going to need that.*

That isn't really unique to this feature though: this is just the standard downside of information hiding. When returning an opaque result type you are choosing to reserve flexibility at the expense of your users benefitting from (or, to put it another way, tightly coupling to) implementation details of the specific returned type. But this is already the case. For example, an API might return a Collection<Int> when really the user needs a RandomAccessCollection<Int>, or even specifically need or want an Array. The framework might even be returning one, opaquely, but the user doesn't get to benefit from it. That's the trade-off the framework author makes when they choose to minimize their interface.

* Here is a attempt to describe a case where this might happen: There are two developers working on the same codebase, one maintaining an app and the other a supporting framework dedicated to the app. The app developer finds they need to hash an index, but the framework developer has made it an opaque collection to keep themselves decoupled from the app developer. They discuss it and agree that the framework start constraining their index type to be hashable. This scenario is plausible to me but still not very compelling.

1 Like

Again - I think there is confusion here.

Opaque parameters are, IMO, by far the most common expression of this proposed feature. Result types are also very interesting, and enable new expressive capabilities, but when we consider the teaching and approachability side, parameters are possibly more important.

2 Likes

I think Ben and I are saying the same thing (in different ways); this proposal is NOT * about adding expressivity to opaque result types. It happens to do that in a limited way, but that is not the goal of the proposal.

EDIT: sorry, one word really changed the meaning of this comment :sweat_smile:

1 Like

The argument is circular: generic result types are uncommon because they are limited.

Again, sorry for insisting, but some Publisher<Int, Never> would be happily welcomed by all Combine users, who are not charmed (euphemism) by eraseToAnyPublisher. We're pretty happy SwiftUI was privileged with some View, but there are many other apis that are looking for improvements.

And if this is not "the goal of the proposal", I'm sorry but it is very much related. The pitch is entirely focused on a single primary associated type, and has a very short "future directions" section. The PersistentSortedMap example in the pitch text is not clear.

7 Likes

The proposal has been updated to include multiple "primary" associated types.

9 Likes

A framework vendor probably won’t have much incentive to express any constraints on their opaquely returned Collection’s associated Index type. They’ll probably just return some Collection<Int> even though they could also make guarantees about other associated types. I agree.

But a framework vendor probably do want to express constraints on their opaquely returned Publisher’s associated Error type, and return e.g some Publisher<Int, Never>.