[Pitch 2] Light-weight same-type requirement syntax

I fully agree with the actual point, that Identifiable<Int> does not feel natural in the same way that Collection<Int> does. It’s also true that:

…but I think it would be disingenuous to argue that this is anywhere near as common as wanting to constrain the element type, or that Collection<Int> // indexed by int makes exactly as much sense as Collection<Int> // contains ints.

Leaving this as a purely intuitive distinction is uncomfortable for the kind of person who spends their weekend litigating language evolution proposals, but I think there’s a clear formal distinction hiding in plain sight: every case I have seen (or found in our own code) where a primary/anonymous associated type feels right has been a monad (or at least a functor). Or, in more Swifty terms, a type that admits a reasonable definition of flatMap (or at least map) over the primary type.

It’s no coincidence that map and flatMap are the canonical examples of Swift functions where it’s okay to use T as a type parameter: monads and functors don’t impose meaning on their subject types and labelling them doesn’t add much value.

Given this, I would like us to get to a place where both labelled and unlabelled constraints can be attached directly to the type, with a recommendation that unlabelled ones are used for “container-like or publisher-like types”, or some such euphemism. I don’t particularly care which order we do it in.

(As far as I’m concerned SE-0081 is actively harmful and going against it is highly desirable.)

6 Likes

I don't think that point is being made; just that if you do want to add an additional constraint, you need to totally rewrite your function signature. There is a biiiiiig cliff.

func doSomething(_ items: some Collection<Int>)

// Oh no! We need Index to conform to Hashable!
// It's just a simple matter of... oh...

func doSomething<C>(_ items: C) where C: Collection, C.Element == Int, C.Index: Hashable

That's not good for usability or approachability. You can do the very specific things the proposal wants you to do, but as soon as you experiment a little bit, it kicks you in to the wilderness.

7 Likes

Hmm. Did you delete a bit about IdentifiedBy<Int> working better with the proposed syntax? I think that’s true, and it also somehow waggles its eyebrows at the insight that you could map over it if you really wanted to. I feel that this somehow simultaneously undermines and reinforces my point about m-words.

Yeah, I have a tendency to write too much so I'm trying to limit myself to single points wherever possible :sweat_smile:

But sure, if we had this feature already, there's an argument that Identifiable would actually be called IdentifiableBy<ID>. Because:

  1. Users will want the ability to use constrained opaque types
  2. You can only get constrained opaque types using this syntax
  3. This syntax doesn't work well for Identifiable<ID>. You kind of need the word "By" for it to work.

Library authors design their libraries for the language we have, so I think the order definitely does matter. If we do the shorthand and no general syntax, it biases library design and authors will try to make things fit. There will be a lot of pressure from their users to make this work.

If we did shorthands via typealiases (or some other kind of alias), there would be more room for library authors to make those kinds of decisions. Collections, for sure, are often constrained by their element type, so the library author might decide that CollectionOf<Element> is a better shorthand than just Collection<Element>, the same way they might decide to add a variants of a function with slightly different labels.

If the idea is that the library author should be allowed to decide, then we should let the library author actually decide.

3 Likes

We know why we are discussing light-weight syntax: the full syntax is too complex for many users (all credits due to @hborla).

But I agree with @Karl: this proposal as well [SE-0341 Opaque Parameters] both simplify the generics syntax for a few privileged cases. Anything that goes beyond need to switch to the "full" generics syntax, and there lies the "cliff".

This cliff creates two different problems:

  1. Developers who aren't familiar with generics (the targets of the simplified syntaxes). They don't even know where to start from. The full syntax is not derived from the simplified syntax.
  2. Developers who are familiar with generics. They feel fatigue when they know they have to rewrite their function signature (or protocol declaration). It is boring, and it is possible to make a mistake.

Very quickly speaking, and not wanting to pretend I know what's inside their minds, it looks like @Karl is talking about problem 2, when @hborla may like to address problem 1.

May I suggest that both problems could be helped with an automatic refactoring tool? Xcode already provides some of them (Refactor > Rename…, Refactor > Generate Memberwise Initializer…, etc.) Why not add a "Refactor > Convert to Generic Signature" refactoring tool?

-func doSomething(_ items: some Collection<Int>)
// Shazam!
+func doSomething<C>(_ items: C) where C: Collection, C.Element == Int

Now Karl can easily extend the refactored declaration with the C.Index: Hashable requirement :+1:

And developers who aren't familiar with generics are guided on their way to becoming an expert :+1:

3 Likes
Meta comment: Swift Evolution and the Conway's law

Swift Evolution proposals almost never extend to the tooling areas, or documentation.

For a recent example, my comment about SE-0340 and documentation was totally ignored. Of course, I do not claim that my humble contributions all deserve an answer.

Yet I think Swift Evolution is an exemplar case of Conway's Law. The Swift Evolution process is isolated from documentation and tooling, and it shows.

I can understand that our proposals can not make requirements on documentation and tooling. Even the Core Team can't.

But does it imply that we have such important blind spots? Documentation and tooling should be able to enter our language discussions. We should actually be able to leverage documentation and tooling towards our goals.

7 Likes

Yeah, if we're introducing the shorthand now, I definitely think we'll need to think about how we might design various aspects if we already had the long-hand Collection<.Element == Int> (or Collection<Element: Int> syntax. Your typealias suggestion here seems reasonable, so it would be kind of a bummer if protocol authors had to do something like:

#if swift(>= X)
  typealias Collection<T> = Collection<.Element == T>
  protocol Collection: Sequence {
#elseif swift(>= X-1)
  protocol Collection<Element>: Sequence {
#else
  protocol Collection: Sequence {
#endif

So to the extent that our preferred syntax for enabling the shorthand may depend on a future mechanism for more complex constraints in angle brackets, I agree we have a bit of a logical dependency that we'll want to consider carefully. But other that that I only meant to point out that I don't see this proposal as cutting off future evolution for the Collection<.Element == Int> syntax or similar.

I'm of the mind that we would still want this shorthand even if we had the longer angle-bracket syntax, but I pretty much agree with you that having just the shorthand leaves the language in a bit of an awkward place, so I would hope that the general feature comes quickly after. IMO the benefits to the end user would be worth it, but I do worry that we'd create a bit of an attractive nuisance that library authors would reach for in ill-advised manners simply because the general feature is still lacking.

For instance, will authors of protocols like Identifiable update ID to be a primary associated type in Swift N (simply because they want to let users avoid where clauses and there's no other way to do that) enabling the sub-optimal Identifiable<Int> syntax, only for users to be able to specify Identifiable<ID: Int> in Swift N+1?

Taking this analogy to it's logical end would allow the API author to specify a generic argument label in addition to a generic parameter name:

protocol Collection<Of Element, IndexedBy Index: Strideable> where ... { ... }
...
func doSomething(with collection: Collection<Of: Int>

which I... actually might like?

5 Likes

I don't think so; as far as I'm concerned, I'm (intending) to talk about both.

Whenever I learn a new programming language or concept, I start with basic examples, then I play around with them and add things, applying the rules until I develop an intuitive understanding of how things work. I think that's a very general thing.

The problem with this design is that you can't do that. As I've shown - want to add some collection indexes to a Dictionary or Set? Then you start needing to rewrite the function signature in a dramatic way and everything starts to fall apart.

Not only does that not help, I actually think it hurts. It makes you feel like adding a Index: Hashable requirement is some super-advanced concept that should only be attempted by rocket scientists. It's so painful that it acts as a disincentive to learn about other things.

--

Also, one thing I find a bit irritating: it isn't just Holly who cares that generics are approachable and learnable - I've also been mentioning it and offering ideas for years.

For example, back when the idea was that we needed "partial protocols" or "existential self-conformance" to make our generics system flow better, I set out what I thought the issues were.

  • That existential syntax was too simple, and too easy to confuse with generics
  • That the compiler would automatically box types in to existentials, but not automatically open them

And that these combined to make generics difficult to understand.

Maybe others had mentioned some of these issues before, but I've been active on the forums since they started, and I'd never seen them expressed in that way in that context. Those were original thoughts, and I took the time to write them up, and the risk to express them in an open forum. The reactions at the time seemed to suggest they were not obvious comments.

Now those things are coming in to the language - which is great! I'm not acknowledged as contributing towards any of those features, of course.

I don't participate in these forums to chase acknowledgement and I would never ask for it, but in the context of your post and the points that you're making, I would like to draw attention to the fact that I do very much care about making generics learnable. I'm not only interested in advanced use-cases.

But that's all I'd like to say on that subject. Again, I don't ask for acknowledgement and find it a bit embarrassing to even talk about it.

4 Likes

My "not wanting to pretend I know what's inside their minds" precaution wasn't enough, so please accept my apology for the artificial distinction between you and Holly I made in order to support my argument. It was probably tactless. Still, I think it was useful to split the "cliff" problem into two sub-problems.

3 Likes

It's all good, no apology necessary :slight_smile:

3 Likes

I can empathize with the sentiment you’re expressing—and certainly appreciate that you took the time to write out your thoughts then as now—but do recognize that yours was the 137th post in a thread started nearly two years prior, and as the contributor of some of those preceding 136 posts in that thread I wonder if you could have in turn expressed your irritation without doing the very thing you were irritated about, dismissing those contributions in preceding years with a nameless “maybe others had mentioned some of these issues before.” Personally, I’d rather be overlooked silently than be one of the “maybe others.”

I'm still working on a reply to the technical points on the proposed design that have been made, but I want to quickly address this comment.

Clearly I am not the only person who cares about making the generics system more approachable, nor am I the only person actively working on language enhancements to accomplish this goal and contributing to design discussions. There are many compiler engineers working on this effort, as well as an entire community of programmers contributing to discussions on the forums here. The valuable insight from the community -- including yours -- has been heard and seriously considered when putting together all of the proposals that have come to the forums lately in this area. As a result of the feedback here, I'm putting together the set of ideas for a general opaque result type constraint syntax that I know of to start a new discussion to get all of your insight and brainstorm together.

We can't possibly name every contributor in the "acknowledgments" section of proposals, because there are so many valuable contributors participating in these discussions and advocating for specific language features that they believe will help accomplish the goal. I hope this goes without saying, but I value all of the perspectives in this discussion and the ones that preceded. Everybody here clearly cares about the goal at hand.

12 Likes

Can you cite some? Specifying the index on a generic collection is almost always an error that represents an attempt to do something better done using index manipulation methods on the collection (for example, you should not lock it down to Int In order to do index arithmetic), so it may be instructive to see examples of correct usage. In particular I am not aware of a reason to specify the index type of an opaque result, in fact it would probably be dangerous to do so since you do not know the type of the collection and interchanging indices between different types is a no no.

3 Likes

I don't think the missing acknowledgment refers to the proposal text, but rather the discussion itself: There are surprisingly many cases where someone brings up an idea which is more or less ignored, but than later somehow makes its way into the implementation, without a single reply which would signal to the "inventor" that their input had an impact.

The two obvious explanations for this is that either the proposal author did not consider the input valuable, or that they simply did not take the time to read the original post. Neither is very encouraging...

Already quite off topic, but I wouldn't want to start a new thread

1 Like

(posting as a moderator)

It is indeed off topic and should have either been a new discussion topic or left alone.

In general I’m concerned this thread is getting increasingly personal. Even comments praising individuals for their motivations, ideas, or involvement in a discussion, can be taken the wrong way and are best avoided.

Bear in mind the goal here is discussion of the pitch not a meta discussion of how evolution proposals are conducted. If you have thoughts on the process please start a new thread (maybe citing posts here) or possibly DM the core team.

More active moderation of this thread might be needed if it diverges again.

3 Likes

Is there a reason why (eventually) we couldn't have syntax like

func doSomething(_ items: some Collection<Int> where Index == Int)

(And hopefully drop the requirement to specify some too.)

Obviously, this is a future direction but I don't see the pitch as closing off corners, or making a chasm between simple generics and more complicated expressions. The leading angle brackets syntax would remain the fullest and most capable form, but the kind of sugar being discussed in this pitch can be used very frequently.

Learnability is subjective and I personally can remember when I was a newcomer to the language with 1.0, trying to write generics using syntax just like the proposed sugar ('cos if you can write Array there why not Collection?'). To me at least, the pitch seems more like a stepping stone than a weird magic special case.

Just adding signal: I often push Swift into teams that are extremely well entrenched in C/ObjC/C++. One of the biggest things I struggle while integrating Swift into these teams are "one-off" features that don't appear to be obviously composable*. The Protocol<SpecializedType> syntax appears to be like it will be another one of these features.

I think starting with a method of applying constraints to Protocol associated types generally would a lot easier to teach and explain.


*I know all of the above languages have bespoke non-composable features, but Swift is a lot more complicated than C & ObjC, so being able to easily explain features is extremely valuable.

9 Likes

Remember we already have a composable way to apply constraints to protocols as parameters. We are purely talking about the ability to apply arbitrary constrains to opaque result types. If implemented this would be an extremely niche feature: in this threads’ many posts there has not been a single compelling example. The need is always presupposed.

So regarding how to teach the general syntax if we had it, to a first approximation it should not be taught. My experience working with beginners learning swift, even experienced programmers from other languages, is they are better off avoiding non trivial generics until they have a very solid grounding in the language, because there is a tendency to get overexcited and build some very complex and problematic code. One symptom of this is protocols bristling with associated types and complex constraints, when extensions on concrete types or other simpler strategies would be far better. From this perspective “composable features” are a mixed blessing, and obscure ones of questionable benefit a footgun.

This is not to say that arbitrary constraints on opaque result types aren’t something we want to have. But the challenge is that the possible different syntaxes for them are not great. It is a hard problem to solve well and we should take our time with it. But it is also a feature that is neither urgent nor important. Constraining by a “primary” associated type on the other hand is both urgent and important. The lack of this feature is actually hampering real APIs every day. That is why the in theory “sugared” syntax is being prioritized.

2 Likes

Big +1 to what @Karl , @DevAndArtist and @Max_Desiatov said, i think this proposal needs another iteration with carefully consideration of the alternative ideas and future directions (generic protocols)

i would like to also hear about the ideas of Chris cc @Chris_Lattner3

1 Like

I really did not want to jump in again because at this point I sound like a broken record, however I need to highlight this part of the conversation. To me, a person whose opinion is seemingly not important in this conversation or the entire evolution process of the language, this particular part reads as following. There is again an internal need for a feature that could / would / should improve APIs that heavily build on some DSLs with a lot of opaque result types in the returning position (not pointing any fingers here as I'm capable to understand why it's actually an issue). Stamping the general feature as "not urgent nor important" is just a way of admitting that there is some deadline and a requirement for some solution approaching. So what should we do? Well let's introduce some "sugar" and work our way from back to front this time, I'm sure we'll figure it out somehow. Forgive me for that slightly sarcastic wording.
We've already seen very similar approach with the enormous discussions around the trailing closures in the past. To this day I didn't touched that feature because at least for me personally it simply does not drive anywhere without the ability to optionally use the first trailing label. That's off-topic, but it's an example of a "we need THAT feature", regardless how hardly the community tried to find the middle ground and articulated for an optional first trailing label. Anyways, I'm really sad that in recent years we trap ourself in these passively toxic dilemmas - for both sides. I do not argue that this feature is worthless, it isn't, I just do not want to see it done THIS way which will break my neck, the neck of the language user, not yours, the developer / maintainer of the language and its frameworks.

TLDR; let me bring this quote up again:

And no this quote isn't a pass for a NO in every discussion around sugar code, but there's definitely a good reason for why some people resonate with this opinion.

20 Likes