SE-0257: Eliding commas from multiline expression lists

After waffling, ultimately I find I agree with @codafi, @Chris_Lattner3, and @allevato in being not in favor of this proposal.

I am convinced by the examples here that on the whole there will be more cases of ambiguity for a human attempting to parse the language with their eyeballs—and that is quite unfortunate.

I would also say that comparisons with semicolon elision are not apt in my mind for at least one simple but major reason:

There have been many, many times where my C/C++/Java code has been invalid because I left off a semicolon somewhere, but many fewer times where it’s been because of a comma—and of those, all would have been avoided if trailing commas were allowed. I suspect this to be the case for most people here too.

4 Likes

I'd be thrilled to see this feature in Swift for both productivity and aesthetic reasons. I'll elaborate a bit on my reasoning from the pitch thread.

In terms of productivity, juggling commas when making changes to a non-trivial list is a distraction. I find it common to add, remove, and reorder some items in a list, then to have to take an extra step of scanning through it to fix up the commas (or wait for the build to fail and tell me what to fix). It can be a distraction while I'm in the middle of making the changes — it's easiest to fix a missing or extra comma when the cursor is right next to the change, but doing so while the changes are in flight interrupts focus on the actual task at hand.

It's a relatively minor burden in any given instance, but if you frequently work with declarative-style code, it adds up. I think those who are seeing no value at all in this feature probably don't work in this style very often, and that's fine, but I can say with confidence that not having to think about commas in this context would be a noticeable improvement for me personally.

Aesthetically, I also think there is a real improvement to non-trivial lists. I liked @Michael_Ilseman's excerpt from the Swift benchmarking suite as an example of declarative code that would benefit from this change. To me at least, it's quite clear that the first reads better than the second here:

public let StringComparison: [BenchmarkInfo] = [
  BenchmarkInfo(
    name: "StringComparison_ascii"
    runFunction: run_StringComparison_ascii
    tags: [.validation, .api, .String]
    setUpFunction: { blackHole(Workload_ascii) }
  )
  BenchmarkInfo(
    name: "StringComparison_latin1"
    runFunction: run_StringComparison_latin1
    tags: [.validation, .api, .String]
    setUpFunction: { blackHole(Workload_latin1) }
    legacyFactor: 2
  )
  BenchmarkInfo(
    name: "StringComparison_fastPrenormal"
    runFunction: run_StringComparison_fastPrenormal
    tags: [.validation, .api, .String]
    setUpFunction: { blackHole(Workload_fastPrenormal) }
    legacyFactor: 10
  )
  // And so on
]

vs

public let StringComparison: [BenchmarkInfo] = [
  BenchmarkInfo(
    name: "StringComparison_ascii",
    runFunction: run_StringComparison_ascii,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_ascii) }
  ),
  BenchmarkInfo(
    name: "StringComparison_latin1",
    runFunction: run_StringComparison_latin1,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_latin1) },
    legacyFactor: 2
  ),
  BenchmarkInfo(
    name: "StringComparison_fastPrenormal",
    runFunction: run_StringComparison_fastPrenormal,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_fastPrenormal) },
    legacyFactor: 10
  )
  // And so on
]

The newlines function perfectly well as visual separators between items here — the commas become unnecessary at this point. Ultimately whether this is appealing is a subjective call, but given that it's an optional feature, everyone would be free to make their own choice about whether to use it or not. I think it would be a shame to prevent people from using this feature if they want to when those who don't want to use it wouldn't have to anyways.

I think the issues of ambiguity are important to consider, but imo they're very much edge cases whose danger is being blown out of proportion a bit. I think it would be quite rare for both foo(bar.baz) and foo(bar, .baz) to make sense, let alone be valid code, and I think confusion in this area should be attributed to the API design rather than the absence of a comma.

I think so for the reasons stated above.

I believe it aligns with the principles that resulted in semicolon elision. I think it's also in the spirit of Swift being a multi-paradigm language. This feature may not be that important for all styles of code, but it's a significant benefit to declarative code.

I've enjoyed using the feature in multiple other languages, and I think it would work just as well here.

A thorough reading of the pitch and proposal.

I re‐read the proposal and the following is what I am trying to understand about @hartbit’s example:


When parsing a multiline expression list featuring a member expression which appears after a newline [...] the member expression will be interpreted as modifying the expression that preceded it.

Without convenience properties:

let a = [
    FooBar.foo
    .bar // Compiler error! FooBar has no instance property “bar”.
]
assert(a.count == 2)

With convenience properties:

let a = [
    FooBar.foo
    .bar
]
assert(a.count == 2) // Trap! Count is 1.

When this situation comes up, the issue can be ameliorated via a good diagnostic experience. If you attempt to compile the original function call in a context where baz is not an instance member of bar 's type but is a static member of the type of the second parameter accepted by foo the error can be diagnosed with a fixit to insert a comma at the end of the line before .baz , right after bar , leaving you with the code you intended

Without convenience properties:

let a = [
    FooBar.foo
    .bar // Compiler error! FooBar has no instance property “bar”.
         // Do you want to add a comma?
]
assert(a.count == 2)

With convenience properties:

let a = [
    FooBar.foo
    .bar // Compiler warning:
         // Do you want to add a comma, eliminate the newline, or indent?
]
assert(a.count == 2) // Trap! Count is 1.

Did I get that right? If so it isn’t as bad as I originally understood. (It still is confusing though.)

1 Like

Humans had been writing for thousands of years when Aristophanes invented his notation for marking breathing pauses. Programming languages aren't limited by the lung capacity of clever primates. This is still a new frontier, with only 60 or so years of tradition; obviously the best notations are yet to be found.

(It's curious that we're still entering code by typing individual characters into into linear text files. Parsing should've been eliminated by now.)

7 Likes

I found myself in favor of this after reading and participating in the pitch thread. I've worked with configuration languages that didn't require commas when constructing tables, and it's one of the little things I miss.

Many of the formatting ambiguities presented were issues I think should trigger warnings regardless of whether or not this proposal is accepted. For example:

let list: [MyEnum] = [
    foo,
    bar
    .barProperty,
    bing
]

I'd like to add that I take issue with the characterization of the proposal that it dismisses separators as having no value. That's not really accurate--the separators in this case are newlines. Most people don't write semicolons in Swift for that reason. I only call that out because it was repeated here despite responses in the pitch thread.

Another counterargument is that it would look ugly if commas are inconsistently used, but that's no different from an inconsistent use of semicolons. Again, we find semicolon elision serving as a response.

So, being in support of this, the only hesitation I'm left with is due to the comments of those with greater expertise who claim this will impact evolution of the language. Some of us who don't work on programming languages have leaned on whichever particular language implementer who falls on their side as a stand-in for a technical opinion. I'll do no such thing and simply say that I trust the Swift team to debate this and make an appropriate determination.

2 Likes

I'm sorry I haven't had a chance to read all of the posts on this thread or on the pitch thread, but from what I have seen, there is some systematic confusion about the difference between the proposed comma elision and the semicolon elision that exists in Swift already, and I'd like to draw attention to them. I apologize if someone else has already brought this up.

The proposal, as far as I understand it, does not allow the elimination of all commas everwhere, even when entries are spread across multiple lines. There are well known cases (e.g. an array of enumerators) where you are likely to have commas on every line (because you need them for disambiguation), and there are other cases where you'll end up with commas in some cases - but not others - within the same array or dictionary literal.

If we accept this proposal, the result of this is that Swift code in practice will be a be a mish-mash of different things driven by what happened to be acceptable to the parser, it will not be simple, principled, and predictable. People will literally add commas because the compiler (or source formatting tool) tells them to, which erodes Swift consistency and feel, all for the benefit of removing a few commas.

This situation is strikingly different to the situation with semicolons, where you really only need them to separate statements on the same line. There are cases where you could theoretically have to use them to separate statements, but we've carefully designed the grammar of the langauge so that does not occur in practice on pretty much any real world code.

The proposal does not have this behavior, because side-effect free values (like enum cases) are perfectly reasonable array entries, even though they are not typically useful statements. The proposal does not address this, and (in my outspoken opinion) completely misses the design premise that made semicolon elision work so well in practice.

Others have pointed out very well that common use cases that should work well with a proposal like this (e.g. SwiftPM manifests) actually fail in practice, so this proposal is not even achieving the goal.

The stated claim that EDSLs will be made better by this is not well defended, and there is no robust examination of other ways to improve EDSLs. There are many other things on the list that we should consider ahead of this if that were the actual goal - this is pretty much the last kind of change you want to ever make to the expression grammar of a language, because it has such profound and non-local effects.

I also find it surprising and deeply concerning that we are considering a wide range of syntactic sugar proposals that are trivial syntax micro optimizations (eliding a single return keyword, eliding syntactically light-weight separators) that do not actual affect the clarity of code, or improve higher level expressivity of the language. There is a recent blossoming of syntactic sugar proposals on the list of similar ilk, and there are far bigger fish to fry. This is a really concerning to me direction to see the community take.

-Chris

28 Likes

This. +1

7 Likes

I don't think this is totally fair — I disagree that two syntactic refinement proposals constitutes a wide range. Bigger fish are in fact still being fried, and I see no reason that we shouldn't make a small amount of room for small refinements like this. There will always be bigger problems to address than syntax refinement, but by that logic there will never be a time to address these issues.

4 Likes

The same "special case" already exists for semicolon elision in statements.

This analogy does not fit at all. The closest analogy is eliding semicolons between statements; it is literally the same feature.

It's a separate value, just like if you wrote

foo
(6 * 9)

in today's Swift.

There is literally nothing new in that section: these are the rules of Swift's grammar today for semicolon elision with statements. There is a practical difference in that you're more likely to bump into these cases in an expression list (comma elision) than in a statement list (semicolon elision).

This does not follow at all from the proposal. The proposal applies precisely the same logic used for semicolon elision of statements to comma elision of expression lists; it does not otherwise expand the rules.

For reference, I posted an extensive reply to your critique within the pitch thread, to which you did not reply.

(I'll separately post my own thoughts on this part, which are quite subjective)

As I noted in my reply on the pitch thread, this is incorrect. Specifically, I wrote:

Also as I noted in the pitch thread, this is incorrect. Specifically, I wrote:

(skipping the more-subjection comment, so...)

To which I replied in the pitch thread:

Our 8+ years of semicolon elision have prepared the language and tools for this change. So while it is a significant stylistic change to the language, it is not a radical technical change.

Doug

8 Likes

Semicolon elision in statements isn't quite so perfect as you make it sound---we have newline-sensitive rules around the opening parenthesis for call expressions that would surprise people if laid out in detail like this proposal's "exceptions" section, for example---and whitespace sensitivities around operators that can be a surprise. As I noted elsewhere, the exceptions will occur somewhat more often because of leading dot syntax, but it's far from the radical departure you describe.

Doug

6 Likes

I'd add that sugar is a common topic dating back to the old mailing list, and in fact, the two examples given have a history of older related threads. I don't think that means the ball is being dropped in other areas. Point taken that this is low priority, but I think everyone acknowledges that. That said, when it comes to using Swift for configuration or other problems involving lists, which motivates my interest in this change, I don't think this is as superficial a thing to give some thought to as suggested.

2 Likes

Indeed. Of the recent proposals that we reviewed, two are syntactic tweaks (this and SE-0255), two are providing more-strongly-typed counterparts to existing dynamic features (SE-0252 and SE-0253), two improve numerics support (SE-0251 and SE-0246), and several others make substantive expressibility improvements (SE-0244, SE-0249). Combine that with active movement on property delegates, custom attributes, explicit member wise initializers and variadic generics, I'd say we have a healthy mix of proposals moving through the pipeline.

Doug

7 Likes

+1. This eliminates unnecessary noise in Swift code, giving it a lighter feel and eliminating a minor day-to-day nuisance with trailing commas. It makes simple things like reordering elements in a large array/dictionary literal easier. And much like with semicolons, once they're gone I realize that I simple don't miss them---because they weren't adding anything to my code

It's a syntactic refinement, but it's one that makes a lot of code just a little less noisy, and the cumulative effect is quite strongly positive. After looking at the examples for a while, the extraneous commas begin to feel very heavy.

In-depth study, including participation in the pitch thread and review of the implementation.

Doug

4 Likes

I'm an enthusiastic supporter of this change.

I was pretty disappointed to see SE-0084 rejected back in the day. I have often found the lack of trailing commas on expressions frustrating given the circumstances that make them useful for arrays apply to other expression lists too.

The rejection rationale also didn't ring true:

For parameter lists and tuples (the specific topic of the proposal), the trailing terminator of the list is typically placed on the same line as the other elements.

The standard library actually follows a formatting convention (subsequently also adopted by Google's style guide) that absolutely can end up with parameter lists with their trailing terminator on the subsequent line, so all the same good arguments in favor of trailing commas in arrays (like reducing diff noise) also apply. I raise this because a lot of the discussion has been focused on DSLs, but this is an example of it impacting regular code.

So I had for a while been hoping that the subject of SE-84 might get revisited. It had never occurred to me that eliding commas, just like eliding semi-colons, was a far more elegant solution. The subsequent pitch discussion has reinforced this, to the point where, like @nnnnnnnn points out, some of the commas I see are now really bugging me! Thinking about how the standard library would look without them makes me confident this would be a positive change for the language.

I understand that some might feel we've gone beyond that point in the evolution of the language. I really hope not. This kind of change – the adoption of which is entirely discretionary and can be done at the pace of the adopter's choosing – seems like the perfect way to evolve the language and continue to make it better for many new use cases.

9 Likes

I couldn’t agree more. The recent period has been the most active in a long time for SE. This is very exciting! One of the most exciting things about it for me has been seeing new community compiler implementers making contributions. I think it is perfectly reasonable for new implementer to choose to start with relatively small features, which will often be some kind of syntactic sugar. It would be quite unreasonable to expect such contributors to start with large features that impact every corner of the language, change the type system, etc. These contributions are not holding up work on larger features. They represent additional work that would not otherwise happen (at least not right now).

For this reason, I find the tone in some of the comments in the threads about this topic rather troubling. There have been a few that are rather condescending with a rather strongly negative vibe. IMO, this is disrespectful towards the time and effort @nate_chandler has invested in getting up to speed on the compiler and making contributions to the community. We can and should be able to disagree without being disrespectful or condescending. In fact, the code of conduct requires this of all of us. I hope the remainder of this review thread will take on a more respectful tone, even where people strongly disagree with this proposed direction.

9 Likes

After spending some time with the toolchain, I rather like it.

I’m a comfortable but not vehement +1 on this, with one caveat about function declarations.

  • What is your evaluation of the proposal?

I came to it with mixed feelings.

On the one hand, I am an enthusiastic proponent of removing syntactic noise. I used to sneer at syntactic gymnastics to reduce delimeters, but actually working with languages that did such gymnastics convinced me of the readability benefit. It’s simply easier to see the code if there’s less visual distraction in it. I’ve come to believe that Edward Tufte’s “data-to-ink ratio” applies to code as much as any other form of information design.

On the other hand, this seemed like such a fiddly little change that it’s hardly worth the syntactic confusion I assumed it would create. And I assumed there would be lots — as there is with Javascript’s vexingly almost-but-not-quite-optional semicolons.

After spending some time with the toolchain, I didn’t find any Javascript-like bizarre corner cases, even with my bizarre taste in formatting. (Whitesmiths! don’t @ me) This proposal really does slip into the language with shocking ease, just as its authors suggest!

The result is often nothing more than a subtle and pleasant shift:

return RequestTransferMetrics(
    requestBytesSent:      task.countOfBytesSent
    requestBytesTotal:     task.countOfBytesExpectedToSend
    responseBytesReceived: task.countOfBytesReceived
    responseBytesTotal:    task.countOfBytesExpectedToReceive)

…but in some cases opens up striking possibilities, as in this fussy little helper method that sans commas becomes reminiscent of Smalltalk’s conditionals:

withOwner(owner
    ifObserver: {
        observerIsOwner = false
    }
    else: {
        externalOwners.insert(WeakRef(owner))
    }
)

Here a print statement benefits from breaking a long string across lines:

print(
    "WARNING: Received response for request that was already completed:"
    delegate.requestDescription
    "This may indicate a bug in your NetworkingProvider, your custom"
    "RequestDelegate, or Siesta itself. If it is the latter, please"
    "file a bug: https://github.com/bustoutsolutions/siesta/issues/new"
    "\n    Previously received:", existingResponse
    "\n    New response:", newResponse)

The real clincher for me is what the proposal does to SwiftPM package config:

let package = Package(
    name: "Paper"
    products: [
        .executable(name: "tool", targets: ["tool"])
        .library(name: "Paper", targets: ["Paper"])
    ]
    dependencies: [
        .package(url: "http://github.com/SwiftyJSON/SwiftyJSON", from: "1.2.3")
        .package(url: "../CHTTPParser", .upToNextMinor(from: "2.2.0"))
        .package(url: "http://some/other/lib", .exact("1.2.3"))
    ]
    targets: [
        .target(
            name: "tool"
            dependencies: [
                "Paper"
                "SwiftyJSON"
            ])
        .target(
            name: "Paper"
            dependencies: [
                "Basic"
                .target(name: "Utility")
                .product(name: "CHTTPParser")
            ])
    ]
)

That’s nice. The raggedy commas in Package.swift always bother me!

The one snag I hit is that the proposal does not support comma elision in function signatures:

public func request(
        _ method:        RequestMethod   // compiler error here
        data:            Data
        contentType:     String
        requestMutation: @escaping RequestMutation = { _ in })
    -> Request

The language really ought to support comma elision in the declaration, since it supports it in the corresponding call. It will certainly cause frustration otherwise.

  • Is the problem being addressed significant enough to warrant a change to Swift?

This is certainly debatable (as the debate here shows).

Opening up the possibilities of more fluid and readable Swift DSLs seems to me to be worth the fuss, but I don’t have vehement feelings about this.

  • Does this proposal fit well with the feel and direction of Swift?

To the extent that Swift goes out of its way to reduce visual noise (optional semicolons, type inference, .-prefix use of the contextual type as an implicit namespace, trailing closures, anonymous closure args, type sugar for arrays and dicts), I’d say yes, it does fit with the feel of Swift.

To the extent that Swift is a C descendant, it feels odd. However, Swift is syntactically already a long way from C; see previous paragraph.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

My time with Ruby, Python, Smalltalk, and Elm — all relatively low-delimeter languages — has made me appreciate the value of reduced visual noise in code.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I only skimmed the proposal, but spent some quality time with the toolchain.

11 Likes

This is a great way to experience a proposed new feature, especially one that has a stylistic impact like this proposal. Cool!

Doug

7 Likes

Ben’s sentiment here resonates with me. Swift is a pleasure more often than not, but still doesn’t feel fully mature. Despite the push for stability of user experience in recent versions of Swift, I’d like to think it still has room to grow.

Whether this proposal is accepted, I hope that we will keep shaving away the language’s burrs and rough spots, with an eye not just to completeness (huzzah for the recent generics push!) but also to lowering user friction.

1 Like

I'm exceedingly aware of the rules around whitespace, the special cases we built to make semicolon elision work in practice, as well as the sharp edges. Are you claiming that there is a widespread use of necessary semicolons outside of intraline separators? Please provide some examples (i.e., links to github projects).

The proposal gives several examples of where comma separation will be necessary in common cases and prominent members of the community have already pointed out that this proposal is a fail for core infra of the Swift project like SwiftPM, so I have a hard time understanding your position here.

-Chris

1 Like

I'm sorry for the delay and interruption responding to this thread. As the core team was aware when they scheduled and ran this, I was traveling and have not had time nor internet connectivity to participate in a very engaged way.

As I pointed out with a detailed comment, this is not the case despite repeated claims about it. Semi colon elision allows effectively 100% elimination of semicolons which make end of line separators effectively "not exist". This proposal eliminates some large but non-total number of commas (80%, 90%, has anyone actually measured?) which leads to the language being inconsistent, and feel unpredictable to users. These are completely different things. Unless the proposal effectively eliminates commas as line separators, it is not at all in the same ballpark.

No, it is not. Again, the point is this is not at all consistent like semicolon elision is, and I have a hard time understanding how you would continue to claim this when the proposal itself explicitly acknowledges this. This is a special case that makes some demos look less punctuated, but when prominent members of the community are saying things like:

"""
I have never looked at code and thought "you know what this needs? fewer commas".
"""

Then perhaps there is no problem that needs to be solved here.

I apologize for not responding. As I mentioned before, I have been traveling, which you knew.
The standard expectations of the community for pitch thread is that the community does not need to follow the entire thread - the expectations are that the feedback from the pitch threads are incorporated into the proposal which was not done in this case.

Agreed, you are correct, there is no backwards compatibility issue here.

Correct, but those cases (by design!) don't happen in a statement scope because they don't produce a side effect. No-op values at statement scope aren't something people try to write, and if they did, they wouldn't be something you'd make explicitly "work" by adding semicolons. Swift was designed for this specific case, and claiming that the same logic generalizes to commas is missing the core design of the expression grammar.

This is ignoring the indentation rules that make that case "do the right thing" in practice for statements which does not generalize to commas. However, I'll ignore that for the moment to dive into the deeper issue:

Let me rephase your argument: you are saying that this proposal is going to regress the user experience of using swift so badly that you'll be forced to do QoI improvements that could have been done at any point, but which never mattered enough to justify the engineering effort. You appear to be claiming that this is a net positive for the Swift language, community, and long term evolution of the language. I will simply state that this logic doesn't make sense to me.

Among other things, I would love to see the data you have that shows this is the only user experience regression that people will hit in practice, I would also love to see the code bases that you've converted over, and the indication of how successful comma elision will be in practice (90%? 99.9%?) and therefore how inconsistent we can expect Swift code to be in the future in practice if this proposal is accepted.

This proposal is a huge change to the user-visible feel of the Swift language. Such a change, particularly given the low benefit demands overwhelming evidence that this will be a positive move for the language and community. I simply cannot believe how fast you are apparently pushing for this to happen. Even if this did make the Swift language better, doing this with the proper data, justification and experience would be absolutely expected of any other sugar proposal - I don't see why such an invasive change could be done with less diligence.

-Chris

8 Likes
Terms of Service

Privacy Policy

Cookie Policy