Pitch: Eliding commas from multiline expression lists

Thanks to everyone for your feedback so far!

A few points of clarification:

(1) The pitched change would keep commas in the language, as semicolons are part of the language, to be used in cases where there is ambiguity for human readers and where are a particular interpretation needs to be communicated to the compiler.

For example, it would be reasonable to add commas when there is the possibility of ambiguity between prefix and infix operators:

combine(
    1,
    -1,
    +1
)

@DaveZ The situation with semicolons and commas seems even more similar--just like semicolons, commas are only sometimes necessary to provide clarity about where one element ends and the next begins.

(2) This is not a source-breaking change. Extensive source compatibility tests have been run against the changes and no breakages have been found. I went ahead and updated the source compatibility section to reflect this, thanks for the suggestion @beccadax.

The "exceptions" mentioned in the pitch are not cases where the interpretation changes from the current interpretation. Rather, they are cases where commas will remain necessary to indicate that one expression has ended and another is beginning.

For example, the following is currently legal code:

enum E {
    case one
    case two
}

func foo(_: E, _: E) {}

foo(
    .one,
    .two
)

With this change, it would remain legal code. Removing the comma, however, would change the meaning into something illegal

foo(
    .one
    .two
)

because .two would be parsed as a member access on .one. Note that this also is how that code is parsed today, without this change.

The closure case mentioned in the pitch is similar. The following is currently and would remain legal code:

func foo(_: Int, _ two: (Int) -> Int) {}

let bar = 42

foo(
    bar,
    { $0 }
)

Removing the comma would again change the meaning into something illegal

foo(
    bar
    { $0 }
)

because { $0 } would be parsed as an attempt to invoke bar with a trailing closure. As before, this is another case where the parsing without the comma is unchanged from today.

2 Likes

That's a great point. In fact, @anandabits had a very nice example of exactly this use case in the original thread where this was pitched about three years ago: SE-0084 spinoff: Newlines as item separators - #12 by anandabits . In addition to his excellent example, he makes the following point:

Thanks for this piece of feedback too:

I will survey the examples you listed and more and update the pitch with "before and after", illustrating the readability benefits of comma elision.

2 Likes

Does this pitch allow for commas to be elided in function/method parameter lists and generic parameter lists?

func parseVarRec<
    T: ContiguousBytes
    Header: FixedWidthInteger
>(
    buffer: T
    headerType: Header.Type
    output: (T) throws -> Void
) throws { [...] }
func bar(c: () ->()) { }

let a: [Any] = [
    bar,
    { }
]

print(a) // [(Function), (Function)]

let b: [Any] = [
    bar
    { }
]

print(b) // [()]
1 Like

It’s good that the compiler puts its foot down, but I’m ultimately more concerned with code that’s ambiguous to humans.

While these ambiguities exist in semicolon elision too, it feels like they’re more likely to arise in expression-lists than sequences of statements.

5 Likes

The same argument could have been made for semicolons, yet we "know" they don't provide value; or for commas after "case" declarations in an enum, yet of course we wouldn't have those (despite C having them). I look at something like this (grabbed at random)...

let best_pictures = [
    1973: "The Godfather", 
    1989: "Rain Man", 
    1995:"Forrest Gump"
]

And the commas add nothing except a weird inconsistency at the end with the missing comma . Or an initialization (taken from Alamofire):

let protectionSpace = URLProtectionSpace(
    host: host,
    port: url.port ?? 0,
    protocol: url.scheme,
    realm: host,
    authenticationMethod: NSURLAuthenticationMethodHTTPBasic
)

The commas add nothing to clarity; they're just line noise, like the semicolons we eliminated before.

Your second statement is factually incorrect: allowing the elision of commas does not break any source code. A correct statement would be "if you remove all commas from all expression lists in existing Swift source code, it will break some of that source code."

The "exceptions" part of the proposal is listing those places, and that list should look very, very familiar to a compiler implementer: it's exactly the same set of places where you would need to write a semicolon to separate two statements. Comma elision and semicolon elision are the same feature, with different contexts.

Now, it is the case that expressions with a leading dot (.foo) are more common in expression lists than at statement scope, so the exceptional cases will come up more often with comma elision. However, given that we've seen very few problems in practice with the semicolon elision rules over the last 5+ years, I'm not concerned that we're actually causing problems here.

Juxtaposition is already restricted by semicolon elision for statements. It's not a viable mechanism for new bricks (which I hope are tasty ones), so this sugar isn't cutting off future evolution.

Regarding the "extra comma" discussion, some times you need to see what the cost of uniformity is (e.g., put commas after everything as a stylistic choice) to challenge your own assumptions. I thought we needed them, and had convinced myself that somehow semicolons were unnecessary yet commas were, and now that I've looked at it a lot closer... I was wrong. We just don't need those commas. They're noise and they get in the way.

I would have thought so, too! My main concern would be around the

  foo
 .bar

cases, for which we've never had good diagnostics in the semicolon-elision case because it hasn't been important enough. Now we'll have to do it, and all of Swift will benefit because we're applying the same rule more generally.

The other stuff in the IDE works shockingly well. We took the implementation of this and dropped it into SourceKit to see what would break when editing Swift sources where extraneous commas have been dropped, and... it worked. Because semicolon elision has always been a thing, the tools are prepared for it. We don't feel we need any SourceKit/code completion/etc. changes to make this work.

Semicolon elision paved the way for this change, already. We're in a good place to make it, now that we've realized we can.

Doug

20 Likes

Examples included in the pitch are nice and seemingly they look great without separator chars. However I fear lists expressed this way are easy to break with an unintended reformat made by a tired developer or an auto-formatter tool.
Swift becomes sensitive to white space changes like Python. And it is better to avoid.

1 Like

Neat, dictionary literals and named-parameter lists are good examples of some benefit to clarity.

Another would be listing of a struct's decl and the corresponding member-wise init. Example adapted from the benchmark suite:

struct TestConfig {
  let delim: String
  let sampleTime: Double
  let numIters: Int?
  let numSamples: Int?
  let quantile: Int?
  let delta: Bool
  let verbose: Bool
  let logMemory: Bool
  let afterRunSleep: UInt32?
}

let s = TestConfig(
  delim: "abc",
  sampleTime: 42.0,
  numIters: nil,
  numSamples: nil,
  quantile: nil,
  delta: true,
  verbose: false,
  logMemory: true,
  afterRunSleep: nil
)

I tend to stick in that trailing comma and/or forget to put any in the first place. The semi-colons are not needed to separate the decls, so it feels like this initialization shouldn't need it either. (This is an aesthetic and not a rational argument; I don't know the deep ramifications in the language if this were to be adopted).

If I look at benchmark declarations from the suite, like:

public let StringComparison: [BenchmarkInfo] = [
  BenchmarkInfo(
    name: "StringComparison_ascii",
    runFunction: run_StringComparison_ascii,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_ascii) }
  ),
  BenchmarkInfo(
    name: "StringComparison_latin1",
    runFunction: run_StringComparison_latin1,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_latin1) },
		legacyFactor: 2
  ),
  BenchmarkInfo(
    name: "StringComparison_fastPrenormal",
    runFunction: run_StringComparison_fastPrenormal,
    tags: [.validation, .api, .String],
    setUpFunction: { blackHole(Workload_fastPrenormal) },
		legacyFactor: 10
  ),
 ... a dozen more ....

I don't feel like the commas help and are more of an annoyance and distraction. These declarations are almost functioning like a mini-DSL.

@nate_chandler, I'd suggest looking at this suite for some more potential benefits for comma elision.

edit: More examples, this time from StdlibUnittest:

It's not everyday that we're directly calling into the collection validation framework, but it would be nice if it felt more like a DSL:

StringTests.test("AssociatedTypes-UTF8View") {
  typealias View = String.UTF8View
  expectCollectionAssociatedTypes(
    collectionType: View.self,
    iteratorType: View.Iterator.self,
    subSequenceType: Substring.UTF8View.self,
    indexType: View.Index.self,
    indicesType: DefaultIndices<View>.self)
}

StringTests.test("AssociatedTypes-UTF16View") {
  typealias View = String.UTF16View
  expectCollectionAssociatedTypes(
    collectionType: View.self,
    iteratorType: View.Iterator.self,
    subSequenceType: Substring.UTF16View.self,
    indexType: View.Index.self,
    indicesType: View.Indices.self)
}
...

And for declaring new tests to loop over and run

let replaceSubrangeTests = [
  ReplaceSubrangeTest(
    original: "",
    newElements: "",
    rangeSelection: .emptyRange,
    expected: ""
  ),
  ReplaceSubrangeTest(
    original: "",
    newElements: "meela",
    rangeSelection: .emptyRange,
    expected: "meela"
  ),
  ReplaceSubrangeTest(
    original: "eela",
    newElements: "m",
    rangeSelection: .leftEdge,
    expected: "meela",
    closedExpected: "mela"
  ),
  ReplaceSubrangeTest(
    original: "meel",
    newElements: "a",
    rangeSelection: .rightEdge,
    expected: "meela",
    closedExpected: "meea"
  ),
...

In fact, I've found comma separators annoying enough in the past that I use multi-line string literals and a custom .lines() to avoid them! String comparison benchmarks from back in the 4.1 days:

    payload: """
      café
      résumé
      caférésumé
      ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º
      1+1=3
      ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹
      ¡¢£¤¥¦§¨©ª«¬­®
      »¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍ
      ÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãä
      åæçèéêëìíîïðñò
      ÎÏÐÑÒÓÔÕÖëìíîïðñò
      óôõö÷øùúûüýþÿ
      123.456£=>¥
      123.456
      """.lines()

I found this pattern helped clarity (thanks to awesomeness of multi-line string literals), even though it was slightly icky and had runtime overhead (which we make sure to do prior to measuring runtime).

@nate_chandler, if there is no cost to syntactic evolvability of Swift, consider me a convert.

11 Likes

I think the title of this pitch should specify that it's about multiline expression lists, just to avoid any misunderstandings.

Having worked with the Squirrel scripting language, for which commas are optional in multline dictionary construction, I appreciate this pitch. I'll be honest--commas can become ugly and annoying in long lists, such as XCTest's allTests.

However, the strong opposition from @Chris_Lattner3 gives me second thoughts. Would this really make evolution of the language more difficult? Is it possible to think of a contrived example to illustrate this? Are there any other potential issues?

2 Likes

I realize this won't be taken very seriously, but COBOL requires none of commas, semicolons or even line-delimiters. The following 3 sets of statements all of the same meanings.

compute a = b + length(c) 
call s using a b c returning r 
display r

compute a = b 
          + length(c)
call s using a 
             b 
             c
       returning r 
display r

compute a = b + length(c) call s using a b c returning r display r

While I would not recommend this last one, it works just the same.

2 Likes

This is like when you notice that you can see your own nose, and then you can't not see your nose anymore. Now that we're talking about it, those commas look terrible.

(Apologies to everyone reading this who now has to stare at their nose for a while.)

16 Likes

While I appreciate the sentiment for this proposal, the exceptions in the “When will you still use commas?” are reason enough for me to go against it.

For example, while I agree that a multiline list can look nice without commas:

let list: [MyEnum] = [
    foo
    bar
    baz
    bing
]

I’m annoyed that the ambiguity forces me to add a single comma as soon as I want to add a literal enum case:

let list: [MyEnum] = [
    foo
    bar,
    .default
    bing
]

Even worse, it can make a small indentation error result in code that is difficult to parse for humans. It’s easy to forget that commas are sometimes necessary to avoid ambiguity and be caught off thinking the following list is of 4 elements when in fact it’s of 3 elements:

let list: [MyEnum] = [
    foo
    bar
    .barProperty
    bing
]
6 Likes

Ideally, the compiler would warn about that possible mistake and suggest adding a comma. Additionally, a style formatter should probably go further and add commas for all elements in a list that has one or more commas, to make it visually consistent.

This is the same comma requirement we have today, though I acknowledge that argument isn't entirely fair--making comma elision the norm suddenly makes adding commas feel like a standout burden. On the other hand, there's precedent for burdening the user to resolve ambiguities, such as backticking a reserved word.

The compiler won’t warm about it if bar.barProperty is a valid MyEnum. That’s the point: valid code can be harder to understand if not use indented properly.

-1 here. :-1:

Comma is not a noise. It's a necessary separator which would make our code more readable. If you think it's a noise then you should also elide all comma in other cases as well. That would be much more consistent with the motivation.

The exception cases are against the motivation of the pitch itself. The simpler solution would be just allow parameter list to behave like item list so the code below would be accepted by Swift compiler.

print(
    "red",
    "green",
    "blue",
    "cerulean",
)

Everybody happy! :blush:

2 Likes

If anything the statement, as I understood it, by @Douglas_Gregor that removing semi colons blocked the language from evolving properly (in his answer to @clattner) makes me wish for those back instead of removing the commas I also find this to be further sugar that makes the language less readable than with.

So yeah, a -1 on this here.

This is the thing, like with ternary operators or default formatting, that makes me a bit concerned about this is that I would expect this when things such as co-routines / async-await / “actors” model are done dusted, generic existential done, argument labels in closures back, etc... can more things be looked at in parallel? Yes, sure... but there is opportunity cost to thinking about any of them.

Where is this statement?

Apologies, it was not Dave A., but Douglas G. whose words I was referencing.

What I'm saying is that if comma elision leads to a potentially misleading situation, the compiler should warn about it and suggest resolutions: merging the elements into bar.barProperty, separating them with a comma, or specifying Bar.barProperty if such a type property exists.

The code is practically just as deceptive with commas and should be warned about today:

let list: [MyEnum] = [
    foo,
    bar
    .barProperty,
    bing
]
1 Like

The issue is not the deceptiveness. There's no intent to mislead anyone. The issue is that, without knowledge of what bar is, it's not possible for a human to reason about that list.

2 Likes