Proposal: Python's list, generator, and dictionary comprehensions

If further discussion is desired on this topic would folks mind moving it to a new thread? Previous participants from back in 2015 may be getting unwanted notifications/emails from this thread.

4 Likes

Swift has class inheritance because it was designed to inter-op with Objective-C. Hard to call ObjC a popular language.

Beyond that, I think you're being hyper-literal just to argue a point. If I had written that Swift does not take every feature from every popular language merely because it's popular, would that have been better understood?

NIH = Not Invented Here. It's an excuse often cited for why a group reinvents the metaphorical wheel.

Preface: I like Python, it’s the first language I suggest for new aspiring programmers.

As someone who writes Python somewhat often, I do almost always prefer comprehensions over the alternatives, but only because the alternatives suck.

Compare this comprehension:

numbers = [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]

result = [n * 10 for n in numbers if n % 2 == 0]
result.sort() # Sort operates in-place and returns `None`, so you can't chain it.

print(result)

...with how you would need to write it if you used map/filter:

result = list(map(lambda n: n * 10, filter(lambda n: n % 2 == 0, numbers)))
result.sort()

There's a bunch of things that suck about it:

  1. map and filter are free functions (and not methods on something like iterable, so you need to nest them, rather than chain them.
  2. The lambda keyword is kinda heavy weight, and there's no implicit parameter names
  3. The result is a lazy iterable, which you need to copy into a list.
  4. Sort can't be chained

The only other alternative I could think of is to use manual loops, but I don't have to explain why that's suckiest of all.

I think the Swift equivalent of this is simply better, fully stop.

let numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

let result = numbers
    .filter { $0.isMultiple(of: 2) }
    .map { $0 * 10 }
    .sorted()
  1. There's a clear linear flow
  2. Unlike a comprehension, you can always know what's available to you by typing . and looking at the auto-completion results.

Another complaint about comprehensions is that the order of their syntax kinda jumps all over the place. Even for a seasoned Python dev, it can be a bit tricky to write complex comprehensions first time without looking at a reference.

Consider even this simple example, and how the control flow jumps around:


result = { n: n*10 for n in numbers if n % 2 == 0}
#             ^ 3  ^ 1              ^ 2
# 1. First you iterate (in the middle)
# 2. Then your predicate is evaluated to filter (at the end)
# 3. Lastly you transform the value (at the start)
11 Likes

In your example here, doesn't the Python list comprehension evaluate the map/filter in just one combined loop, vs. the two loops for Swift's .filter and .map?

IIRC, that's an implementation detail that's left unspecified, but if you want lazy behaviour (which isn't always faster btw, two sequential loops can be faster than 1 combined one for some data sizes ... it's complicated), you just tack on one more word:

let result = numbers
    .lazy
    .filter { $0.isMultiple(of: 2) }
    .map { $0 * 10 }
    .sorted()
3 Likes

It is worth bringing up other languages which do have for comprehensions that don't read so weirdly. I agree with @AlexanderM's writeup a lot here: the order in which one has to read a python for comprehension is pretty weird.

In scala a for comprehention is rather nice, and reads like this:

// numbers = [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
// result = [n * 10 for n in numbers if n % 2 == 0]

val numbers = List(10, 9, 8, 7, 6, 5, 4, 3, 2, 1)

// Scala 2
val result = 
    for (n <- numbers if n % 2 == 0) { 
        yield n
    }

// Scala 3
val result = 
    for n <- numbers if n % 2 == 0
        yield n

They can also nest nicely:

// Scala
def foo(n: Int, v: Int) =
   for i <- 0 until n
       j <- 0 until n if i + j == v
   yield (i, j)

foo(10, 10).foreach {
  (i, j) => println(s"($i, $j) ")  // prints (1, 9) (2, 8) (3, 7) (4, 6) (5, 5) (6, 4) (7, 3) (8, 2) (9, 1)
}

this also naturally extends to the "everything is an expression" where for comprehentions return values like that, the same way an if also is an expression etc.

So if anything, I'd rather explore a direction of making for more powerful like that, since it already is quite similar to Swift's powerful for + where, if only it also was allowed to yield.

// just an idea
let numbers = [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]

let evenNumbers = 
  for (async) n in numbers where n % 0 == 0 { // can even be async etc,
    yield n 
  }
11 Likes

What is the type of evenNumbers here ? A Sequence<Int> or a List<Int> ?

Would this be allowed ?

let evenNumbers = 
  for n in 0... where n % 2 == 0 {
    yield n 
  }

In Scala the type is determined basically by the map/flatMap/filter signatures involved. A for comprehension ends up just calling those methods on the underlying collection type, so the result type here would depend on what a map does on 0...n. So arguably, for your example of filtering over the ClosedRange<Int> it'd still be a ClosedRange<Int>.

+1 on this! Comprehensions may not be fit for large production codebases where clarity is paramount, but specifically in the realm of numerical programming and ML, they’re almost necessary. There’s a vision for Swift to become a top-tier language for ML and numerics, but without comprehensions, it’s still way too tedious to preprocess matrix and tensor data to feed into a model. This is especially apparent when experimenting in playgrounds as a stand-in for, say, Jupyter, where clarity is less of an issue.

Again, map and filter do exactly that. It’s just different (and imo clearer) syntax.

I think that this is an example of the tension between reading and writing code. Comprehensions are easy to abuse to make code highly unreadable (though I would argue that when used judiciously, they can be more readable than even “map” and “filter”). In most cases, readability is most important, so you might want to avoid comprehensions. However, there are some contexts in which the ease of writing, say, complex matrix manipulation is more important than readability. That’s where comprehensions are invaluable. One of those contexts is the Jupyter notebook (or, in the Swift ecosystem, the playground). Right now, it’s far easier to do this in Python, which is a problem if we want to make Swift the go-to language for numerics.

Can you provide an example? I fail to see how complex matrix manipulation would be any easier to do in Swift using comprehensions.