Understanding the limitations of open existential

Hello!

Trying to understand why this does not compile.

// All okay
do {
  let collections = [[1,2,3], [1,2,3]]
  let total1 = collections.reduce(0) { $0 + $1.reduce(0, +) }
  let total2 = collections.flatMap { $0 }.reduce(0, +)
}

// All okay
do {
  let collections = [AnyCollection([1,2,3]), AnyCollection([1,2,3])]
  let total1 = collections.reduce(0) { $0 + $1.reduce(0, +) }
  let total2 = collections.flatMap { $0 }.reduce(0, +)
}

// Half okay
do {
  let collections: [any Collection<Int>] = [[1,2,3], [1,2,3]]
  let total1 = collections.reduce(0) { $0 + $1.reduce(0, +) } // Okay
  let total2 = collections.flatMap { $0 }.reduce(0, +) // ERROR: Cannot convert value of type 'any Collection<Int>' to closure result type 'Int?'
}

I am trying to understand what the compiler is doing in the last line. Any insights greatly appreciated.

2 Likes

Thanks. You gave me a clue. :smiley: But I still am confused! :sweat_smile:

let collections: [any Collection<Int>] = [[1,2,3], [1,2,3]]
print(type(of: collections.flatMap { $0 }))

This prints:

swift test2.swift
test2.swift:2:28: warning: 'flatMap' is deprecated: Please use compactMap(_:) for the case where closure returns an optional value
print(type(of: collections.flatMap { $0 }))
                           ^
test2.swift:2:28: note: use 'compactMap(_:)' instead
print(type(of: collections.flatMap { $0 }))
                           ^~~~~~~
                           compactMap
<<< invalid type >>>

If I go ahead and change it to compactMap, the program prints:

<<< invalid type >>>

I was hoping for it to print [Int].

FWIW, Swift version 5.8 (swiftlang-5.8.0.124.2 clang-1403.0.22.11.100). I am using a playground but behavior is the same for the command line.

The flatMap you want (as opposed to the one that got renamed to compactMap) is declared like this:

func flatMap<SegmentOfResult>(_ transform: (Self.Element) throws -> SegmentOfResult) rethrows -> [SegmentOfResult.Element] where SegmentOfResult : Sequence

However, any Collection<Int> does not itself conform to Sequence. And the compiler doesn’t open that existential to (effectively) some Collection<Int>, because (a) that only happens for arguments, not return values, and (b) if there were two elements in your outer collection, they might not have the same underlying type. So there’s not actually a type signature you could write for the transform closure that satisfies the requirements using only implicit conversions.

Unfortunately, the compiler then notices the old, deprecated flatMap, and you don’t get a diagnostic about any of what I just said.

6 Likes

Thank you for connecting all of the dots for me! Hope it helps others too. :grinning: