Higher order functions and function types in Swift

This program compiles as expected:

func hmm(_ fn:  (Int, Int)  -> Int) {}
func hmm(_ fn: ((Int, Int)) -> Int) {}

Now, the problem is that I can't seem to use these in any way at all without getting a compile time error.

You'd think it should be straight forward; the first hmm takes functions of two Ints returning an Int and the second hmm takes functions of a pair of Ints returning an Int.

But as far as I can tell, Swift can only tell them apart just enough to allow their declaration, and every attempt to touch them afterwards will result in a

ERROR: Ambiguous use of 'hmm'

I've tried Xcode 10.1 and a recent Swift 5 snapshot.

2 Likes

This seems like a bug in a post-SE-0110 world. I know we still have logic to let closure expressions match either type, but even if I assign the closure to an explicitly-typed variable first I still can't call the first one. (I can call the second one that way.)

func hmm(_ fn:  (Int, Int)  -> Int) { print("two ints") }
func hmm(_ fn: ((Int, Int)) -> Int) { print("pair") }

func test() {
  let twoIntsFn: (Int, Int) -> Int = { $0 + $1 }
  let pairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }

  hmm(pairFn)
  hmm(twoIntsFn) // fails
}

test()

@rudkx, @xedin, what do you think? Even if we continued accepting the conversion it seems like the non-conversion one should be preferred.

2 Likes

What version are you using to get that result?

Here's what I get with both the default toolchain of Xcode 10.1 and dev snapshot 2018-12-04:

func hmm(_ fn:  (Int, Int)  -> Int) { print("two ints") }
func hmm(_ fn: ((Int, Int)) -> Int) { print("pair") }

func test() {
    let twoIntsFn: (Int, Int) -> Int = { $0 + $1 }
    let pairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }
    
    hmm(pairFn) // ERROR: Ambiguous use of 'hmm'
    hmm(twoIntsFn) // ERROR: Ambiguous use of 'hmm'
}

test()

Or more specifically:

test.swift:8:5: error: ambiguous use of 'hmm'
    hmm(pairFn) // ERROR: Ambiguous use of 'hmm'
    ^
test.swift:1:6: note: found this candidate
func hmm(_ fn:  (Int, Int)  -> Int) { print("two ints") }
     ^
test.swift:2:6: note: found this candidate
func hmm(_ fn: ((Int, Int)) -> Int) { print("pair") }
     ^
test.swift:9:5: error: ambiguous use of 'hmm'
    hmm(twoIntsFn) // ERROR: Ambiguous use of 'hmm'
    ^
test.swift:1:6: note: found this candidate
func hmm(_ fn:  (Int, Int)  -> Int) { print("two ints") }
     ^
test.swift:2:6: note: found this candidate
func hmm(_ fn: ((Int, Int)) -> Int) { print("pair") }

The result is the same for both a debug and release build.

I was using the master-ish build that was on my laptop. I guess things have changed since then, or that master has diverged from 5.0 on this already.

Some more examples of the same issue(s):

func takesTwoIntsFn(_ twoIntsFn: (Int, Int) -> Int) {
    print("twoIntsFn:", type(of: twoIntsFn))
}
func takesIntPairFn(_ intPairFn: ((Int, Int)) -> Int) {
    print("intPairFn:", type(of: intPairFn))
}

func test() {
    let twoIntsFn:  (Int, Int)  -> Int =  { $0   + $1   }
    let intPairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }

    print(type(of: twoIntsFn)) // (Int, Int) -> Int
    print(type(of: intPairFn)) // (Int, Int) -> Int   (bug?)
    print(type(of: twoIntsFn) == type(of: intPairFn)) // false (as expected)

    takesTwoIntsFn(twoIntsFn) // (Int, Int) -> Int
    takesIntPairFn(intPairFn) // (Int, Int) -> Int    (bug?)
    
    // Note that the following compiles, but I assume it shouldn't?
    takesTwoIntsFn(intPairFn) // (Int, Int) -> Int    (bug?)
    takesIntPairFn(twoIntsFn) // (Int, Int) -> Int    (bug?)
    
    // While the following line results in an expected error:
    // let a: [(Int, Int) -> Int] = [twoIntsFn, intPairFn] // ERROR: "Cannot
    // convert value of type '((Int, Int)) -> Int' to expected element type
    // '(Int, Int) -> Int'"
    // note that the following compiles without errors:
    var arrayOfTwoIntsFns = [(Int, Int) -> Int]()
    arrayOfTwoIntsFns.append(twoIntsFn)
    arrayOfTwoIntsFns.append(intPairFn) // (bug?)

    var arrayOfIntPairFns = [((Int, Int)) -> Int]()
    // arrayOfIntPairFns = arrayOfTwoIntsFns // ERROR as expected
    arrayOfIntPairFns.append(arrayOfTwoIntsFns[0]) // (bug?)
    arrayOfIntPairFns.append(arrayOfTwoIntsFns[1]) // (bug?)
    // arrayOfIntPairFns[0] = arrayOfTwoIntsFns[0] // ERROR as expected
    // arrayOfIntPairFns.append(contentsOf: arrayOfTwoIntsFns) // ERROR as
    // expected but with a nonsensical error message.
}

test()

And, I think the following is a bit puzzling to think about:

func test() {
    let twoIntsFn:  (Int, Int)  -> Int =  { $0   + $1   }
    let intPairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }

    // Some inputs of type (Int, Int):
    let inputs: [(Int, Int)] = [(1, 2), (3, 4), (5, 6)]

    // Both function types can be used with this (pair) input: 
    print(inputs.map(twoIntsFn)) // [3, 7, 11]
    print(inputs.map(intPairFn)) // [3, 7, 11]

    // I guess this have to be expected behavior (unless Swift introduced
    // some other way than tuples to define arg-lists), but it raises some
    // questions about in what sense the two functions have the same or
    // different type ...
}

// And, if (1) the above should continue to be expected behavior, is that
// really consistent with (2) treating the following (or the overloaded hmm
// in the OP) as different functions? I think 1 and 2 might be mutually
// exclusive ...

func foo<A, B, R>(_ fn: (_ arg1: A, _ arg2: B) -> R) {
    print("A two-args-function!")
}

func foo<A, B, R>(_ fn: (_ pair: (A, B)) -> R) {
    print("A one-pair-arg-function!")
}

test()

(I get the same result with default toolchain of Xcode 10.1 (10B61) and dev snapshot 2018-12-06.)

This area of Swift, ie essentially the whole conflation-of-parentheses-complexity-of-problems, including single-element tuples etc, has been broken in various different ways for years.

Any chance of these things ever getting fixed, or is it too complex and too late? (serious question)

Can anything be said about the priority/severity, relation to ABI stability and future language features, etc?

Filed SR-9446 for the issue of higher order function accepting invalid arguments, although I'm not sure what fixing that would mean for the above example, repeated here:

func test() {
    let twoIntsFn:  (Int, Int)  -> Int =  { $0   + $1   }
    let intPairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }

    // Some inputs of type (Int, Int):
    let inputs: [(Int, Int)] = [(1, 2), (3, 4), (5, 6)]

    // Both function types can be used with this (pair) input: 
    print(inputs.map(twoIntsFn)) // [3, 7, 11]
    print(inputs.map(intPairFn)) // [3, 7, 11]

    // I guess this have to be expected behavior (unless Swift introduced
    // some other way than tuples to define arg-lists), but it raises some
    // questions about in what sense the two functions have the same or
    // different type ...
}

Seems to me there is a situation where we have to choose between two seemingly buggy behaviors, or there is some other more satisfying solution.

The implementation of function types has gotten a lot better and most of the technical debt of the old representation is gone.

However there is an intentional implicit conversion between a function taking multiple arguments and a function taking a tuple argument in some narrow cases. This was introduced so that you could, for example, pass a closure taking two arguments to map() on a dictionary (where the elements are logically tuples).

Single-element tuples are a separate issue not related to the function type representation. It's a matter of banning single-element tuple expressions, and I really should just fix that.

2 Likes

We can't remove these conversions, because it would break source compatibility.

Note that gmm(pairFn) is ambiguous only with -swift-version 4, because we allow the splatting conversion in both directions. If I recall, this was due to an oversight in the old implementation.

In -swift-version 5, only one direction is allowed.

If you look at CSSimplify.cpp you'll see that now both conversions are explicitly codified because the new representation is quite different between the two arguments and single argument of tuple form.

It would be nice if ranking would prefer the version without the conversion if possible. Ranking is itself a bit of a mess right now, and needs some focused redesign.

2 Likes

I explained what's going on with the implicit conversion in the bug.

I'm not sure why the behavior is buggy. Note that the two functions have different types, ie

print(type(of: intPairFn) == type(of: twoIntsFn))

Will print false.

Huh, I just realized that my command line target's Swift Language Version has been set to 4.2 while I've been switching between the default toolchain and the recent master dev snapshot (which I thoughtlessly just expected to use Swift 5) ... So I got the wrong impression about the Swift 5 behavior ... Happy to see that I made a stupid mistake and things have been moving in the right direction.

However, the option for 5 is lacking from Xcode's UI, there's only 3, 4, 4.2 and unspecified, I've asked about this in a separate thread.


I just figured the overloaded-parentheses-and-single-element-tuples-thing might have something to do with things like the following printed function type representation SR-8235:

func f(_ a: Int, _ b: Int) {}
func g(_ pair: (Int, Int)) {}
func h(_ fn:  (Int, Int)  -> Void) {}
func i(_ fn: ((Int, Int)) -> Void) {}
func test() {
    print(type(of: f)) // Prints (Int, Int) -> ()
    print(type(of: g)) // Prints (Int, Int) -> ()    <--- bug?
    print(type(of: h)) // Prints ((Int, Int) -> ()) -> ()
    print(type(of: i)) // Prints ((Int, Int) -> ()) -> ()  <--- bug?
}
test()
I'm really sure this is the Swift 5 behavior this time.
$ cat > test.swift
func f(_ a: Int, _ b: Int) {}
func g(_ pair: (Int, Int)) {}
func h(_ fn:  (Int, Int)  -> Void) {}
func i(_ fn: ((Int, Int)) -> Void) {}
func test() {
    print(type(of: f))
    print(type(of: g))
    print(type(of: h))
    print(type(of: i))
}
test()
$ /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-12-06-a.xctoolchain/usr/bin/swiftc --version
Apple Swift version 5.0-dev (LLVM 491b123d06, Clang 9bb9e9884e, Swift 00d2acd809)
Target: x86_64-apple-darwin18.2.0
$ /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-12-06-a.xctoolchain/usr/bin/swiftc -swift-version 5 test.swift && ./test
(Int, Int) -> ()
(Int, Int) -> ()
((Int, Int) -> ()) -> ()
((Int, Int) -> ()) -> ()
$ 

Why was the distinction not generally dropped for the sake of consistency, so that the second hmm would count as an invalid redeclaration? I feel this would also avoid some tuple destructuring in closures and weird "double" parentheses ...

I think the current Swift 5 behavior is still clearly buggy:

func hmm(_ fn:  (Int, Int)  -> Int) { print("two ints") }
func hmm(_ fn: ((Int, Int)) -> Int) { print("pair") }

func test() {
  let twoIntsFn: (Int, Int) -> Int = { $0 + $1 }
  let pairFn: ((Int, Int)) -> Int =  { $0.0 + $0.1 }

  hmm(twoIntsFn) // error: ambiguous use of `hmm` <--- buggy behavior
  hmm(pairFn) // prints pair, as expected
}

test()

The reason behind that error is either very counterintuitive or a compiler bug.

I actually noticed the behavior you describe last night and pushed a fix. Glad to see there was a bug for it already, not sure why it slipped through the cracks but its fixed now. The problem was that the code to construct a mangling from a type metadata had not been updated for the new representation yet.

4 Likes

There are two overloads of hmm. When you call hmm with twoIntsFn, both overloads apply; the first one as-is, and the second with the implicit conversion that I describe. There's no general rule that says an overload is always preferred if it has fewer conversions or anything like that. So in the absence of a tie-breaker, the type checker concludes that neither solution is better than the other and diagnoses an error.

I agree this is a bit confusing, and in general the solution ranking needs an overhaul, but that's going to be difficult without breaking source compatibility.

2 Likes

As far as I can see, the down sides (complexity, inconsistency, limited expressibility) of allowing that implicit conversion outweigh the upside (an inconsistent "convenience") that motivates it.

For example, it makes it impossible to write a function that can tell a single-(N-tuple)-argument function from an N-argument function in Swift. Or put differently: Higher order functions in Swift can't help but mixing up certain function types. I'm color blind and Swift's functions are tuple-argument blind.

Allowing that implicit conversion was specifically requested by many users after SE-0110 was implemented:

https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170619/037616.html

Without explicit tuple splatting I don't see how this feature can be removed.

1 Like

This was the case before Swift 3 -- all functions logically took a single argument, which could be a tuple, and this assumption was baked in throughout the compiler. In Swift 3 a series of proposals were accepted and implemented:

(The last one is marked as 'deferred' but it was partially implemented; we went back and added the implicit conversion I described as an escape hatch. Note that in the old implementation, this conversion was free because both were literally the same type; when we added it back, we had to write explicit code to thunk between the two representations).

A big source of confusion in Swift 3 and 4 was that these proposals were only partially implemented because internally the compiler still modeled function types as taking a single argument. We finally mostly completed the transition away from this model on the master branch a few months ago.

4 Likes

The following is the behavior of both Swift 4.2 and 5 with recent snapshots:

func f(_ x: Int, _ y: Int) { print("I was given two args:", x, "and", y) }
func g(_ pair: (Int, Int)) { print("I was given one pair:", pair) }

func test() {
    
    let a = f
    var b = g
    
    a(1, 2)    // I was given two args: 1 and 2
    b((3, 4))  // I was given one pair: (3, 4)

    // b = a // expected error: cannot assign value of type …  
    b = { $0 }(a) // 😎
    
    a(1, 2)    // I was given two args: 1 and 2
    b((3, 4))  // I was given two args: 3 and 4 🤔

    print(type(of: a) == type(of: b)) // false 
}

test()

Is this working as intended?

I think b = { $0 }(a) is not really intended to work, but I guess because of how the check is written we accept it...

Slava

On the other hand, I don't know that it's supposed to not work, though. It is using the closure to get the implicit conversion escape hatch that closures are supposed to provide.

I guess we've unintentionally created a horrible syntax for explicit tuple splatting... :crazy_face:

func f(_ x: Int, _ y: Int, _ z: Int) {
  print(x * y * z)
}

let x = (1, 2, 3)
({ $0 }(f) as ((Int, Int, Int)) -> Void)(x)
3 Likes