Why can't variadic functions be "fully abstracted" in this context?

Using Swift
1

This code fails to build with the error message "Cannot fully abstract a value of variadic function type '(repeat each U) -> ()' because different contexts will not be able to reliably agree on a calling convention; try wrapping it in a struct"

func f<T, each U>(
    instance: T,
    kp: KeyPath<T, (repeat each U) -> ()>,
    args: (repeat each U)
) {
    instance[keyPath: kp](repeat each args)
}

This isn't discussed anywhere in the original SE, but I'm going to assume that this is not a bug, and is intended.

But why is it intended, and how does "wrapped it in a struct" help? I tried doing so, to the best of my understanding, and got an increasing strange series of errors, culminating in the compiler crashing while generating SIL.

@convention(swift) doesn't help, so I'm guessing this has more to do with thin, thick, etc conventions, of which my understanding is dim at best.

I'm happy to post what I'm Really Trying to Do if it would be helpful, and I may have the possibility to convert from a "raw" parameter pack to a variadic tuple for the keypath type, though I want to be able to pass the args as a non-tuple ideally.

So there's two asks to this question: first, for my understanding, why does this happen. Second, any pointers on how to work around it?

EDIT: I tried wrapping the args in a tuple, which gets past type checking, then crashes while generating SIL:

func f<T, each U>(
    instance: T,
    kp: KeyPath<T, ((repeat each U)) -> ()>,
    args: (repeat each U)
) {
    instance[keyPath: kp]((repeat each args))
}

One More EDIT: I guess this works:

struct S<T> {
    var wrapped: T
}

func f<T, each U>(
    instance: T,
    kp: KeyPath<T, (S<(repeat each U)>) -> ()>,
    args: (repeat each U)
) {
    instance[keyPath: kp](.init(wrapped: (repeat each args)))
}

I originally thought that this was about wrapping the function, not the arguments, which was kind of silly in retrospect..

2 Likes
2

"Fully abstracting" over a type means that the type is all we know statically about how values of the type are represented in memory. Because Swift supports running generic code without specializing it, our static knowledge of that type may still have "holes" in it that are still generic, and we have to make code with holes in different places interoperate with each other.

For most types, that doesn't matter, because we're happy to always give the type a single representation. It doesn't matter whether we know statically that a value is a Double or only know that it's some type T (that happens to be Double at runtime), it still takes up 8 bytes of memory and has its bits in the same place.

For function values, though, that wouldn't be good. The representation of a function value includes the calling convention for passing arguments and receiving return values. A single representation means a single calling convention. That means either very expensive dynamic calling-convention lowering or always using a less efficient calling convention because of the mere possibility of generics. We don't want either of those, so instead we use a system where we pick the calling convention based on how generic it has to be in a specific context. In most contexts, we have a declaration, like a specific parameter or a property, and that declaration tells us how generic it needs to be. (That's what wrapping the function in a struct achieves — everybody can agree on how generic the struct property is because it's always accessed through the property declaration.) But sometimes we don't have that, e.g. because the declaration itself is generic at that position; KeyPath falls into that case.

In those situations, we have to use a convention that works for any possible generic type that could substitute to the type; that's a rule that works no matter where the holes are. Unfortunately, it turns out that the rules we came up for that ten years ago didn't anticipate variadic generics. Basically, we pass all of the arguments by address, but we pass them individually by address, which doesn't work when we don't even know how many arguments there are statically. So we have to forbid these conversions of function types with a variadic parameters in and out of a fully-abstracted position because we can't make them work because of decisions we made a long time ago. Anything you do to get away from that, like passing a single argument that happens to be a variadic tuple, puts you back in a supportable path because we can make contexts agree on how to represent a tuple.

Everything else you're describing sounds like a compiler bug. Unfortunately, variadic generics is a complicated feature that we're still finding gaps in our implementation of. We are actively working on squashing those, so filing a bug with a concrete test case is your best bet.

10 Likes
3

As John said, please file some bugs for the crashes; the diagnostic is intended, but the crashes are not. The specific advice here is to wrap the entire function value in a struct, like this perhaps:

struct Fn<each T, U> {
  let fn: (repeat each T) -> U
}

If you have a generic container like Array, we can't allow you to form the type Array<(repeat each T) -> U>, because then you could "view" it as Array<(Int, Int, Int) -> String> say, and the representation happens to not work out. But Array<Fn<repeat each T, U>> is totally fine, or Array<Fn<Int, Int, Int, String>>, or whatever.

5 Likes
4

I'll file more bugs.

The specific advice here is to wrap the entire function value in a struct, like this perhaps

So, just to clarify, merely wrapping the arguments like I've done in the last example code isn't sufficient? And should not have been accepted by the compiler?

The codebase I am working in still has a minimum deployment target of iOS 16, which means that I can't have a variadic generic struct. So I've been using various tricks like this:

struct Fn<T, U> {
    let fn: (T) -> (U)

    func callAsFunction<each Argument>(_ arg: repeat each Argument) -> U) where T == (repeat each Argument) {
            fn((repeat each arg))
      }
}

These fake variadic generic structs don't always involve functions; sometimes it's just a matter of making sure the type system knows there can be multiple of some generic things. Is it always bad? Only bad when it involves functions?

Thanks to both you and @John_McCall for these very helpful and extremely detailed answers.

1 Like
5

Wrapping the arguments in a tuple fixes the formal problem with genericity over argument count. Further wrapping the tuple in a struct is theoretically unnecessary but probably does work around compiler bugs in practice.

2 Likes
6

If you do go the route of using a struct wrapper, it's also worth considering how much genericity you really need, since you can get some additional benefit from making the wrapper struct specific to that level of genericity. With something like

struct IntAndOtherArgsFn<each T> {
  let fn: (Int, repeat each T) -> Int
}

since the struct's genericity can't affect the Int parameter or return type, we can always use a direct pass-in-register convention for those parts of the function signature.

1 Like