Variadic Zips... are they possible yet?

Development Compiler
1

I’m playing around with various forms of zip for a class I’m teaching. I can write the following:

func zip<A, R, S>(
_ f: @escaping (A) async -> R,
_ g: @escaping (A) async -> S
) -> (A) async -> (R, S) {
    { a in await (f(a),g(a)) }
}

and I can make it variadic

func variadicZip<A, each R>(
    _ fs: repeat @escaping (A) -> each R
) -> (A) -> (repeat each R) {
    { a in (repeat (each fs)(a)) }
}

and I can write the inout form:

func nonCopyingZip<A: ~Copyable, R, S>(
    _ f: @escaping (borrowing A, inout R) -> Void,
    _ g: @escaping (borrowing A, inout S) -> Void
) -> (borrowing A, inout (R, S)) -> Void {
    { a, pair in
        f(a, &pair.0)
        g(a, &pair.1)
    }
}

the problem is that I can't seem to make this form variadic. Every attempt I make doesn't just fail to compile, it crashes the compiler. Example:

func variadicNonCopyingZip<A: ~Copyable, each R>( // escapes, serializes
    _ fs: repeat @escaping (borrowing A, inout each R) -> Void
) -> (borrowing A, inout (repeat each R)) -> Void {
    { a, outputs in
        let pairs = repeat (each fs, each outputs)
        for var pair in repeat each pairs {
            pair.0(a, &pair.1)
        }
    }
}

This yields:

1.	Apple Swift version 6.2 (swift-6.2-RELEASE)
2.	Compiling with the current language version
3.	While evaluating request ASTLoweringRequest(Lowering AST to SIL for file "...")
4.	While silgen emitFunction SIL function "@$s20Tests21variadicNonCopyingZipyyx_q_q_Qp_tztcyx_q_ztcq_QpRv_Ri_zr0_lF".
 for 'variadicNonCopyingZip(_:)' (at ...)
5.	While silgen closureexpr SIL function "@$s20ParallelizationTests21variadicNonCopyingZipyyx_q_q_Qp_tztcyx_q_ztcq_QpRv_Ri_zr0_lFyx_q_q_Qp_tztcfU_".
 for expression at [... line:38:5 - line:43:5] RangeText="{ a, outputs in
        let pairs = repeat (each fs, each outputs)
        for var pair in repeat each pairs {
            pair.0(a, &pair.1)
        }
    "

Question: should this be doable?

1 Like
2

and. fwiw, I get similar crashes trying to make these functions be async. This was just the simplest form that shows the issue. I would also add that the various AI tools don't seem to be able to offer anything useful either.

3

Hi rvsrvs, I’m investigating another crash in the compiler around packs (https://github.com/swiftlang/swift/issues/84490). I think this issue is different but it could have a similar related cause. I think this issue is worth a separate report.

1 Like
4

I also made an issue for the crash related to variadic zips:

1 Like
5

Here ya' go: https://github.com/swiftlang/swift/issues/84568

3 Likes
6

The bigger question remains... I know that the code above is wrong, but I can't quite figure out the syntax for inout parameters like this that should work of if, at this stage of swift's development, this should be doable at all. Any suggestions appreciated.

7

There are workarounds: Variadic generic zip · GitHub

8

so, if I'm understanding that pattern, the key is going to be to create a nominal type to use internal to the function which will allow me use mutating instead of directly acting on the inout tuple being passed in?

9

Yes it avoids direct mutation of the parameter pack. There are quite a few open issues related to mutable parameter packs in the swift repo, a few of which I’ve filed myself

10

I don't know if it's official, but they're not supported (for closures).

Compiles.

func assign<each Parameter>(
  value: repeat each Parameter,
  to output: inout (repeat each Parameter)
) {
  output = (repeat each value)
}

func call<each Parameter>(
  function: repeat (each Parameter) -> Void,
  parameter: repeat each Parameter
) {
  repeat (each function)(each parameter)
}

Error. No way to use an & to fix it.

func call<each Parameter>(
  function: repeat (inout each Parameter) -> Void,
  with parameter: repeat inout each Parameter
) {
    repeat (each function)(each parameter)
}

Unnecessary inout Crashes compiler.

func call<each Parameter>(
  function: repeat (each Parameter) -> Void,
  parameter: repeat inout each Parameter
) {
  repeat (each function)(each parameter)
}

The simplest form is also uncallable.

func f<each Parameter>(_: repeat inout each Parameter) { }

var v = 1
f(&v) // Couple of nonsense errors.
11

hmm. now you guys are making me go check out what happens with borrowing and consuming. nothing good is my guess.

12

Hah. You can get it to work, but the compiler can't deal with it outside of 1-tuples. It gets confused about whether the functions are supposed to work with tuples, or their constituents.

func variadicNonCopyingZip<A: ~Copyable, each R>(
  _ f: (repeat Each<(borrowing A, inout (repeat each R)) -> Void, each R>)
) -> (borrowing A, inout (repeat each R)) -> Void {
  { repeat (each f)($0, &$1) }
}

typealias Each<Value, each Element> = Value

var variable = 1
variadicNonCopyingZip({ $1 += 1 } as (_, inout _) -> _)((), &variable)
#expect(variable == 2)
13

Ok, got it*.

var tuple = (2, 2, 2)
variadicNonCopyingZip(
  { $1 += $0 },
  { $1 *= $0 },
  { $1 -= $0 }
)(3, &tuple)
#expect(tuple == (5, 6, -1))

let zipped = variadicNonCopyingZip(
  { $1 += $0 } as (_, inout Int) -> _,
  { $1 *= $0 },
  { $1 -= $0 }
)
zipped(2, &tuple)
#expect(tuple == (7, 12, -3))

func variadicNonCopyingZip<A: ~Copyable, each R>(
  _ f: repeat @escaping (borrowing A, inout each R) -> Void
) -> (borrowing A, inout (repeat each R)) -> Void {
  { a, output in
    output = (
      repeat { f, output in
        var output = output
        f(a, &output)
        return output
      } (each f, each output)
    )
  }
}

You can help a bit with some sort of apply, but I don't think you can do better?

func variadicNonCopyingZip<A: ~Copyable, each R>(
  _ f: repeat @escaping (borrowing A, inout each R) -> Void
) -> (borrowing A, inout (repeat each R)) -> Void {
  { a, output in
    output = (
      repeat { f, output in
        output&.apply { f(a, &$0) }
      } (each f, each output)
    )
  }
}
&.apply 
public struct And<Value> {
  @usableFromInline let value: Value
  @inlinable init(_ value: Value) { self.value = value }
}

postfix operator &
@inlinable public postfix func &<Value>(value: Value) -> And<Value> {
  .init(value)
}

public extension And {
  @inlinable func apply<Error>(
    _ mutate: (inout Value) throws(Error) -> Void
  ) throws(Error) -> Value {
    var value = value
    try mutate(&value)
    return value
  }
}

* Your original for loop compiles when changed to the following, but there's no way to feed that through to the actual output.

for (f, var output) in repeat (each f, each output) {
  f(a, &output)
}
1 Like
14

Very clever! I like it but... unless I am mistaken, its making a copy of output back into itself when you create the tuple. I'm going to spend some time today seeing if I can follow this recipe for manipulating the inputs and create a form where a tuple of MutableSpans are passed in and where the functions operate on MutableSpan instead of directly on the inout tuple.

It would be nice if the authors of MutableSpan provided a mechanism for getting the spans of a tuple directly. (hint hint :) )

15

I suspect that I'm going to need to make a variadically generic, nominal Tuple type to do this.