Let's talk about parameter packs

I work on a few projects that are good targets for parameter packs. Things around testing, command line tooling, etc. It is almost always a struggle against the compiler. Parameter packs are more crash prone than any other language feature I touch.

I’ve reported a few of these crashes over the years, but it seems like they just keep coming. I’m even seeing regressions. This issue was marked fixed, but it looks like it’s crashing again (or maybe it always crashed) on main. I’m starting to feel like this isn’t an issue that will get resolved with one more bug report. There’s something very wrong here.

I started looking into one of them, and I have a fix working, but it got me wondering why parameter packs are so problematic. What do folks who are more familiar with this area of the codebase feel is the root cause?

19 Likes

Just wanted to post this:

I could likely add a few more recent examples of my own, If I will have the time to minimize them.

Point is, variadics appear to be in an poor state, and unfortunately there has been little visible improvement.

5 Likes

Is your current crash repro code different from the original one? If so, would you mind filing a separate issue on GitHub?

1 Like

Broadly speaking there are two reasons for what you’re seeing:

  1. Parameter packs interact with a rather large cross section of the language. There are lots of features that now need to take packs into account but currently don’t, so a long tail of fixes remains.
  2. The implementation of parameter packs hasn’t received much attention since the initial push to get the feature done in the Swift 5.9 development cycle.

I agree this is an unfortunate state of affairs, but it’s temporary and not an inherent property of the feature.

21 Likes

Nope it’s exactly the same snippet. It was crashing as of the December 1st snapshot. Do you still want me to open a new issue?

I can confirm that the first test case seems to have been fixed by @John_McCall's change but the second test case in your bug report still crashes. I re-opened the issue.

7 Likes

I agree... I've been running into this one a lot myself: Fix issue when creating a `some` type from a type with a parameter pack by GeorgeLyon · Pull Request #85894 · swiftlang/swift · GitHub

I’ve been eagerly waiting for parameter packs to be more stable as well. Especially same type requirements would be very nice to see working.

4 Likes

My experience won’t be helpful to get you past these bugs, but simply for the record: when this feature was introduced I immediately implemented something that would have been very desirable in my codebase, more or less:

with(Optional<a>, Optional<b>, Optional<c>, ...) { a, b, c, ... in 
    closure executed only if all Optionals a, b, c wrap an actual value, and arguments to the closure are said value
}

It’s just pseudo-code but you can imagine variations on this theme that might be useful in other contexts (take a list of optionals, execute the closure with the first non-optional value bound as the only argument, etc.). There are obviously other ways to look at/solve this, but since Swift is both obsessed and stimulates obsession into syntactic sugar, this is the road I wanted to take.

Got very excited to see it working but then noticed compilation had become incredibly slow and unpredictable. Far from having the time or desire to dig into compiler sources to understand why this was happening, and whether there was a solution, or it required filing bugs, etc. I simply gave up. This is a story that repeats itself quite often with Swift. The first time you have to rely on -warn-long-function-bodies -warn-long-expression-type-checking to get past a nasty situation, you realize no-one is paying you for this, and maybe rather than getting excited about new language features you have to exercise self-restraint and focus on what creates value. There is a delicate balance between “this feature will make me more productive and hence help deliver sellable features” and “this feature will waste me a week just because I wanted the code to look prettier”. I don’t think it’s normal or desirable for a language to induce its users to think like this so often.

No matter what new feature you come across and get burned from, the game becomes one of searching periodically (mostly these forums) whether the problem you ran into has a solution yet. You are relying on “volunteers”, heroes in my book, with time on their hands to do the dirty work of documenting and filing problems. It feels wrong. It’s one more task that shouldn’t really be part of a developer’s schedule. When Macros were first introduced, early adopters noticed a huge compilation penalty, which was explained and discussed ad nauseam. Is the problem solved now? Macros would be a killer feature for my project, but the more cynical and productive part of me has learned to live without it.

Some on these forums are brilliant enough to understand Swift compiler internals and have time left over to turn that knowledge into better, faster “output” that turns into profit. I, of a lesser breed, see a huge and very different problem. Whether you’re trying to figure out parameter packs or just about any other language feature with its own baggage of problems, you butt against the bureaucracy in charge of all this, a group of exceedingly smart PL enthusiasts that doesn’t seem to understand the huge externalized costs it constantly puts on the developer community. It appears to thrive because enough brilliant people outside this bureaucracy are paying whatever price it takes to test these features and maybe help bring them to a usable state within a couple years (your post suggests the alternative outcome, i.e. a feature left unattended). Under these incentives, the system mostly captures feedback that validates the original goals. You won’t capture any feedback from those who don’t buy into their new role as “unpaid language and framework designer and tester for Apple, Inc.”

Looking at Swift and Apple frameworks I see the cost curve of understanding and using native everything going up and up and up. My best guess is that fewer teams are able to justify or crucially, enjoy, any of this. The unwanted side effect will be to see less, not more, software based on native languages and frameworks rather than vilified web-based equivalents.

These are just opinions of course. Apologies for the long post, I wish nothing but success for Apple’s platforms and all of us here.

4 Likes

Even if it didn't have a toll on compilation performance, this has bad ergonomics compared to if let because you have to match up the names on the inside and outside.

1 Like

Since the trouble I encountered with parameter packs, you can imagine that my code already uses chains of if let … or flatMap or even guard lets with early exits. You’ll have to take my word that a single with(…) statement designed for groups of optionals and with an optional return type made things spectacularly more readable for parts of my code. This being syntactic sugar, I don’t know if it’s worth anyone’s time arguing over issues of taste – subjective as it is. The point of my post is not to draw attention to my specific attempt to use parameter packs for personal gratification, but to how we collectively react to bugs we come across when using new language features advertised in Swift, and what our experience does to adoption rate and that other subjective metric, enjoyability of the language and frameworks as a whole.

2 Likes

I think it's just zip + map?

I haven't seen Gabriele's problem with that yet. It would be nice to have warnings about when we can expect to have them.

As you can see in the comment here, parameter packs require kid gloves, but I think they're still usable …until? :sad_but_relieved_face:

let values = (1, 2.0, "3")
var optionals = values as _?.Mapped

// Parameter packs are in bad shape.
// The placeholder types below are not inferred properly for `var`. 
// Only `let`. 🤷

do {
  let optionals = optionals
  #expect(_?.zip(optionals).map { $0 == values } == true)
}
  
do {
  optionals.1 = nil
  let optionals = optionals
  #expect(_?.zip(optionals) == nil)
}
Parameter pack-using Optional extensions, for the above:
public extension Optional {
  typealias Mapped<each _Wrapped> = (repeat (each _Wrapped)?)
  where Wrapped == Never
  
  @inlinable static func zip<each _Wrapped>(_ optional: (repeat (each _Wrapped)?)) -> Self
  where Wrapped == (repeat each _Wrapped) {
    try? (repeat (each optional).get())
  }
  
  @inlinable func get() throws -> Wrapped {
    switch self {
    case let wrapped?: return wrapped
    case nil: throw nil as Nil
    }
  }
  
  struct Nil: Swift.Error {
    @available(*, unavailable) private init() { }
  }
}

extension Optional.Nil: ExpressibleByNilLiteral {
   public init(nilLiteral: Void) { }
}
1 Like

Just put up 11 issues related to parameter packs

I also have fixes for all of these issues in a fork: Repro15 by alex-reilly-dd · Pull Request #3 · alex-reilly-dd/swift · GitHub
The fixes are likely not very high quality. They were produced with claude code and I don’t have the domain knowledge to review the code in any meaningful way. This fork was produced during a side project I’m pursing at work. I’ve been doing work on it until I run into a compiler crash, tasking claude to fix the crash, and then continuing with my work once the crash has been fixed.

That said, I’ve been having it introduce unit tests for every crash, so the tests are probably worth taking at the very least.

I’m working on open sourcing the project that was used to discover all these crashes, hopefully within the next month or so. I’ll post again when I do since it heavily uses parameter packs, so it might be worth having as a benchmark for the compiler.

5 Likes

One other thing I forgot. Parameter packs are pretty poorly optimized for runtime performance. iirc code that uses parameter packs doesn’t get specialized at compile time or something. Perf is much lower priority than getting these crashes fixed, but it’s worth bringing up anyway.

1 Like