SwiftPM Build Animation

Since August, on master, SwiftPM builds using llbuild as a library instead of through an executable, which allows it to access more information on the progress of the build and more importantly to generate its own command-line output. For now, SwiftPM just replicated the build output that the llbuild executable used to generate, but I'd like to move forward and provide a more pleasant and useful progress animation.

Before I go forward with this, I'd like to get some feedback on what kind of presentation people would prefer. The two important questions are:

  • Should it be percentage or item based? (item = compilation unit)
  • Should it display a progress bar?

To help discussion, here are two examples that I could quickly capture:

item based without a progress bar (ala Ninja)

percentage based + progress bar (ala SwiftPM tests)

Please disregard the fact that the Ninja example updates per-file and that the SwiftPM example updated per-module, this is something that will be improved on independently.

9 Likes

Fantastic work! Both examples look very nice.

I think that item based display is slightly more useful, but then again I can see why someone would prefer to have a percentage. Since you seem to have both implemented, why not make it a flag?

Is the jumping insertion point in the progress-bar row something that can be fixed, or just an oversight in the recording?

Out of curiosity, why is the cursor above the line it ends at? At first I thought it was part of the progress bar, but then I realized that it's part of the second line but it always appears on the line above it.

Awesome!

My personal preference is we start with the Ninja-style, which is well tested and works well in practice.

If we wanted to move to a multiline thing, I would rather explore something a la what Buck has, which displays the behavior on multiple cores. However, I think this can also be quite noisy, so I'd prefer start simple and then explore further innovations...

Thanks for working on this, I have long wanted something better than what we have now! :slight_smile:

FWIW, at some point (hopefully not too far off) I would like llbuild to start tracking item times, so that it can offer intelligent heuristic/past-behavior based time estimates (as well as maybe things like critical path times). If you were interested in those, I'd propose we try and do them in llbuild so all llbuild clients can leverage them.

2 Likes

Both are not implemented. SwiftPM currently only has code for handling the percentage-based one. I would personally prefer we start with one and put all the effort into making it awesome.

That's a bug I think. Should be fairly easy to fix.

2 Likes

Ah, I see.

I assume you have seen the code in llbuild that has a the Ninja-style one?

If we go Ninja style, should we also display a progress bar above it?

No, but I'll make sure to check it out before doing any implementation work. We'll have to tweak ProgressBarProtocol to support updating through a fraction (34/202) instead of a percentage.

I prefer to start with the single line style, which can be rendered in a few places that multiline ones can’t (terminals which honor \r but not full ANSI codes)

This looks great!

One thing to bear in mind perhaps is that some CI systems have trouble displaying in-place updates. (I think this is what @ddunbar is referring to as well.)

I suspect the former version might give better results in that case.

1 Like

I prefer the Ninja-Style (x/y) because it is clear that it does not show duration but files. What does percent show? Percent of ... files/lines/time?

If the SwiftPM knowns that there will be multiple (x/y), it should make that clear, too:

[module 1/7][file 120/768] <file>

(Include an estimated duration, if possible)

Slow internet/remote session optimisation: Do not update per file but only once per second.

The consensus seems to be on a one-line Ninja-like animation without a progress bar. Are people ok if I replace the default progress bar to that so that the parallel tests also use that same animation? I'd like both testing and building animations to be consistent if possible.

7 Likes

I would be fine with that but we should have an option to use the current one using some env variable maybe?

SWIFTPM_TEST_RUNNER_PROGRESS_BAR=lit

I agree with @finestructure here. CI systems need to be considered. The output of the build process shouldn't be destructive in these cases. We need a full output log in CI.

2 Likes

We detect if the output stream is tty and switch the progress bar. Some services (like Travis) have issues with that so we can just provide an option to control the progress bar type.

Do you have a bug reference that explains this issue?

Really love the suggestion to have ability to switch between multiple output styles, like Ninja style, Buck style, simple output for CI and so on. I find Buck's per build thread output very helpful.

For those interested to try it out, the new ninja-style progress animation has been implemented and is pending review: https://github.com/apple/swift-package-manager/pull/1782

It's now merged in master. Thanks everybody for your feedback.

8 Likes
Terms of Service

Privacy Policy

Cookie Policy