[GSoC 2025] Swift Testing Project

Hello everyone! :waving_hand:

I’m Mhmd from Egypt, a final-year Computer Science student passionate about iOS development. I was an iOS intern at 4Sale, a marketplace in the Gulf region, and this year, I participated in the Swift Student Challenge with my game.

I love contributing to open-source projects—recently, I worked on migrating a Swift server package from XCTest to Swift Testing and on building a progress bar for Noora , a terminal design system from the Tuist team. That experience aligns well with my current interest: enhancing Swift Testing’s console output as part of GSoC.

After discussing with @smontgomery, who gave me great advice and answered some of my initial questions, he suggested continuing the discussion here. So, I have some new questions and thoughts:

  • This great PR seems highly relevant to our goal. What do you think about considering it for this project? Would it align well with the direction we want to take?

  • You mentioned the IPC mechanism for communication between the harness process and runner process. Could you explain more about how this works in practice? Also, is the harness process similar to the current event system in Swift Testing, or is it fundamentally different?

Right now, I’m getting familiar with Swift Testing and the project scope , exploring similar testing tools to draw inspiration and validate ideas.

Big thanks to everyone who set this project idea and contributed to this project and related efforts—your work has been incredibly helpful in my research! @grynspan @Sajjon @maartene @NeoNacho @hassila @rauhul

I’d love to hear from you and the entire community:

  • Any insights, advice, or resources that could help with this project?

  • How would you like Swift Testing’s console output to look?

Looking forward to your thoughts!

6 Likes

I am not involved and can't judge whether it is applicable to GSoC, but I can only echo what I wrote in that PR - if such terminal progress is generalised and broken out, we'd be very interested in adding it to the Benchmark package!

2 Likes

Hi Mhmd,

Thank you taking an interest in improving Swift Testing.

I do need a bit more time to respond to your questions. But I will try to explain what I would like the console output to look like (I want to add examples, that takes a bit of time). That would then be my personal preference.

1 Like

Hello! I think it would be great with a design which looks good and works on as many systems and terminals as possible.

I think it might be worth looking at yarn for inspiration there - widely used on many different systems.

Personally, I would also like to be able to opt in to other ”themes”, some of which might be platform specific. The drawback of this idea is it creates fragmentation in the community, but I think it might be ok since it would be opt in. Eg a flag passed to swift test or possibly an ENV variable. By theme I mean which ”series” of symbols are used for the various states/updates

1 Like

Appreciate it! Excited to see your thoughts

Hello! Thanks for the POC and idea

I think the main goal of the project should be to establish a solid foundation with
live progress reporting, nested output reflecting the suite hierarchy, and test metadata (display names, tags, etc.) this forms the base of how the output should appear.

When it comes to themes, we have two possible approaches:

  • terminal-agnostic design to support different environments
  • Opt in themes via a flag or ENV variable for customization

Not sure which approach fits best yet

As promised, a little more input about what I would like cli output to look like.
Also, it's probably good to mention that I see the cli output from swift test most often in CI output (locally I use VSCode/Xcode to view test results). And then only when the run didn't report green. So for me, it is a bit of a niche case.

Anyway
For me, vitest is probably my favorite example of a great CLI output testing experience.

Green test experience

> homepage-bff@0.1.0 test
> vitest run


 RUN  v2.1.1 /Users/maartene/Developer/Dojo/GreedIsGoodCalisthenics

 âś“ app.test.ts (7)
   âś“ Greed is Good should (7)
     âś“ return 0 points when no rules apply
     âś“ return 1000 points when we have three ONEs and no other rules apply
     âś“ return 600 points when we have three SIXs and no other rules apply
     âś“ return 700 points when we have three SIXs and one ONE no other rules apply
     âś“ return 100 points when we have one ONE no other rules apply
     âś“ return 1100 points when we have four ONEs no other rules apply
     âś“ return 1200 points when we have four ONEs no other rules apply

 Test Files  1 passed (1)
      Tests  7 passed (7)
   Start at  08:43:10
   Duration  212ms (transform 53ms, setup 0ms, collect 48ms, tests 2ms, environment 0ms, prepare 32ms)

(or image with correct colours


)

Why do I like this overview:

  1. Clear hierarchy of the test suite
  2. Clear summary of results (files and tests)

Compare this to current swift test output:

Note:

  1. there is no indentation
  2. tests run in parallel, so tests start, end, and suite 'summaries' are all mixed.

Red test experience
First, in the summary I can see which tests failed at first glance:

Then, for every test that failed, I get more in-depth information about why it failed and where:

Compare the swift test output:


The main problem for me in parsing this output is that results from the parameterised tests, the overall tests and the suite get mingled in the output stream. Making it more difficult to find the test(s) that failed.

In summary

  • Organising the results by hierarchy (i.e. Suite -> Test) would be great
  • Just adding indentation could be a great first step
  • Risk: If you want to have the hierarchy when reporting, you can only present the output once all tests are done. Which might take a while depending on the test volume. And/or require adding a progress view.

Hopefully this gives you some ideas. Love to hear your feedback!

1 Like

Thanks a lot! I’ve been exploring similar ideas and looking into how we can implement this effectively. Your insights are really helpful

1 Like

To add—progress bars are devilishly hard to make portable and functional when all you have is UTF-8 and VT-100 terminal commands!

If you’re referring to progress bar styles and terminal capabilities, this might help—check it out But if you’re talking about handling updates, could you clarify?