Reusable collections of traits

Related Projects swift-testing
1

Does there currently exist a way to model a kind of "base" reusable collection of traits that can be applied to any @Suite/@Test. I can see grouping some common traits into a single package:

let baseTraits = [
  .resetDependencies,
  .recordSnapshots(true),
  .disabled(if: ProcessInfo.isCI)
  …
]

…and then using those traits on any suite type or test function:

@Suite(
  .tag(.featureA),
  baseTraits
)
struct FeatureATests {
  …
}

I think this style of base traits will really shine once CustomExecutionTrait is public, but even with the simpler traits that exist today it can be useful.

However, this does not compile because a collection of traits is not a trait itself. Could it be? I know that would complicate trait discovery (i.e. Test.current.traits), but perhaps nested collections of traits can be dynamically flattened when added to a @Suite or @Test?

The way one solves this in the XCTest world is to create a base class that all tests inherit from. That is of course very inflexible, and so it seems that traits can greatly improve upon that.

One way I have found to approximate this is to create an empty suite type with the traits:

@Suite(
  .resetDependencies,
  .recordSnapshots(true)
)
struct BaseSuite {}

…and then suites that want to "inherit" these traits can put their tests inside an extension of this type:

extension BaseSuite {
  @Suite(.tag(.featureA))
  struct FeatureATests {
    …
  }
}

However this isn't ideal for a few reasons:

  • It requires an additional level of indentation for each test.

  • If you have multiple sets of "base" traits then you incur even more nesting:

    extension BaseSuite {
  extension FeatureSuite {
    extension SnapshotSuite {
      @Suite(…)
      struct FeatureATests { … }
    }
  }
}

Perhaps what I'm asking for is already possible or planned. Looking forward to hearing!

2

This is a good question. Thanks for asking! :smile:

It is intentional that the testing library doesn't provide API for exactly this functionality. The @Test and @Suite macros need to be able to look "inside" their traits lists in order to do compile-time checking (e.g. to make sure tags are specified correctly, or .bug() has the right format, or .serialized is not applied to a non-parameterized function, or or or…)

However, your approach of nesting in a suite is what we'd recommend today. To avoid extra nesting, you can do something like:

extension BaseSuite {
  @Suite(.tag(.featureA))
  struct FeatureATests {}
}

extension BaseSuite.FeatureATests {
  @Test func f() { /* ... */ }
  // ...
}
3 Likes
3

Thanks for the explanation, that makes a lot of sense. And thanks for the flattening trick, that should help.

1 Like
4

Hi @grynspan, I've been trying to use this pattern of the base test suite via extensions, but sadly it does not play nicely with Xcode at all. If I have a base suite in one file:

// BaseSuite.swift
@Suite()
struct BaseSuite {}

…and then a test that uses this suite in one file:

// FeatureATests.swift
import Testing

extension Tag {
  @Tag static var featureA: Self
}

extension BaseSuite {
  @Suite(.tags(.featureA))
  struct FeatureATests {}
}

extension BaseSuite.FeatureATests {
  @Test func basics() {}
}

extension BaseSuite.FeatureATests {
  @Test func other() {}
}

…and another test that uses the suite in another file:

// FeatureBTests.swift
import Testing

extension Tag {
  @Tag static var featureB: Self
}

extension BaseSuite {
  @Suite(.tags(.featureB))
  struct FeatureBTests {}
}

extension BaseSuite.FeatureBTests {
  @Test func basics() {}
}

extension BaseSuite.FeatureBTests {
  @Test func other() {}
}

…then Xcode cannot properly discover these tests at all. Sometimes Xcode only sees the FeatureATests, and other times only FeatureBTests. And even when it sees both, only one of the tests are recognized properly:

Screenshot 2024-07-15 at 11.32.43 AM
Screenshot 2024-07-15 at 11.32.43 AM550×596 41.7 KB

Note that FeatureATests have a different icon from FeatureBTests (I'm not sure what "rT" means), and clicking on those tests do not jump to the correct file/line.

I understand that this is an Xcode bug, and I did file a feedback (FB14320969), however I am curious if this pattern of base suites will ever play nicely with Xcode. What if I were to extract out BaseSuite to its own module so that it could be used from multiple test targets. Will Xcode be able to see that information?

1 Like
5

That looks like a bug in Xcode's Test Navigator, yes. Thanks for filing feedback! I'm afraid I am not able to answer your question regarding future changes to Xcode.

6

Hi @mbrandonw, this Xcode UI bug should be resolved in the recently-released Xcode 16 Beta 4, as mentioned in the release notes:

  • Fixed: Swift Testing tests in an extension which is in a separate file from the extended type’s main declaration are now discovered and shown in the Test navigator and source editor correctly. (123030494)
7

Hey @smontgomery, thanks for following up, and this does indeed seem to be working much better now!