[RFC] Attachment lifetimes

Development Swift Testing
,
1

Hello Swift enthusiasts! I'm looking for feedback from you all about a future enhancement to attachments in Swift Testing. Read on!

We introduced attachments to Swift Testing in Swift 6.2. This feature allows you to create a file containing data relevant to a test. In XCTest on Apple platforms, you can also specify the lifetime of an attachment by setting the lifetime property of an XCTAttachment object. This lets you avoid serializing and saving an attachment if its data won't be necessary (i.e. "if the test passes, don't bother saving this attachment.")

We didn't initially include this feature in Swift Testing so we could keep the scope of the feature narrow, but we're now looking at implementing it for an upcoming Swift release. So my question to the community is: if you could tell Swift Testing to only record an attachment to a test under certain conditions, what would those conditions be?

For example (these are not the only possible answers!):

  • Always record this attachment
  • Record this attachment only if the current test fails
  • Record this attachment only if the current test passes
  • etc.
4 Likes
2

One use case could factor in issue severity, which was a concept introduced in ST-0013: Test Issue Severity. As an example,

  • Attach if any issues with warning (or higher) severity were recorded.

This would be useful because sometimes an attachment may help track down the reason why a warning-severity issue was recorded.

2 Likes
3

I love this. Yes to all of those options + Stuart’s suggestion of severity-based.

Maybe as an escape hatch, a custom option that evaluates some passed-in closure with the attachment and current state of the test?

1 Like
4

We will likely need to evaluate this property after the test has ended, so it's probably not practical for us to try to encode any "current state", but a closure with barebones info like:

case custom(
  _ filter: @Sendable (
    _ attachment: borrowing Attachment<T>,
    _ testPassed: Bool
  ) -> Bool
)

Might be feasible.

2 Likes
5

Pitch thread is up here.