Test authors frequently need to include out-of-band data with tests that can be used to diagnose issues when a test fails. This proposal introduces a new API called "attachments" (analogous to the same-named feature in XCTest) as well as the infrastructure necessary to create new attachments and handle them in tools like VS Code.
I read the future direction but I see both RawSpan and the adoption on stdlib types as accepted and implemented. In particular, this part of the future direction is interesting
and our minimum deployment targets on Apple's platforms do not allow us to require the use of `RawSpan`
That's not true, right? You can just mark all the newly proposed types here with the same availability as RawSpan and then you can use it.
As an Apple employee, I cannot speculate on Apple's future plans. But even assuming a future version of Apple's OS includes RawSpan, we want this interface to be available on current Apple platforms too.
Pure bikeshedding: Does the word "attachable" in the property/parameter names attachableValue/AttachableValue really carry its weight? You can’t very well attach something that isn’t attachable. It feels like this would read much better if the property/parameter names were just value/Value. Or maybe content/Content?
I'm open to renaming the property and/or generic type. The name is fairly subjective:
value felt… bare? Like, we see rawValue and intValue and similar in Swift API, but just value is uncommon.
rawValue doesn't fit perfectly either because this type doesn't conform to RawRepresentable (and can't in the general case because it's conditionally move-only.)
content isn't otherwise used as a term of art here, so I wouldn't reach for it unless it were established convention elsewhere for this sort of concept.
Bikeshedding the names is a good thing at this stage—happy to consider other alternatives!
Aren’t all of those to disambiguate from other types of values? "raw" in rawValue is to disambiguate from other possible value representations of a RawRepresentable type, "int" in intValue is to disambiguate from stringValue, etc. Here, there is no other value to disambiguate from.
content or contents sound natural to me since an attachment is a file-like thing, and we regularly talk about the contents of a file. I'm not sure if either is a term of art really, but it is used in API (contentsOf etc).
Interestingly, this concept of "thing that you put in a container that can contain a single containable thing" seems hard to name. CoreTransferable uses the term Item. Also not a very good name, but at least it’s short:
struct DataRepresentation<Item> where Item : Transferable struct FileRepresentation<Item> where Item : Transferable
Sure—I'm just explaining my thought process here. If the community prefers value, then we can of course rename the property.
In that context, the word is used to refer to the contents of a file (i.e. the bytes within the file.) That isn't the same as what we're trying to describe here.
If the primary concern here is the length of the name, then it's worth considering that in practice test authors probably won't actually type it all that often. You don't need to ever type it when declaring conformance to Attachable or creating and attaching an instance of Attachment:
(As an aside, we did consider a global/free attach() function here, but there are potentially a number of places where you would want to attach an attachment in the future, so we didn't want to camp on that name just yet.)
I get why, internally, the Attachment type needs to be a thing. However, this feels a little awkward to actually use. I can imagine some amount of confusion by developers not realizing that they have to call attach() in order to actually record the attachment in the test.
I think that a separate attach(...) (or createAttachment(...)?) function which essentially wraps Attachment(...).attach() would be a good idea.
Additionally, if we decide to add this function, then I don't think that the Attachment type needs to be exposed publicly, as the attach function would essentially duplicate the public functionality.
Some disclaimer: I haven't used the attachments API in XCTest much, so I might be missing some use cases here.
This is a reasonable thing to ask about, definitely. We did consider writing the function as a global/free function, but there are a number of caveats:
Free functions are harder to discover using autocomplete in Xcode or when reading documentation because they don't show up relative to any symbols you may have already typed.
Free functions exist in a (mostly) flat namespace and you may end up seeing several different, unrelated attach() functions that take unrelated arguments.
Although this initial proposal does not include any additional configuration for attachments, we are exploring adding things in the future such as attachment lifetime, sidecar data/metadata, arbitrary "user info", etc. If we add them and our interface is a free function, we need to add overloads that take this additional state as arguments, which quickly degenerates into:
We anticipate being able to attach attachments to multiple "things" in the future, not just the current test context. With XCTest, you can attach an attachment to an activity, so we would want something like attachment.attach(to: activity). We could write that as attach(attachableValue, to: activity) but see above for the downsides.
I will add a summary of this explanation to the "alternatives considered" section as I realize I neglected to do so.
Thank you for that extra context! Definitely given the context of additional configuration in the future, then keeping these as separate calls makes sense to better support generic usage.
I still think that a lot of teams will create shorthand on their own for this, but certainly able to create whatever shorthand they want.
I agree with @younata that Attachment(...).attach() feels awkward, mainly because its unclear what, specifically, you're attaching the attachment to. You must infer that it will be attached to the currently running test.
I also share your opinion about free functions being hard to discover.
Personally, I'd like to see a static method on the Test struct:
We could potentially offer two overloads of Test.attach(), one that takes the attachable value directly (and acts as shorthand) and one that takes an Attachment for more complex use cases. Would that make sense?
I'd like to offer a counterpoint: As the proposal currently stands, with attach() being an instance method on Attachment, the implementation can attach the value to whatever it determines is the "most relevant" entity. Right now, the only thing you can attach a value to is the currently Test instance. But as @grynspan mentioned (emphasis mine):
If we structure this API such that users need to call Test.attach(attachment), that would presumably mean that attachment would always be associated with the currently-running Test, even if the code which generated the attachment is running in a narrower sub-scope within the test, such as an activity.
Note: The concept of an "activity" is borrowed here from XCTest. Although no one has begun working on an analogous feature in Swift Testing, I think it's very likely something we'll pursue before too long because subdividing a larger test into smaller units of organization for reporting purposes can be very useful. My argument here mostly assumes that such a feature will come along in the future, and attempts to accommodate it in the current discussion about attachments.
In a testing system that offers organizational features such as activities, it's a common workflow for a test author to start out simple without any activities, then later, as the test becomes more complex, carve up the test into more manageable pieces by enclosing sub-scopes into activities. In general, it's helpful for result data from each scope to be presented to the test author in a way that is associated to its "nearest" activity. Concretely, to me that means that an attachment should always prefer to be associated with the "nearest" activity in its scope, if there is one, and fall back the current Test as a last resort. (Alternatively, it may help to think of a test as the "outermost" activity.)
Thinking about how this all applies to the proposed API: I worry having Test.attach(attachment) will make it harder for users to incrementally adopt activities if/when that feature is introduced. If you call Test.attach(), but later enclose that into an activity scope, it would (wrongly, IMO) continue to associate the attachment with the outermost Test scope instead. This would likely be an oversight, but the API would make that kind of mistake too easy. And activities may be performed in helper/utility functions, separated from the @Test function, making such errors harder to spot.
If attach() is an instance member on Attachment I think it could much more easily be modified in the future to always use the "nearest" activity, falling back to the Test last.
Separate from that point, I agree with @younata and @plemarquand that a shorthand or convenience form of the API would be very helpful in addition to the currently-proposed API. @grynspan , would it be possible for this to be a static method not on Test, but on Attachment? E.g.
Attachment.attach(value)
To me, placing it on the Attachment type instead of Test would avoid the problems I described above about assuming the attachment will be associated with a test even when in an activity scope. Would it be possible to offer a shorthand API with this shape given the generic typing of Attachment?
I think my main sticking point is that to me an Attachment is "inanimate". It is just a bag of data. Being inanimate it can't do anything. This is why to me Attachment("some data").attach() or Attachment.attach(value) reads awkwardly, since it reads as if the attachment is doing the work. It also doesn't make clear what the attachment is doing the work to.
The Test, however, is the thing doing the work. Just like a person going page by page verifying some paperwork, the test is the actor going through line by line and verifying functionality, and when it encounters an attachment the test is the thing that attaches it to the results. Tests can act, attachments cannot.
As for activities, I envisioned activity attachments looking like Test.attach(attachment, to: activity) or maybe Test.attach(attachment, with: activity).