[Pitch] Image attachments in Swift Testing

Evolution Pitches
,
1

In Swift 6.2, we introduced attachments to Swift Testing, primarily dealing with untyped or loosely-typed values such as [UInt8] or String. I propose adding support for attaching images (namely, instances of Apple's CGImage, CIImage, NSImage, and UIImage types) and having them be automatically serialized to an appropriate image format such as PNG or JPEG:

import Testing
import UIKit
import SwiftUI

@MainActor @Test func `attaching a SwiftUI view as an image`() throws {
  let myView: some View = ...
  let image = try #require(ImageRenderer(content: myView).uiImage)
  Attachment.record(image, named: "my view", as: .png)
}

Read the full proposal here.

Trying it out

To try this feature out, add a dependency to the main branch of swift-testing to your project:

...
dependencies: [
  ...
  .package(
    url: "https://github.com/swiftlang/swift-testing.git",
    branch: "main"
  ),
]

Then, add two target dependencies to your test target:

.testTarget(
  ...
  dependencies: [
    ...
    .product(name: "Testing", package: "swift-testing"),
    .product(name: "_Testing_ExperimentalImageAttachments", package: "swift-testing"),
  ]

Add the following imports to any source file that will attach an image to a test:

import X
@_spi(Experimental) import _Testing_X

Where X is the module name of the image type you're using:

Image Type Module Name
CGImage CoreGraphics
CIImage CoreImage
NSImage AppKit
UIImage UIKit

Note: This extra product dependency and extra import statement are a side effect of how packages are built. They will not be required in the final implementation of this feature.

13 Likes
2

This looks really nice! Do you plan to support other image encoders, such as JPEG XL, PNG 3.0, or AVIF? Or allow custom image encoders ?

3

This is really great to see. Thanks for doing this!

Another future direction I can see is attaching video.

Second this, though I can also see an argument that "other image encoders fall under the underlying platform support for exporting images".

1 Like
4

This looks really nice! Do you plan to support other image encoders, such as JPEG XL, PNG 3.0, or AVIF? Or allow custom image encoders ?

Support for image encodings falls to ImageIO on Apple's platforms. If ImageIO does not support a given format, it's not really practical for us to include custom support for it in Swift Testing (see e.g. the example of including ImageMagick discussed in the proposal.)

You are, of course, able to write your own Attachable and AttachableWrapper types to provide support for additional data formats if you need them. For example (pseudocode):

struct PNG3Image<Image>: AttachableWrapper where Image: AttachableAsCGImage {
  var wrappedValue: Image

  init(_ image: Image) {
    wrappedValue = image
  }

  func withUnsafeBytes<R>(for attachment: borrowing Attachment<Self>, _ body: (UnsafeRawBufferPointer) throws -> R) throws -> R {
    let data = PNG3CreateDataFromCGImageSomehow(wrappedValue)
    return try data.withUnsafeBytes(body)
  }
}

Attachment.record(PNG3Image(someImage))
5 Likes
5

Really useful feature!

My question is: does it need to be part of Swift Testing? As it is so tied to Apple platforms, would it also be possible to add this as a separate package? At least the concrete implementations - the protocol itself should be generic enough to be part of Swift Testing.

My assumption here is that most users would expect that Swift Testing features work on all platforms.

2 Likes
6

Overall big +1 from me!

Designing a platform-agnostic solution

I wonder if adding support for Data and a default set of identically named image type names (under a pseudo-UTType-based struct) would be enough to support cross-platform projects that already know how to assemble image data without guarding these calls behind availability and #if checks? Better, AttachableAsCGImage can just conform to AttachableAsImage with the sole requirement of providing a data representation to save to disk.

7

I could possibly update swift-image-formats to provide swift-testing integration (behind a package trait or in a separate module). That’d allow image attachments to be encoded as PNG, JPG or webp on basically all Swift platforms.

To my knowledge, only SwiftCrossUI uses swift-image-formats, so in order to make things maximally useful I’d have to add conversions for the various Apple image data types and a few other common image data types from the open source Swift ecosystem.

Once SwiftCrossUI supports snapshotting views on every platform, such an integration would become particularly useful.

2 Likes
8

Wait, what? When did this syntax become supported?! :heart_eyes:

9

Windows and Linux/FreeBSD are both listed as future directions in the proposal. In particular, I hope to follow up with a Windows-specific proposal later this year.

A UTType-like interface is something we considered and eventually rejected for the overall attachments feature. The UniformTypeIdentifiers framework[1] and underlying database are a lot more complicated than they appear to be at first glance, and Swift Testing is not the right place to try to reimplement it.

The short answer here is that providing a more abstract protocol wouldn't solve the problem of cross-platform support.

The longer answer: the value you start with (that is, the instance of CGImage or HBITMAP or QImage or whatever) is going to be platform-specific anyway. There is no platform-agnostic "image" type in C or Swift other than a pre-serialized bag of bytes, and if you have one of those you can just pass it as-is to Attachment.record() without any more ceremony.

So back to "support for Windows and Linux/FreeBSD are future directions"! If you have a platform-specific image type, you are also dealing with a platform-specific API for serializing it:

  • The Darwin implementation is more or less what we already have and uses ImageIO.
  • The Windows implementation probably uses GDI+ and I'd expect to have a AttachableAsGDIPlusImage protocol (naming is hard) to which types like HBITMAP, HICON, etc. conform.
  • Linux/FreeBSD are more problematic because they don't ship with such an API by default, but there are a few different libraries that might be installed that we could pick from such as Qt. That the libraries aren't guaranteed to be present poses layering problems for the Swift toolchain that we would need to resolve.
  • Android has android.graphics.Bitmap and AndroidBitmap_compress() which are accessible via the NDK.

What we'd want to see is an interface that is call-site-compatible with the one we're proposing here so that you could ultimately write something like this in your test:

#if os(macOS) || ...
import CoreGraphics
func makeImage(of stuff: [Stuff]) -> CGImage { ... }
#elseif os(Windows)
import WinSDK
func makeImage(of stuff: [Stuff]) -> HBITMAP { ... }
#elseif canImport(Qt)
import Qt
func makeImage(of stuff: [Stuff]) -> QImage { ... }
#else
...
#endif

let image = makeImage(of: [.lions, .tigers, .bears, .ohMy])
Attachment.record(image, named: "great-and-powerful-oz.jpg")

And have it "just work".

This is a very nice package you've put together! I'd be happy to help you provide Attachable support for your clients if that'd be useful to you. :slight_smile: Feel free to DM me.

SwiftUI.ImageRenderer is available as of macOS 13 and iOS 16, so they've been there for a few years now.

  1. I maintained this API for several years as an Apple employee (I even did a WWDC tech talk about it! There's a kitty!) so this is one of those rare occasions where I can wear my "domain expert" hat unironically. :grimacing: ↩︎

2 Likes
10

I mean, when did writing spaces in the function name become valid (surrounded by ticks)

11

Since SE-0451 (Swift 6.2)!

4 Likes
12

+1 from me. Looking forward to adopting this for snapshot testing results :+1:

13

Happy Monday, all! I've done some fiddling over the weekend (:violin:?) and have a PR open to introduce a platform-non-specific AttachableImageFormat type to represent the image formats you can use to encode an image. This means this type can be shared between macOS, Windows, etc. with minimal platform-specific API.

I spent some time trying to lower _AttachableImageWrapper but unfortunately it's not possible to do so without introducing a Core Graphics dependency in Swift Testing's main module (which would be a major layering violation). Swift's type system doesn't let us say something like:

extension Attachment {
  public init<T>(
    _ attachableValue: T,
    named preferredName: String? = nil,
    as imageFormat: AttachableImageFormat? = nil,
    sourceLocation: SourceLocation = #_sourceLocation
  ) where AttachableValue == _AttachableImageWrapper<T>
}

Without _AttachableImageWrapper<T> providing full Attachable conformance in the same module, even if the conformance is provided in a (platform-specific) module in the same package. This is something that's worth exploring on the compiler side, but shouldn't hold up this proposal. (All that said, the vast majority of the code in _AttachableImageWrapper is platform-specific anyway, so we wouldn't save all that much by lowering it.)

I'll update the proposal to reflect this change if/when my PR is merged, and I'll keep an eye out for additional API we can make platform-agnostic.

3 Likes
14

Lest y'all think I was kidding about other platform support: [WIP, DNM] Windows image attachments by grynspan · Pull Request #1245 · swiftlang/swift-testing · GitHub

4 Likes