Hello here !
I'm quite new to Swift, and want to build a new B2C application for my company.
One of the requirements we have is to be able to do some end to end testing, as this is someone very common here with standard web projects.
Could you point me to any way of doing this ? I've done a bit of research on my own but did not find anything really relevant ?
Or tell me the Swift way to achieve "non regression testing" ?
Thanks !
Hello @Durnan and welcome to the forums!
It's not obvious from your post, but I assume you're talking about an app for iOS or another Apple platform and not a web-app build as a server-side Swift project, yes?
In that case I'd say the most common way to go is with UI tests based on Apple's XCTest framework, see here (official docs) or, for example, here.
From experience I can say, however, that you need to invest time in thinking about how any potential backends are involved in this and where the tests will actually run (on bare metal you own that has access to your company's intranet, a provider like Azure DevOps or CircleCI). That usually means your app has to be designed to allow staging these backends so that your tests work with the correct stage (unless you want to run UI tests against PROD...).
There's a couple of pointers I could give, but before I do that I'd like to know if I was right in assuming you're talking about a "regular" iOS or macOS app.
Lastly, another experience I had in my company was the often encountered lack of understanding what automated testing actually means from management. In short: If the focus in E2E tests means less effort is spend on unit tests (worst case: you won't do them) you have a whole other problem... 
While E2E is important I personally think a proper focus on unit tests first gives you more bang for the buck.
Hello Gero, yes exactly, I'm talking about an iOS app, sorry for not having been clear enough.
It has to be connected to a backend we already have, on AWS. This app would be an extension of some web app we already have.
About the CI part, we already have an Xcode cloud account and think about running them there, along with our development or pre-production environments.
And yes, of course we want to have some unit tests as well 
but I should be able to figure out this part on my own because there are many more resources available on the topic. The purpose here with our E2E tests would be to have to test the critical paths of our future app.
Thanks for your help.
Perfect, that fits the bill!
I personally (and sadly) have no experience with Xcode cloud yet, but from what I saw of it that should help you a lot in setting up your CI pipeline.
I'm pretty sure UI tests are what you want then, as they're basically equivalent with something like browser tests you can get done for web-apps by e.g. something like browserstack.
Basically they're blackbox tests that internally rely on the accessibility features iOS provides. That means running such a test (from within Xcode, or using xcodebuild in some CI pipeline) installs a regular build on the device/simulator as well as a testrunner app. That testrunner app runs the tests, which in turn start and "use" the app via the accessibility features (i.e. it "fakes user input").
Tests identify buttons, etc. on screen via their accessibility identifiers (if they don't have that explicitly it's usually the "normal" thing, like a button's title and so on) or by traversing the view hierarchy. For example, to tap a tab bar button and then ensure the scene you land on shows a specific text element you'd write this:
let app = XCUIApplication() // usually set as a property of the test case class
app.tabBars.buttons["Button title"].tap()
guard app.staticTexts["Some label text or the like"].waitForExistence(timeout: 3.0) else {
XCTFail("Did not transition to expected scene!")
return
}
You can even "record" test actions with Xcode, which autocreates the swift code for this, but in my experience that usually needs tweaking afterwards to make it more readable.
Now to the staging part which is usually super important if your app does any calls to a backend during usage that you need to cover in your tests. You'll have to do some things in your production code to help you with tests (but that doesn't mean adding test-specific branches in code!).
DEVELOPMENT_ASSET_PATHS).let app = XCUIApplication() // see above
app.launchEnvironment["YOUR_LAUNCH_ENV_KEY"] = "DEV" // example values
app.launch() // during UI tests you manually launch the app anyway, usually in thesetUpWithError method
In the (production) code of your app you can access the environment variable with ProcessInfo.processInfo.environment["YOUR_LAUNCH_ENV_KEY"]. In my current project I have that abstracted into an enum:
enum EnvironmentKey {
case normal // this is PROD
case develop
case testing
init(launchParam: String?) {
switch launchParam {
case "TESTING":
self = .testing
case "DEV":
self = .develop
default:
self = .normal
}
}
}
For my URL "repository" (in the sense that it's the central place where I get my URLs from in the app) I have then this:
struct URLRepository {
static let environment = EnvironmentKey(launchParam: ProcessInfo.processInfo.environment["YOUR_LAUNCH_ENV_KEY"])
static func plistName(for environment: EnvironmentKey = URLRepository.environment) -> String {
var prefix = String(describing: URLRepository.self) // my plist files are named like this
switch environment {
case .testing:
prefix.append("_TESTING")
case .develop:
prefix.append("_DEVSTAGE")
default:
break
}
return prefix
}
init(repoPlistName: String = URLRepository.plistName(), bundle: Bundle = Bundle.main) {
// ...
}
}
Note that all this is fully testable (with unit tests), there's no compiler directives including different code during tests builds than during release builds. I do have unit tests for all of this and never had any problem with my staging so far.
I use a similar approach when setting up my Core Data storage to use different persistent stores (i.e. files) for different stages (that way I don't mix data if I start the app for a different stage).
I think that should set you up with a general strategy to add UI tests. The tests also allow you to make screenshots, which you could then keep in your CI pipeline's artifacts for later inspection.
As a last aside (and in spite of this being so long already): If you cannot reach your backend from the CI at all (like me) you can still make use of this my mocking it out. I use Telegraph for this. It's a small webserver I run inside my testrunner app that answers with fixed responses for each request the app makes, I just have a url repo file that contains URLs to localhost for this. The only other thing needed for this is an exception entry for ATS in my app's info.plist (I only do that for debug builds), let me know if you need that, too.
Thank you very much for all your insights, it was really helpful !