Swift testing docs include unreleased APIs with no indication

Ran into a pretty frustrating docs bug today. Consider these three pages:

The information presented about inspecting a thrown error is incorrect. The first two pages refer to a version of the macro that will be released with Swift 6.1, and the third page states that the API that currently works is deprecated. None of the pages make mention of Swift 6.1 or give any guidance to the user to understand that the API being described is unreleased.

I've encountered similar problems dealing with missing Foundation APIs on Linux.

It seems to me that is a threat to the adoption of Swift by new developers and in new contexts that the documentation situation fails us so badly. Is there anything being done? Are there any plans for the open source project to host its own docs so that we aren't constrained by inaction from Apple?

I would love to hear that I've failed in some way to use these resources correctly, and there is a solution to this problem that already exists.

4 Likes

Swift Testing’s documentation is available on Swiftinit, with separate documentation trees for each tagged release of Swift Testing.

1 Like

Thanks for replying.

Removing the remainder of my comment because I think I put my foot in my mouth a bit.

i’ve certainly given my share of Unvarnished Feedback, so i guess the old adage about Giving It Out and Taking It applies to me as much as it does to anyone.

to answer your previous question, no Swiftinit is not “officially” owned by the Swift Project, although its operator (me) also happens to be part of several Swift Project work groups, so it’s in that gray area of Not Apple but Not Totally Random either.

regarding the visual appearance of the site, i’m open to hearing the reasoning why you are dissatisfied with the current styling — that’s a lot more actionable than a generic I Don’t Like It.

It is really helpful knowing it is maintained by such an active member of the community. You might consider giving yourself credit somewhere on the homepage. This is really a very valuable resource, especially knowing that someone who cares is involved.

I think I initially associated it with sites I’ve seen in other communities that just reproduce the official docs, but with ads. My preconceptions made me overlook the very consequential differences between those and your project, including the very obvious absence of ads. I’m genuinely sorry I didn’t pay more attention.

As for the design complaint, I’ve been looking more carefully and I think light mode is great. For some reason, I find dark mode harder to read on desktop, but I’m not having the same problem on mobile. My laptop is always in dark mode, so I’ve never seen the site in light mode before this. I’ll play around with the CSS this weekend and see if I can give you something a bit more actionable.

Anyway, thank you for your work on this, and thank you for taking my regrettable comment with such grace. I really appreciate both, and I’ll try to lead with the same grace in my future comments.

3 Likes

at some point, Swiftinit will need to run at least some ads, as the site and its infrastructure does cost a modest amount of upkeep, although my hope is that it can be limited to the homepage and a handful of other unobtrusive locations. we archive documentation for every version of every package in the Documentation Index (this is how Swiftinit’s API redirects work), and this gets a little more expensive to maintain every time someone tags a new package release.

in the beginning, Swiftinit was essentially set up as a honeypot/telescope for malicious bots and crawlers, and providing free cross-package documentation was a way of “giving back” to the community while collecting data about these threats and actors. but the value of the security research i derive from Swiftinit has diminished over time (for a variety of reasons), so i mostly just keep it online for the benefit of the small group of users who actually rely on its documentation and rendering capabilities. (for example, https://swiftonserver.com/)

we’re aware the homepage needs a lot of improvements, right now the markup is entirely hard-coded in swift-unidoc at the rendering engine level and i haven’t had time to implement anything more flexible. one analogy i’ve been mulling lately is that Swiftinit is kind of like an ASIC (heavily optimized to efficiently support one type of database access pattern that would otherwise be cost-prohibitive to run), but to really succeed on the internet, it needs to become more like an FPGA.

Apple's online documentation tracks the latest Xcode release (including content introduced the current Xcode 16.3 Beta.) The documentation included in Xcode will match the version of Xcode you have installed, so if you have Xcode 16.2 locally, the documentation available within Xcode will not include these changes.

I can appreciate this may be confusing. I would encourage you to file feedback with Apple here.

1 Like

@grynspan Showing beta docs on the web by default, with no other option is a pretty wild decision, but if that debate hasn't happened within Apple by now, I'm not sure my feedback is going to make a difference.

I think for my server-side work, I just need to learn to avoid Apple's docs, which is not meant to disparage the work you and your colleagues are doing in any way, I'm just not your intended audience when I'm working outside of Apple platforms.

2 Likes

As far as I'm aware, this is not intentional. Apple's documentation is mostly geared toward frameworks that ship with the OS, but Swift Testing is part of the Swift toolchain and ships (in this scenario) with Xcode, not the OS. Since API in Swift Testing doesn't list OS availability (because, in general, it's bound to your Xcode version rather than your OS version), I think the documentation system just doesn't know to present it with the "beta" slug like it would with new OS API.

I have let our colleagues in Developer Publications at Apple and on the DocC team know about the issue either way.

2 Likes

Ah. That's good to hear. I'm sorry if I misunderstood your first response.

This has come up in the website workgroup a few times and will probably fall underneath the new Information Architecture project. But it's definitely a goal of ours to host the open source documentation for things like foundation and other stuff bundled into the toolchain on swift.org

2 Likes

Hi, thank you for reporting this. @grynspan is correct, the version of the swift-testing docs published in the Apple documentation doesn't have the information it needs to report that these new APIs are "betas", because of the distinction between how the open source project is released and how Apple tracks API changes in Xcode.

I've filed a bug report on this internally (rdar://145474319), please do provide feedback with your experience that we can use to help solve this problem.

1 Like

FYI I've opened a PR to update the documentation for these macros to explain when the return value was added. The lack of a "Beta" tag is a technical constraint on Apple's end, but hopefully this will help make the behavioural change clearer. Note in documentation when `#expect(throws:)` started returning a value. by grynspan · Pull Request #980 · swiftlang/swift-testing · GitHub

2 Likes