Improving Crypto on All Platforms

Server Development
1

To be clear - this post is not supposed to be a criticism of Swift Crypto but more a call for discussion and some constructive feedback.

When Swift Crypto was introduced the crypto story on non-Apple platforms was painful. You either relied on the system providing OpenSSL (and the correct version of OpenSSL without it updating from underneath you) and knowing the special incantation of compiler flags to make linking work, or you built your own copy of LibreSSL and friends and took the compile time hit. The issue was that with no central recommended option, you ended up with a combination of the two and the terrible compile times to show for it.

The situation these days with Swift Crypto now is much nicer. Most libraries in the ecosystem now use it and with the additions of Crypto Extras, most libraries can switch over to one crypto library and reap the benefits.

However, it now very much feels like Swift Crypto is both constrained by being a client library first and foremost and frustratingly limited by the development speed of Apple platforms, which is an outlier in the server world. Simple things, like Sendability annotations, don't exist (an issue was raised over a year and a half ago(!) and still no movement on it for something that on the Linux side is as simple as marking types with the protocol) whilst APIs that are required by Apple services, such as Sign in with Apple and PassKit, can't be added because it doesn't make sense to add to the client library on Apple platforms.

There's also an awkward distinction where Apple platforms have an OS optimised implementation but other platforms have their own implementation, furthering the perception that Swift on Apple platforms is not equal to Swift on other platforms. Swift is a cross-platform language that is a successor to other systems languages and it should provide a complete crypto library that is the same on all platforms.

Other issues like discoverability of Crypto Extras APIs, documentation of the whole package and needing to know implementation details of CryptoKit are all small annoyances that add up to annoying paper cuts. Just this week I was working with a client that was trying to implement signing on Cloudfront who had spent a while working with the Security framework that provided some of the unnecessary APIs and got stuck simply because they didn't know RSA operations existed CryptoExtras.

So I would like a propose a few stages for discussion:

  1. Rename _CryptoExtras to CryptoExtras - the underscore technically means the API is unstable and subject to change. It would be nice to set a line in the sand and fully support the new APIs that are being used.
  2. Reduce the barrier for getting new APIs added to CryptoExtras - historically adding new APIs even to CrypoExtras has had quite a high barrier which causes problems for people needing APIs that exist in BoringSSL but aren't exposed because it provides no versioning guarantees.
  3. Integrate CryptoExtras into Crypto itself - the split of the library is confusing and is not clear from the outside if you don't have experience of iOS which API should live where. This can be solved with improved documentation but again, is another paper cut that would be nice not to deal with.
  4. Break away from CryptoKit - whilst it has served the ecosystem well and the intention is well understood, it's causing more and more issues. It would be great to live in a world where no RSA APIs are offered because it's insecure and shouldn't be used anymore, the reality is that many services that need to be used in the server world (looking at you SIWA) still use it, so we need APIs for it. Breaking away from the constraints of CryptoKit would allow for better development times, improved APIs and a nicer ecosystem. Anyone wanting to just build for Apple platforms can use CryptoKit, anyone wanting to build cross-platform libraries and apps can use Swift Crypto, which should become the defacto library for stuff under Swiftlang (i.e. not for Apple platforms). This also might provide some incentives for reimplementing parts of the API in Swift natively rather than needing to bridge through to a C library.

It would be great to see points 1 and 2 addressed in the near future, I'm not expecting quick changes for 3 and 4 but hopefully others can chime in with their thoughts, points of views and nuances I've invariably overlooked.

59 Likes
2

i have a similar experience with swift-crypto and can echo many of your frustrations.

i feel that, for these discussions to be effective, we need to start including the funding aspect in these conversations. we spend a lot of time pondering gaps in the ecosystem, and i sense that there is a lot of agreement already about what is missing and what needs to improve.

what we are constrained by is labor — pretty much the entire server ecosystem today save a few “core”ish libraries are maintained by unpaid volunteers who chip in a few hours a week to unstick the gears and keep the lights on. a lot of these folks would love to be able to work part time or even full time on these projects, if they were given the resources to do so.

i will open with a somewhat strawman-y suggestion: Apple should start providing grants for library maintainers to work part-time on their libraries. of course, Apple may or may not be interested in this – i welcome suggestions for alternative Sources of Funding. but it’s not effective to talk about addressing these gaps in the near future without also talking about the single most important factor in project success, which is money.

11 Likes
3

A big +1 for me

I think all of these points are strongly tied together and the main point here is really number 4. In any server side ecosystem a crypto library is essential and with the SSS world growing as fast as it is, it seems like a major drawback to have to rely on an API which wasn't built and isn't maintained for the server side. Right now swift-crypto is the obvious choice for any kind of server-side crypto stuff and it's not great to see people have to rely on other libraries (or even ecosystems) just because the API has to line up with something which server-side developers don't care about. And we also don't want users just rolling their own cryptographic functions for obvious reasons.
Just mashing all of the non-compatible stuff into a separate "extras" module is also not the solution in the long run as it will always feel as a "side project" in my opinion. Extensibility of the package is the key here, and not only in terms of (judiciously) adding unsafe and compatibility-aimed functions and algorithms, but also taking a look at the future with something like quantum cryptography.
This all may sound overly critical but my intention is constructive: what I'm trying to say is that we do need a crypto library which doesn't limit us; whether this is supposed to be swift-crypto or not can be up for debate. I for one would like for it to be swift-crypto because FWIW it has always served me well, it's just missing that extra extensibility and polishing which would make it ideal.

Another point we could make is about moving away from BoringSSL on Linux and using new language features like non-copyability and what not to have a fast and native crypto library we can be proud about, but obviously that's another kettle of fish

2 Likes
4

Although the use of Swift on macOS, Linux and Windows platforms (and not forgetting the embedded one) has improved a lot, there is still a long way to go in terms of the availability of common libraries for fundamental things and frequently we have to resort to mixes of different third-party libraries.

Recently I started a Swift client/server project with my team on macOS, Windows and Linux to develop an application licensing solution requiring a Cryto library.

Starting with the use of the Security Framework on maOS, problems with porting to Linux and the use of the Crypto / Extras library etc. quickly appeared, followed by a third blockage to support Windows. We are now experimenting and considering adding the third-party CryptoSwift library GitHub - krzyzanowskim/CryptoSwift: CryptoSwift is a growing collection of standard and secure cryptographic algorithms implemented in Swift to the equation.

It seems to me that with all the experience of Apple programmers on Swift we should have a 100% Swift Crypto library that is efficient, secure and usable on all macOS, Linux, Windows platforms…

We are also faced with the same problem for network programming. Even basic Socket programming supporting macOS, Linux and Windows is a headache. After the sixth iteration of Swift shouldn't we have a solution like a thin layer Swift Socket library available on all platforms?

I love Swift and it remains my first choice for programming on macOS, Linux and Windows and I try to promote it as a multi-platform language in my organization but I really want to see the emergence of true multi-platform solutions including Network and yes Crypto programming.

2 Likes
5

Out of interest - what was missing from Swift Crypto that required the use of a different library? I've built a few licensing systems in Swift and the only thing that's caused a block is DSA support for working with an old 3rd party.

FWIW GitHub - vapor/jwt-kit: 🔑 JSON Web Token (JWT) signing and verification (HMAC, ECDSA, EdDSA, RSA, PSS) with support for JWS and JWK uses Swift Crypto as the only library for cryptography requirements, supports every JWT algorithm apart from ES256K (there are very few libraries on JSON Web Token Libraries - jwt.io that have more algorithm support than we do in any language) and works fine with Windows (we have CI that now runs our test suite against Windows).

6

Thanks so much Tim for directing me to JWTKit.

One of the headaches of using Swift Crypto is its dependence on BoringSSL on Linux and Windows. In this sense, your point #4 which mentions getting rid of the BoringSSL C library seems to me a priority. We should not depend on C libraries and have to use CMAKE. Add to that using these tools on Windows and it becomes very painful.

I didn't know about JWTKit and this level on top of Swift Crypto / _CryptoExtras to support JWT is very interesting. JWT support on top of a 100% Swift Crypto library would be wonderful.

7

To clarify - Swift Crypto is a Swift Package and can be integrated with Swift PM like any other dependency, you do not need to use cmake. You can use cmake if you wish to depend on Swift Crypto in scenarios that SwiftPM currently doesn't support, but it is not a requirement. There are absolutely no issues with the dependency on BoringSSL on Linux, and I believe the Windows issues have now been solved.

The fact that Swift Crypto uses BoringSSL is (and should be) an implementation detail that any library consuming it really doesn't need to care about. The benefits of writing the underlying Crypto code in Swift is removal of C code that makes it easier to maintain and contribute to as Swift developers and potentially more performant if there's no need to switch between Swift and C types.

Overall, my point is that as a language we should provide a Crypto library that works for all use-cases on all platforms, without the shackles of having to maintain parity with a Darwin based library.

That statement goes for all libraries generally, but especially Crypto where it's easy to make mistakes and the consequences potentially extremely serious when it does go wrong. Having a solid library maintained by a team that know what they're doing is very important.

2 Likes
8

I'm mostly in agreement with this post.

First of all, I agree that _CryptoExtras needs to go in favour of a non-underscored variant. We already must rely on this for essential bits of the ecosystem.

I understand the desire to have compatibility with Apple's *OS releases, which do not need to vendor BoringSSL which will incur an impact on binary size. For the sake of cross-platform code - I stand in the middle. I understand the benefits of either side, and think both use cases must be considered.

Many of my (proprietary) libraries I make for clients depend on this compatibility. Losing it is extremely hard to justify, and would likely result in needing to fork swift-crypto. I especially applaud @Datagram 's mention of embedded support - which will definitely be on my radar soone rather than later. On the other end, my Server libraries (especially Citadel) are struggling without these APIs.

I'm not a fan of duplicating crypto efforts at all, so a hard disagree on point 4 here. I do think that a Swift-Crypto implementation for all platforms as part of Swiftlang is potentially an acceptable option.

I do think that Swift-Crypto lacks a lot of algorithms and APIs that make it unnecessarily hard and to a degree extra insecure to add compatibility with older systems - something that cannot always be changed.

9

Thanks for this feedback Tim: I appreciate you taking the time to put your thoughts together like this, and I'll continue to think on this feedback over the coming weeks.

In the short term, some thoughts:

This seems like a reasonable change. I'd merge a PR that made this adjustment.

I think this is a much more interesting topic. For some background reading, I'd like to suggest Alex Gaynor's Philosophies of Cryptographic Libraries blog post which does a really good job of outlining some of the tradeoffs here.

Ultimately, there is a question of what CryptoExtras is. The viewpoint of the Crypto team thus far has been that CryptoExtras is bridging from the existing Crypto API shape over to "All Secure Algorithms". This means that the API barrier is principally about the definition of "secure algorithms".

With recent PRs such as the ones for PBKDF2, we've tried to lower the bar to inclusion to be that we should never offer an insecure algorithm in a space without also having a secure one.

It would be very valuable to get feedback from folks as to whether they feel that this bar is still too high.

My immediate instinct here is that this feedback is tightly coupled with point (4), so we should address them together.

How do you forsee this new library interacting with CryptoKit? Are there duplicative types? Is there a translation layer?

On this point in particular, for some context, many cryptographic primitives cannot be safely implemented without assembly code, and most cannot be performantly implemented without them. A first-class cryptography library will always have assembly code in it, at the very least.

The real question is whether anyone in the community feels they are qualified to maintain that assembly code. I'll happily go out on a limb here and say that I don't. This is the main thing we're getting from BoringSSL: an implementation that can be relied upon to have mitigated a wide variety of performance and security issues.

Thanks Paul. Can you elaborate on this? What extensibility would you add, if you could?

7 Likes
10

Interesting, this means a pure Swift Crypto library can't reach this goal ?

I watched recently with interest many videos and articles presenting Swift as the best successor to not only objective-c but also C++ and C. Is this realistic ?
In the case of Crypto, if assembly code is a requirement and means C is a requirement so envisioning Swift as a C replacement is questioning too.

Maybe Swift needs to be designed to support some level of assembly code for things like Crypto or Kernel programming ? Is there any work underway in this direction ?

If we must continue to depend on C for the foreseeable future my hope is that the effort will be made to use this mix easily on all platforms including Windows .

11

I think these algorithms likely wouldn’t be implemented in C, either. Any higher-level language is too high level - you want specific instructions, so you hand-write it in assembly.

Even a lot of C standard library functions (like memcmp) will have hand-written assembly implementations (for performance rather than security in that case).

1 Like
12

I will add that the recent additions have been much easier and there seems to be more momentum behind PRs like PBKDF2 - this is noticed and appreciated!

There are a number of ways of doing this, all obviously involving a major version release. My gut reaction is that existing API stays and it knows nothing of CryptoKit. There will be lots of duplicated types and APIs that will evolve separately on Swift Crypto so they may not line up going forward.

This was mostly a throwaway comment that seems to have distracted from the main point of point 4, which is to allow Swift Crypto to evolve separately from CryptoKit. The underlying implementation doesn't matter to the outside and a decision from the Swift Crypto team. (And whilst I am fairly comfortable in assembly in some architectures, I would not fell qualified to maintain the crypto parts!)

13

More rather a performant (and safe) library wouldn't be pure Swift. But that doesn't mean different parts couldn't be rewritten in Swift, with a very small C layer to interact with assembly. This would be much better than the majority written in C. But again, as mentioned earlier, as a user of Swift Crypto, I don't care how it's implemented. I care that it works on all the platforms I want it to and provides the API I want it to, in a safe and performant way.

1 Like
14

@Karl is on the money here. This is not a matter of Swift versus C. Neither can do (nor want to do) what assembly has to offer.

Contrary to what Tim said, Crypto code can be performant and in Swift. But some types of security that are handled by the assembly code cannot be reproduced by these languages. Performance definitely isn't everything in security, because a simple difference in timing might give away essential information about how far you've gotten into a process.

While cryptographic code written in Swift can be safe at a networking level or at rest (on a disk), there are different attacks that can be performed on applications by analysing a device more closely.

Many of these attacks require remote access or close proximity to a device though. But remember; when you're running on shared hosting like Amazon Lambda or EC2, attackers already have physical access to the same device! So it's not far fetched, and attackers can and will abuse hooks you give them. And even air gapped computers can be attacked.

4 Likes
15

How are Java crypto libs doing this, do they always rely on JNI code?

1 Like
16

With Swift as it exists today, yes.

Assembly is a requirement: C is not. Swift can call directly into assembly if needed.

The value of BoringSSL is not that it provides the C. It's that it provides the assembly.

My understanding is that, modulo some bugs in the most recent release, Swift Crypto runs fine on Windows. It's been a supporting part of the Swift Package Manager for a while. Certainly, this is intended to be supported.

The answer is "it depends". Java has the crypto provider abstraction which allows pluggable backends to the same high-level interfaces. The overwhelming majority of these are assembly accelerated in whole or in part, but they don't have to be.

The nature of the acceleration also varies. Some libraries rely only on access to intrinsics, with the implementations in C. Others use assembly. And a few use pure-Java, with no acceleration.

Generally the unaccelerated implementations have higher risk of timing side-channels and worse performance, but those can sometimes be worth it.

5 Likes
17

I guess what I meant is about the same as Tim's "lowering the bar for additions". Maybe the term extensibility isn't ideal, but it wasn't meant to be read as "let's generify an elliptic curve so that people can simply add and use their own" (which would be bad (and quite difficult too I guess)). It was more in the direction of the package being free-er in maintaining whatever is needed, in respect to some kind of philosophy (see second paragraph), and without needing a separate module. Something like the ML-DSA and SPHINCS+ algorithms do not deserve to be in an "extras" package and we should be able to proudly present them as important components in the main module, as there's probably just a handful of crypto libraries providing them (as of right now. This is just an example). To me it does seem a bit weird to have RSA in an "extras" module too: while it's not recommended anymore for new applications, it's certainly still used across a whole lot of different use cases. Along with other functions it might make sense to move this to a separate module for unsafe things, to which functions can be migrated to when deprecated etc.

I think the article you sent is spot on for this discussion as it represents our issue well: what is swift-crypto's philosophy? Why will some crypto-function requests be accepted more than others? What target is swift-crypto aiming for in terms of compatibility, e.g with different platforms? I think being more explicit about that might help, especially as we (possibly) drift away from CryptoKit into unknown territory, and there won't be CryptoExtras anymore to which we can add stuff somewhat blindly.

1 Like
18

Out of interest, what is the reason for depending on the compatibility? Is it binary size? Which is totally understandable.

CryptoKit isn't available in embedded Swift correct? So you'll need Swift Crypto anyway. Making sure it works on embedded devices and on Windows should be a definite goal (including adding CI which I know is being worked on)

19

Source compatibility mainly. I want to be able to vendor the same library and run it on various different platforms.

20

That would work with Swift Crypto though right? If your library depends on Swift Crypto and it works on all platforms then there's no issue? (Aside from apple platform optimisations, binary size etc)