[Request for Feedback] API design of "K1"

Using Swift
, , , ,
1

I've created and I maintain a Swift library called K1, offering Elliptic Curve Cryptography over the curve secp256k1 with API's that are heavily inspired by CryptoKit.

K1 vendors two fundamentally different signature schemes:

  • Schnorr signatures (more modern)
  • ECDSA signatures (older)

And for ECDSA it offers two different variants:

  • ECDSA from which user can recover the public key from the signature (with message that was signed)
  • ECDSA which does not offer public key recoverability (older and more common)

Apart from signing K1 also vendors key exchange (ECDH).

I've recently created an unmerged PR which changes the namespace / API structure from 3 top level namespaces to 4.

In main branch I have these 3 namespaces, one per feature:

  • K1.KeyAgreement
  • K1.Schnorr
  • K1.ECDSA

I want feedback on if you prefer the 3 above, or these 4 below - presented in my unmerged PR:

  • K1.KeyAgreement
  • K1.Schnorr
  • K1.ECDSAWithKeyRecovery
  • K1.ECDSA

I.e. I've flattened the two variants of ECDSA into two distinct namespaces. I put some more context and rational into the description of the PR linked to above.

What do you think makes the most sense and results in the most clear API?

The reason why all features are put inside the root level namespace K1 is to be symmetric with CryptoKit's APIs which put all structs and operations relating to a curve under a namespace for that curve, see e.g. P256, the namespace for the Elliptic Curve P256.

2 Likes
2

I think both are good. If I have to choose one, it's maybe the new approach, due to the same reason you mentioned in the commit message - I can just use the API in ECDSA namespace and don't bother to figure out what the longer namespace means (this is the first time I read about recovering public key from signature). In contrast, in the current approach I have to figure out what recoverable or nonrecoverable means to choose the API (that said, it's not difficult to understand the concept).

BTW, did you consider the following approach? (I'm not an expert on API design, I just happened to think of it).

  • K1.KeyAgreement
  • K1.Schnorr
  • K1.ECDSA
    • K1.ECDSA.KeyRecovery

It resolves the drawback you mentioned in your commit message. You can define the common structs in K1.ECDSA. This approach has one drawback in coding: one may think he defined a type X in K1.ECDSA.KeyRecovery name space but he didn't and hence misuse a type with the same name in parent namespace.

Another possible drawback is the implication that the recoverable variation is an enhancement (or based on) the non-recoverable variation. BTW, you said they are two variations. I wonder what exactly the difference is? Do they use different parameters or algorithm? I did a quick search but none of the materials I found suggested that recovering public key needs a different algorithm or parameter.

3

Yes I did, but I discarded it because of asymmetry. The two variant are suddenly in different "hierarchy depths", which is never the case for CryptoKit. But maybe it is not that bad, maybe it is the way to go, thanks for bring it up! I will consider it some more!

Good question, K1's underlying implementation is Bitcoin Core's libsecp256k1 and (non recovery) ECDSA is declared in secp256k1.h as secp256k1_ecdsa_sign, where as the recovery variant is declared in secp256k1_recovery.h as secp256k1_ecdsa_sign_recoverable and the source file of recovery is the "modules" folder. Further more they produce different structs, secp256k1_ecdsa_signature * vs secp256k1_ecdsa_recoverable_signature *.

A recoverable ECDSA signature can always be normalized to a (non recoverable) ECDSA signature.

Content wise, they only different in size, recoverable signatures contain one single additional byte, sometimes called V or RecoveryID (which can be represented as an enum with values 0, 1, 2, 3).

I think what you might be getting at...? Did you perhaps wanna propose using a single namespace for ECDSA, but with either two sign methods, or a single sign method but accepting an argument isRecoverable: Bool? I actually used the same namespace with two different sign methods earlier. But it became quite confusing because each sign method has itself 3 variants, see documentation

Thus exposing users with 6 methods instead of 3 is really less than ideal.

So let's explore the ability of using still one sign method - just 3 variants, but letting isRecoverable: Bool be an argument. Suddenly the recoveryID becomes an optional in a merge of K1.ECDSA.NonRecoverable.Signature and K1.ECDSA.Recoverable.Signature, which makes suddenly the very straightforward serialization and deserialization API's of K1.ECDSA.NonRecoverable.Signature less easy to understand and use. The reason for this is that only K1.ECDSA.NonRecoverable.Signature is well defined, with standards.

Recoverable signature have not real standardized serialization format. Content wise, as mentioned above, apart from R and S scalars, the contain a single V byte. Internally, libsecp256k1 stores the signature as leR||leS||v where leR and leS denotes inverted R, meaning little endian (at least on my M1). See documentation of libsecp256k:

The exact representation of data inside is implementation defined and not guaranteed to be portable between different platforms or versions

So to export it we use secp256k1_ecdsa_recoverable_signature_serialize_compact, which converts to big endian scalars: R || S || V. But this is a non standardized (outside of Bitcoin) format, and Ethereum uses V || R || S.

Oh I see now that I've forgotten to add documentation to K1.ECDSA.Recoverable.Signature.Compact, which is my swift variant of the output of secp256k1_ecdsa_recoverable_signature_serialize_compact.

Thus the API of the recoverable Signature is less clear than the API of the non recoverable one. which is an argument for keeping them apart. I could return a protocol but I think it less than ideal? If I know I do not care about recoverable signatures, I'd prefer it if I get an existential, with the clean API.

Hmm... what do you think?

1 Like
4

Thanks for the detailed introduction.

Yes, the latter was what I had in mind. But after reading your reply, I doubt if it's a good idea. I think it's important to isolate the two sets of API as much as possible, because:

a) It will be easy for you to maintain
b) It will be easy for user to use (it would be confusing when user import ECDSA module but see parameters/properties/types related to public key recovery which user doesn't need at all).

It would be great if it's possible to work out a single and easy-to-use API. But it would require a lot of addtional work on you side even if it's possible. Also that API would be more different than the underlying C API than your current API.

So I personaly suggest to use the approach in your PR. I think it's more like how the underlying C API is organized (according to the header files here, recovery APIs are considered a special case and isn't exposed in the secp256k1.h).

Regarding the drawback you mentioned in the PR, I can't think out a better approach than to put the shared structs in ECDSA module. I think it's fine (the C API resolves this issue by putting all common types in secp256k1.h).

5

I don't find the serialization API of non recoverable signature in your code, but I suppose you meant the two kinds of signatures have different serialization API (I see in your code that the recoverable one takes an enum because it doesn't have a standard format) and the difference makes it difficul to use a single struct to represent both signatures?

If so, below is a possible solution.

Assumptions:

  • just one sign method (with 3 variants), and letting isRecoverable: Bool be an argument
  • The name space has a hierarchy: K1.ECDSA and K1.ECDSA.KeyRecovery (note: it works to just use a single name space and put recoverable related API in it. I think both are fine.)

API Design:

  • The sign method returns a Signature struct
  • The Signature struct has APIs for common operations (e.g. verify).
  • Since most users don't use recovery featture, this is API they use.
  • For users who would like to recover public key, they need to convert the Signature struct to RecoverableSignature, as folllows.
    import K1.ECDSA
import K1.ECDSA.KeyRecovery

if let recoverableSig = sig.recoverable() {
    let key = recoverableSig.recover()
}

With this approach both kinds of signature share a common API (the Signature struct and its methods), but recovery related API is invisible to users who don't use the API (the only exception is the isRecoverable option, but I think it's fine and it can have a default false value).

EDIT: I just realized the design focuses on how to hide the recovery related APIs but doesn't deal with the different serialization API. How about this?

Assumptions:

  • Non-recoverable signature's serialization API takes no arguments (because it uses a standard format).

API Design:

  • The Signature struct has serialize api. If the signature is non-recoverable, it uses the standard format; otherwise it uses a default one (e.g. the one used by bitcoin).
  • If user want to use a different format for recoverable signature, he needs to call RecoverableSignature.serialize() instead, which takes an enum.
    import K1.ECDSA
import K1.ECDSA.KeyRecovery


if let recoverableSig = sig.recoverable() {
    let key = recoverableSig.serialize(.vrs)
}

I think this approach might be OK for this specific issue. But it feels like a workaround because it can't deal with the general version of the issue: what if both signatures have a "common" operation but takes different argument? I think the root cause is that in that case it's not really a common operation and hence shouldn't be put in Signature struct which serves as common interface. However, that would mean, to deal with a general issue like this, we'll have to define the name space hierarchy as:

  • K1.ECDSA
    • K1.ECDSA.NonRecoverable
    • K1.ECDSA.Recoverable

Hopefully you don't such cases to deal with. Otherwise I don't suggest to use the approach I proposed.

6

Right, so NonRecoverable.Signature has two computer properties: public var rawRepresentation: Data and public var derRepresentation: Data which are both kind of standardised formats, matching exactly the API of CryptoKit's P256.Signing.ECDSASignature.

And yes as you said "the difference makes it difficult to use a single struct to represent both", I don't want users to think they can use rawRepresentation for Recoverable ones, because there is no standard for it, which I why users of K1 today have to do recoverableSignatures.compact().serialize(format: .rsv) (or format: .vrs). Since the order of R, S and V is non-standardized we cannot do derRepresentation either... So a shared Signature type is tricky, because for Recoverable I do not want serialization using public var rawRepresentation: Data nor public var derRepresentation: Data to be available.

Thank you very much for brainstorming with me @rayx, I really really appreciate it! I feel now that keeping them as seperate Signature (as in main today) is the right choice.

And I'm torn behind the namespaces. K1.ECDSA.PrivateKey and K1.ECDSA.KeyRecovery.PrivateKey is not that bad, the asymmetry I mean. I will try it in a PR! And ping you once the PR is up :)

1 Like
7

@rayx I've made a PR for the other variant

vs

They are identical in Schnorr, KeyAgreement and ECDSA (non recoverable) and differ only for recoverable ECDSA, with ECDSA for both looking like this:

let privateKey: K1.ECDSA.PrivateKey = .init()
let publicKey: K1.ECDSA.PublicKey = privateKey.publicKey
let signature: K1.ECDSA.Signature = try privateKey.signature(for: hashed)
publicKey.isValidSignature(signature, hashed: hashed) // true

And here are the shape of the different recoverable ECDSA API, where the first variant uses a distinct namespace as previously discussed, and the second variant uses a nested namespace (I've explicitly declared variables with their types so we can see them spelled out):

let privateKey: K1.ECDSAWithKeyRecovery.PrivateKey = .init()
let publicKey: K1.ECDSAWithKeyRecovery.PublicKey = privateKey.publicKey
let signature: K1.ECDSAWithKeyRecovery.Signature = try privateKey.signature(for: hashed)
publicKey.isValidSignature(signature, hashed: hashed) // true

vs

let privateKey: K1.ECDSA.KeyRecovery.PrivateKey = .init()
let publicKey: K1.ECDSA.KeyRecovery.PublicKey = privateKey.publicKey
let signature: K1.ECDSA.KeyRecovery.Signature = try privateKey.signature(for: hashed)
publicKey.isValidSignature(signature, hashed: hashed) // true

Which one do you like best?

8

I like the first (the one in your original PR). The symmetry in the name space makes it more clear they are two sets of APIs. I think the second one is better if a sigle signature struct is reused for both unrecoverable and recoverable (as discussed above it's infeasible). Just my thoughts.

1 Like