Swift Supply Chain Validation

Evolution Discussion
,
1

The recent spate of cyberattacks demonstrates a need to make validating a build all the way from the most removed tools and 3rd party components through product final test something that's straightforward and secure. Before we go down the build tool road too far, I'm asking that we consider how to build security into the overall Swift supply chain / build / test / deliver architecture.

I'm specifically not trying to address the issue of protecting against attack code intentionally embedded by a developer into their product - I think doing that is in the "too hard” bucket, and in any event it’s a different bucket. If we can reach a state where we can protect our own code against attacks in tools or libraries by others, however, we'd have taken a giant step.

To that end, it's my opinion that certificates aren't enough, even after ignoring that they themselves get spoofed now and then. Without a way to recursively validate all elements back to bare metal - specifically including not only the source code and build tools, but also all related test suites and supporting test environments - we're open to attack.

A product-level identity certificate supports only an assumed level of trust that the associated product hasn't been tampered with, because the composite chain proving the state of the build's components isn't verified by the certificate, only the identity of the build’s developer is. You have to either trust that the certificate holder validated down through every leaf of the tree culminating in the build, or you have to do it yourself. Few developers are funded to do that research, which is why an automated system is so important.

By analogy, consider that although developer certificates are required in the App Store, all they establish is proof of the developer's identity, and otherwise they're content-free. Libraries loaded with malware are not scare-mongering fiction, so if a Swift "Component Store" supporting a framework marketplace uses nothing more than the same identity-based security model, it won't be able to guarantee there's been no tampering.

Instead, we need a way to know that the entire supporting tool and component chain was what the developer thought it was and intended it to be. That means every component of a build has a to be configuration controlled, and the “certificate of authenticity” has to record all the controlled elements. With that, it would be possible to screen the entire dependency tree and be certain of the provenance.

I don’t know that it has to be complicated - I think if each build can have an attached metadata file with hashes of all the component libraries, tests, test results, and build tools, plus a certificate authenticating that metadata, that’s enough to enable automated provenance checking. (I’m assuming there’s a means to avoid do-it-yourself metadata, since the idea is to not be able to spoof the metadata)

I’m more than willing for my ignorance regarding the fine points of security to be roasted; my goal is the discussion and ultimately the capability.

4 Likes
2

In about a year, this post attracted 4 likes and no other responses. Does the lack of discussion mean the topic is unimportant to the larger community, or perhaps what I wrote is simply so far off target that people didn't see it as worthy of their time?

My hope, originally, had been that the metadata I suggest, and its associated recursive checking, can be incorporated into the SPM. Although I'd posted there, it was suggested creating this as a new topic. Perhaps there's a place for "SPM Evolution" discussion I've missed?

3

Your concern seems to target the fact that there are official downloadable binaries and the security implication of distributable binaries for the compiler, runtime and third party packages.

If we are talking about the SwiftPM packages marketplace, even for iOS the fast majority are distributing source code and not binaries, so your concerns don't really apply.
If you are using Swift in some mission critical system, keep a fork of the codebase and build your own compiler and runtime. At the company where I work at we build our own Swift compiler and runtime libraries for Armv7 and internally distribute them.

This is supposed to be the advantage of open source software, you can audit the codebase and compile projects yourself. Most commercial IoT projects and networking gear with Mips or Arm processors are using Yocto or Buildroot to build their firmware from scratch, not a binary distribution like Debian or Ubuntu. If you compile all the code that you use / distribute and don't rely on external build systems, then that eliminates that attack vector.

4

I feel like the level of problem you've laid out here is so broad that it cannot be solved. Builds don't bother verifying arbitrary build details because there's an inherent cycle in trust there: if you can trust that the machine is behaving the way you set it up, you don't need to verify that e.g. the compiler you installed is still intact, but if you can't, you can't trust it to actually run that compiler correctly. Signatures are really just an authenticated anti-tampering system, and having a whole chain of signatures doesn't make it any more than that.

4 Likes
5

I had explicitly intended to bound the problem to one that’s solvable, and to be focused on source code distributions, so here I’ll attempt to better sketch what I’m thinking.

    1. I don’t believe it’s practical (or likely) that projects that transitively use tens to thousands of open-source distributions can audit all that code for security issues.
    1. I think it’s possible to define a set of best practices that establish levels of security validation. For example, the levels might be something along the lines of
    • · none
    • · source code signatures check
    • · compiles without warnings
    • · passes unit tests with xx% code coverage
    • · passes Fortify (or other) scan with TBD results.
    1. I recognize as sketched this should properly be a graph of possibilities and not subsets; obviously work is required in this regard
    1. A transitive level of validation can be defined as the least secure level of validation for the source distribution and all its incorporated distributions – that is, if a distribution and all its components pass Fortify scan but one, and that one has no validation, then the composite distribution itself has no validation
    1. The information required to rebuild the validated product should be recorded as metadata for the distribution. That metadata would include elements such as
    • a. the unit test suite and logs
    • b. the build environment. This might have to dip into definition of the hardware as well as the OS identity and configuration
    • c. the toolchain
    • d. signatures or versioning data, as appropriate
    1. I don’t mean to suggest that the metadata include the actual OS or toolchain, necessarily, although I can envision instances where that could happen. Possible ways to define those in the metadata might include
    • a. URLs of the configured product forked variants of standard / other forked distributions
    • b. Docker containers
    1. Similarly, hardware could be defined a variety of ways. Some of those could be
    • a. Supplier and associated part numbers
    • b. Make / model number. The presence of enough information to enable duplicates to be procured would distinguish levels of validation in the metadata
    • c. Schematics, parts lists, and printed circuit board layout files
    1. The execution environment, with the same hardware & OS concerns as apply to the build environment
    1. Dates defining when the metadata was last built should be recorded; this would include both the date of the distribution itself and the oldest date when the metadata of any component / tool was built. These dates can be used as a “freshness date”; knowing some sub-component has passed its freshness date can be used to trigger a deep re-evaluation of the distribution’s status, while if only the distribution itself has passed its freshness date then only a re-evaluation of the specific distribution may be required.

I think knowing when something has passed its freshness date will itself require infrastructure. One example is the NIST National Vulnerability Database, but ways to address the many distributions in GitHub but not in the NVD, for example, are possibly necessary

Overall, then, the idea is not to guarantee that a framework is secure, but rather to help make informed decisions possible relative to the goals / needs / objectives of the development and user communities:

  • · for developers deciding whether or not to incorporate a framework or other component into their product
  • · for product buyers deciding whether a given product meets their specific use cases
  • · for product users deciding whether a given product continues to meet their specific use cases

Perhaps it’s sufficient for a pitch to define a manifest that would capture the metadata as I’ve described it along with the functionality that SPM or some other tool would have to provide in conjunction with that manifest. What do you think?

6

I still don't see what you're hoping all this cryptographic validation is going to achieve.

Identifying binaries with known vulnerable components seems like a useful problem to solve, but it could surely be done by just voluntarily including an unloaded section with a list of all the components and their version strings.

7

I have no intent to focus specifically on binaries; independent of whether source or binary is incorporated, I believe it’s insufficient to know a component hasn’t been tampered with. Here are some use cases I have in mind that aren’t addressed by version strings:

    1. A developer of a mission-critical application is evaluating frameworks for inclusion in the design. Two candidates are under consideration for a specific role, and both themselves incorporate a non-trivial tree of other frameworks. For both cost and schedule reasons it’s impractical to audit the entirety of those code bases, and a version string on its own offers no help. The composite security validation metadata I suggest lets the developer compare the two, however, and so have a richer basis for decision
    1. Knowing what devices or applications are affected when a vulnerability is detected is an ongoing problem. The freshness dates can be used to help analyze whether a specific vulnerability still needs to be addressed if it represents a threat within a framework
    1. A researcher is developing the well-known “I’ll just use this myself, and only a few times” application, and needs to show a framework passes a threshold value of aggregate code coverage as required by corporate IT before the application using it can be placed on the corporate network; if one specific sub-framework doesn’t meet the threshold, having selected the baselines such that the unit tests and build definition are all available means it’s a reasonable task to enhance the test suite to meet the IT requirement

I’m certain there are more, but I wanted to post this up and see what reactions might be

9

Hey @John_McCall, may I ask why this thread was delisted?

10

@John_McCall, I'd like to know too, because if somehow I'm violating rules of the road, it's unintentional and I need to know what to avoid doing! I believe the topic is vitally important, and I'd hate for it to drop off everyone's radar because I was an idiot

12

It seemed to have turned into a two-person conversation, and I was losing interest in participating. If other people want to engage, that's fine. I was probably too quick to delist the thread. But I do need to see some sign that this is converging towards actionable and useful ideas, because I do not want this to be a distraction for the forums.

Barry, honestly, it really feels like you're starting from the idea of signing all the source code, built components, build hardware details, etc. recursively because it simply seems intuitively necessary to you, and then you're trying to find reasons why doing that would be useful. None of your use cases actually require a tamper-free guarantee; they're all solved as well as they can be by voluntary metadata, because there would be no way to validate the correctness of those signatures from a binary, and clients would simply be trusting that the toolchain had not invented them.

2 Likes
13

I'll agree with John, in that I don't really see how certificate verification can be applied to prove anything interesting about a dependency graph, but perhaps that's just a lack of imagination on my part.

More generally, supply chain issues are clearly one of the eminent problems in the development sphere. I'd welcome someone doing something to tackle it, but I'm afraid it's a tragedy of the commons. We all want secure software supply chains, but ~nobody~ (well, I'll only speak for myself) don't want to invest into doing anything about it.

14

Github is doing something to tackle it, NOT perfect but valuable and meaningful; unfortunately Swift is not currently supported for now.

1 Like
15

The work by GitHub is very much part of what I’d hoped to characterize for Swift. Ignoring the issue of the GitHub work only being for Rust, it seems to me a missing component is the ability to define the status of a leaf/node code base - if I understand what GitHub does, it’s confined to propagating CVE notices. Without question, that’s probably the best way to start, and if there were an approach to adding Swift support I think that’s where efforts from this community should likely begin.

That said, I was trying to include a means to both assert the degree of risk presented by a code base and also (for those who chose to do so) the means to test the assertion.

  • Let’s assume for purposes of discussion that the work can be and has been done to define the best practices that establish levels of security validation. Each level comes with definition of how compliance is demonstrated
  • Any repository can be posted with any asserted security level. At the point it is posted, a bot could check if all the supporting metadata required for the asserted level is present such that the assertion can be independently verified. If someone else does a verification, that could be recorded. Those are of course open to spoofing, so for some level of assurance either the verifier has to be known or else the assertion has be verified by the framework user
  • All that applies as a tree walk; the goal for each user is to walk/examine the tree and ensure that the associated risk meets their requirements

In short, I’m trying to add to what GitHub has in mind a mechanism by which the risk of using a given framework can be evaluated, both initially and on an ongoing basis. I appreciated your time helping to advance the conversation

Thanks

Barry

1 Like
16

(Sorry for the huge delay in responding. I'm out of the hospital now.)

Yes, I effectively did want to sign all those things, because without them I think it's not possible to carry out an independent test using a bot. My interest in doing that is a belief that with sufficient build information a process analogous to code signing permits a catalog associating source to binary, and therefore the cost of performing trustable validations seems OK. Intuitively, that seems like a sufficient basis to be able to look for disconnects in the chain