# Llbuild2 **URL:** \ **Category:** LLBuild\ **Created:** [May 26, 2020, 9:08pm UTC](https://forums.swift.org/t/llbuild2/36896 "2020-05-26T21:08:18Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![ddunbar](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ddunbar/32/1933_2.png) [@ddunbar](https://forums.swift.org/u/ddunbar)\ **Post date:** [May 26, 2020, 9:08pm UTC](https://forums.swift.org/t/llbuild2/36896/1 "2020-05-26T21:08:18Z")
Hi all, Almost 4 1/2 years ago, we open sourced llbuild alongside Swift and the Swift package manager. At the time, llbuild was still fairly new, but it's design has largely stood the test of time, powering SwiftPM builds as well as other products like Swift playgrounds and Xcode's XCBuild build system. And in that time, llbuild has driven a _lot_ of builds! In the meantime, a lot of other things have changed in the world. Swift's ecosystem has grown considerably, including substantial packages like Swift NIO giving us far easier access to sophisticated network technology from Swift. The world outside of Swift has changed too, with Bazel's [Remote Execution API](https://docs.bazel.build/versions/master/remote-execution.html) becoming something of a _de facto_ standard with many backend implementations (like [buildbarn](https://github.com/buildbarn), [buildfarm](https://github.com/bazelbuild/bazel-buildfarm), [BuildGrid](https://gitlab.com/BuildGrid), and [scoot](https://github.com/twitter/scoot)) and multiple frontends (like [Bazel](https://bazel.build), [BuildStream](https://www.buildstream.build), and [recc](https://gitlab.com/bloomberg/recc)). We've been pondering on how best to make forward progress on all the other things we'd [like to do](https://github.com/apple/swift-llbuild/tree/4e5605f287522a90814e12eca3777785446f6027#open-projects) with llbuild, while also taking into account some of the existing [layering challenges](https://github.com/apple/swift-llbuild/blob/41a4664b96313b8b4f83ea0f89e095c21f0ff0dd/docs/buildsystem-internals.md#known-issues) in llbuild. About a month ago, @David_M_Bryson started playing around with a [concept](https://github.com/apple/swift-llbuild/tree/0816cf594c2e4d890d1f188271da667f99763fa9/experimental/cevobuild) of doing a new implementation of llbuild's API, but sitting atop NIO's Futures. We've since written a number of experiments atop it which make us think this is now the direction we'd like to go for a more modern API. At the same time, we've been having a lot of discussions with @Aciid and @sergiocampama about how we should go about supporting Swift "explicit modules" in the build system. One thing that quickly emerged in those discussions is there is an opportunity to get much more alignment between llbuild, SwiftPM, [swift-driver](https://github.com/apple/swift-driver), and the Swift compiler. With that in mind, we are going to try something new... llbuild2! This an experiment, we don't yet know exactly how it will pan out, but our intention is to try it for a bit and see where it takes us. If we like it, we will most likely fold it back into the primary llbuild repository as the future direction for its API. Here is what we are planning: - We've created a new [`swift-llbuild2`](https://github.com/apple/swift-llbuild2) repository to hold this work. This will be a pure Swift package for the time being (no CMake, and not worrying about bootstrap concerns yet). - We will most likely use the existing `llbuild` as a dependency, and for sinking performance critical code into C++ (where necessary). - We've moved the `cevobuild` experiment out into `llbuild2` , and will begin to build up more, better, higher-level build system layers using the lessons we've learned. - Unlike `llbuild`, which tried very hard to have minimal dependencies, we can be much more open on taking on Swift package dependencies to let us build up functionality faster or in a more maintainable way. - In particular, we'd like to adopt Bazel's Remote Execution API so we can begin supporting some forms of automatic distributed builds on top of `llbuild2` . We also intend to adopt[gRPC-Swift](https://github.com/grpc/grpc-swift)for RPC support, when useful in[other contexts](https://bugs.swift.org/browse/SR-6053). - We will also be creating a `swiftpm-on-llbuild2` repository (wow, this is a mouthful), which will extend the existing `swift-package-manager` to use `llbuild2` for its build system. - Our intention is to use that repository to develop SwiftPM-specific build rules on top of the new llbuild2 model, including work to develop the explicit modules support in `swift-driver` to make that performant. ([Fast dependency scanning for Swift by DougGregor · Pull Request #28515 · apple/swift · GitHub](https://github.com/apple/swift/pull/28515)) - When combined with `llbuild2` support for Bazel's Remote Execution API, this should allow us to start exploring fully distributed/cached builds for SwiftPM packages (when a suitable backend is available). - As with `llbuild2`, if those experiments work out, we intend to fold that back into the primary repository. We are pretty excited about this new direction... if you are interested in contributing, come join us in #llbuild-dev on the [SwiftPM slack](https://swift-package-manager.herokuapp.com). Cheers, - Daniel ---
**Author:** ![SwiftUI](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/swiftui/32/24361_2.png) [@SwiftUI](https://forums.swift.org/u/SwiftUI)\ **Post date:** [May 26, 2020, 9:25pm UTC](https://forums.swift.org/t/llbuild2/36896/2 "2020-05-26T21:25:27Z")
looks amazing ---
**Author:** ![Dave\_Lee](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/dave_lee/32/4954_2.png) [@Dave\_Lee](https://forums.swift.org/u/Dave_Lee)\ **Post date:** [May 26, 2020, 9:50pm UTC](https://forums.swift.org/t/llbuild2/36896/3 "2020-05-26T21:50:10Z")
> [@ddunbar](#): > > supporting Swift "explicit modules" in the build system @ddunbar I'm not familiar with the term "explicit modules", can you define it for us? thanks ---
**Author:** ![ddunbar](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ddunbar/32/1933_2.png) [@ddunbar](https://forums.swift.org/u/ddunbar)\ **Post date:** [May 26, 2020, 10:03pm UTC](https://forums.swift.org/t/llbuild2/36896/4 "2020-05-26T22:03:02Z")
The concept "explicit modules" is an idea to get off of the implicit module cache systems used by Clang & Swift, and instead do something based off of [Fast dependency scanning for Swift by DougGregor · Pull Request #28515 · apple/swift · GitHub](https://github.com/apple/swift/pull/28515) and related work, so that the build system could end up understanding how to actually build the modules. I believe @Douglas_Gregor intends to write up a clearer proposal around what some of his thinking is there, so I'm going to 🤞 for that. ---
**Author:** ![FranzBusch](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/franzbusch/32/19447_2.png) [@FranzBusch](https://forums.swift.org/u/FranzBusch)\ **Post date:** [May 26, 2020, 10:16pm UTC](https://forums.swift.org/t/llbuild2/36896/5 "2020-05-26T22:16:26Z")
This sounds great! I really hope the experiment with llbuild2 works out and we get a build system that can support cached remote artifacts and distributed builds. Will you also move over to use file signatures instead of time stamps @ddunbar? ---
**Author:** ![sergiocampama](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sergiocampama/32/9330_2.png) [@sergiocampama](https://forums.swift.org/u/sergiocampama)\ **Post date:** [May 26, 2020, 11:15pm UTC](https://forums.swift.org/t/llbuild2/36896/6 "2020-05-26T23:15:05Z")
@FranzBusch we consider digest based approaches for change detection very much a core feature in modern build system infrastructure, so yes, we are thinking on the invalidation logic being centered around the contents of the sources, not on the timestamps ---
**Author:** ![David\_M\_Bryson](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/david_m_bryson/32/4833_2.png) [@David\_M\_Bryson](https://forums.swift.org/u/David_M_Bryson)\ **Post date:** [May 26, 2020, 11:16pm UTC](https://forums.swift.org/t/llbuild2/36896/7 "2020-05-26T23:16:59Z")
Yes, we plan to introduce a Content Addressable Storage protocol and to have the keys become the DataIDs that identify the content. That way everything is modeled as a hash of its content or description. ---
**Author:** ![Chris\_Lattner3](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/chris_lattner3/32/348_2.png) [@Chris\_Lattner3](https://forums.swift.org/u/Chris_Lattner3)\ **Post date:** [May 27, 2020, 12:20am UTC](https://forums.swift.org/t/llbuild2/36896/8 "2020-05-27T00:20:19Z")
This is super exciting, congratulations - I can't wait to see where this goes! ---
**Author:** ![ddunbar](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ddunbar/32/1933_2.png) [@ddunbar](https://forums.swift.org/u/ddunbar)\ **Post date:** [May 29, 2020, 11:37am UTC](https://forums.swift.org/t/llbuild2/36896/9 "2020-05-29T11:37:30Z")
@Dave_Lee in case you missed it, Doug’s write up is here: [Explicit Module Builds, the new Swift Driver, and SwiftPM](https://forums.swift.org/t/explicit-module-builds-the-new-swift-driver-and-swiftpm/36990) ---
**Author:** ![schuett](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/schuett/32/12935_2.png) [@schuett](https://forums.swift.org/u/schuett)\ **Post date:** [June 5, 2020, 8:09pm UTC](https://forums.swift.org/t/llbuild2/36896/10 "2020-06-05T20:09:47Z")
Are you going to handle jobs with different levels of parallelism? Assume clang jobs are single threaded while lld jobs use 8 threads. I would assume that ninja is unaware of that problem and overloads the machine. ---
**Author:** ![ddunbar](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ddunbar/32/1933_2.png) [@ddunbar](https://forums.swift.org/u/ddunbar)\ **Post date:** [June 5, 2020, 10:18pm UTC](https://forums.swift.org/t/llbuild2/36896/11 "2020-06-05T22:18:09Z")
This is somewhat unrelated to llbuild2 as a specific project... it is a problem we are aware of and would like to eventually have better solutions for (in both llbuild & llbuild2). ---
**Author:** ![Jean-Daniel](https://avatars.discourse-cdn.com/v4/letter/j/977dab/32.png) [@Jean-Daniel](https://forums.swift.org/u/Jean-Daniel)\ **Post date:** [June 2, 2023, 4:44pm UTC](https://forums.swift.org/t/llbuild2/36896/13 "2023-06-02T16:44:22Z")
Probably superseded by [GitHub - apple/swift-llbuild: A low-level build system, used by Xcode and the Swift Package Manager](https://github.com/apple/swift-llbuild) ---
**Author:** ![adincebic](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/adincebic/32/43371_2.png) [@adincebic](https://forums.swift.org/u/adincebic)\ **Post date:** [November 25, 2023, 5:11pm UTC](https://forums.swift.org/t/llbuild2/36896/14 "2023-11-25T17:11:22Z")
Hey @ddunbar any news regarding llbuild2? I see that llbuild2 github repo has some activity but found nothing else. Can you provide us with information on how the project is progressing?