Building CoreFoundation.framework from swift-corelibs-foundation on macOS

build-script

(Daniel Peebles) #1

Hi all,

For a certain project, I've been building the CF opensource drop from opensource.apple.com for years. After swift-corelibs-foundation got released, Apple appears to have stopped releasing CF (there haven't been any releases since 10.10), which makes me sad because I'm missing out on any of the new functionality. I was pleased to see that the CoreFoundation source code in swift-corelibs-foundation actually appears to be far more complete than the CFLite that used to get released, but the build is now very different and it seems quite difficult to wrangle it into producing the .framework properly for me.

I've tried amending the CoreFoundation/build.py to create a Framework instead of a DynamicLibrary, but unfortunately the resulting ninja build file is broken and doesn't work. I'm currently hacking up the DynamicLibrary build and manually copying the dynamic library and headers to the correct locations in CoreFoundation.framework, but this feels hacky and frustrating when the build system is already so close to producing a legit .framework and clearly supports the functionality already (even separating private headers from public ones!).

Are there plans to make the CoreFoundation build in swift-corelibs-foundation more of its own standalone thing? When I tried to build it yesterday, I noticed the build was actually broken (I submitted a PR which is now merged), so I get the sense that nobody uses it except paired with Foundation, for Swift. On one hand, I don't want to go "against the grain", but on the other hand, the lack of CF releases on opensource.apple.com leaves me little choice.

I do realize that this isn't really a supported use case, but it's also so close to being possible that I'd be willing to put in the work to make a separate CoreFoundation.framework build working, if folks at Apple would be willing to accept PRs toward that goal.

Thanks!


(Keith Smiley) #2

I recently filed rdar://41426490 about this.


(Daniel Peebles) #3

@Keith cool! Was your radar about the lack of CF releases or about swift-corelibs-foundation's CF not being particularly useful on its own?

I do have it working right now, but it requires a lot of hackery. For example, the current build in that repo assumes that Foundation is going to call the __CFInitialize initializer, so if you try to use the library in isolation it'll just segfault in mysterious ways until you realize that you need to add the initializer manually (I added an ldflags entry for it, but you could also patch the preprocessor macros).


(Keith Smiley) #4

It was about the lack of open source dumps in general. Specifically some developer tools not being dropped since Xcode 8.


(Lily Vulcano) #5

Right now, CF exists mostly as an implementation detail of Swift Foundation. You can conceivably build it for other uses, but we haven’t put any work into sustainably supporting this for any platform other than Linux. We welcome contributions this way, but note that on Darwin, modulo for the very narrow scope of building Swift Foundation for testing, I think it’s safe to say we really, really would prefer people use the installed CF instead.


(Daniel Peebles) #6

@millenomi thanks for the reply! I have indeed now built it for other uses and after some minor hackery (e.g., that initializer), it seems to be working fine :slight_smile:

I realize that y'all would prefer us to use the system CF but we have differing needs, and in one planned scenario there might not be a system CF (think e.g., OpenDarwin/PureDarwin, although I'm not involved in those projects), so getting this working is still appealing.

I'm mostly just trying to minimize my deviation from upstream CF, so for example I have changes like this which make it work in that scenario without me having to apply local patches. I do have other ones like that, but have hesitated upstreaming them in case this type of work was unwelcome.

Ultimately what I'm hoping is for there to be a version of CoreFoundation/build.py that produces a valid CoreFoundation.framework that one might actually be able to use standalone (so the initializers should actually be in the lfdlags and so on). I'd obviously keep as much shared build code as possible with the build y'all have to minimize maintenance burden (for me and you) but there would probably be a bit of branching in the build.py to accommodate some of the extra frameworky concerns.

Anyway, if you don't hate the idea, I can put up a "tentative" PR or two in the next few days with some ideas for how to do that and see what you think. And if it ends up being too noise or burdensome on your maintenance we'll find some other way.


(Lily Vulcano) #7

I would very much like to see your PRs :) Thank you for helping!


(Tony Parker) #8

In addition to what @millenomi says, I'd like to also verify that our intent is that the swift-corelibs-foundation version of CoreFoundation replace the one on opensource.apple.com. We believe GitHub is a way better open source experience than the static tarball dumps on the web site.

I can only speak for CF itself though -- other projects on that site are managed differently.


(Daniel Peebles) #9

@Tony_Parker thanks! It might pay to make that clear on the opensource.apple.com site if the release mechanism allows for such flexibility in wording. Far better "UX" to say "check out GitHub!" rather than that "coming soon!" which still lingers on the 10.11 and 10.12 releases :slight_smile:

Either way, fully agree that it's a far far better experience. It's lovely being able to submit PRs and watch development as it happens, rather than waiting for faceless dumps of code that may or may not work with no recourse if it doesn't. Whoever made this happen, huge kudos to them!


(Tony Parker) #10

Understood about the wording on that site. I will follow up with the team that maintains it. rdar://44316179