Formalizing the unavailability of Core Foundation

CF on platforms other than Darwin has been a topic on which there has been a consistent response, but that consistent response always included the caveat that things were in a transitory state. The commitment I've echoed so far has been that:

  • Core Foundation is an implementation detail of Foundation outside of Darwin.

  • To underline this, you cannot use CF API if you do import Foundation (which you can do on Darwin).

  • However, due to a set of implementation details, we needed to ship the CF module. This meant that code that did import CoreFoundation explicitly could still use CF on Linux.

  • This has proven a considerable implementation headache. Several features of CF you would reasonably expect to work simply do not or do not do so reliably or correctly (most notably, custom allocators when paired with objects specifically) in a non-Darwin world.

To mitigate this, I've been working with some port maintainers to make sure that Core Foundation is not available to use on platforms we ship anew. Most of this enforcement has been by preventing the linker from seeing CF symbols, which is not ideal and still poses some complications (as the CF module still needs to be shipped even on platforms where it is unusable).

I've been working on a plan to formalize this unavailability over time without breaking any current code today*. However, the plan does have an eventual endpoint in the middle future where import CoreFoundation will be a hard error in your source code.

(* On platforms for which we distribute build products β€” the ones listed at https://swift.org/download.)

(This aside contains a rough timeline of this change.)

Already implemented:

  • Newly brought-up platforms (like Windows) mark all Core Foundation symbols as not exported or private β€” you cannot link your code to these symbols on those platforms.

Today:

  • Foundation moves CF under @_implementationOnly, preventing it from being reexported, in most of its files and all of its implementation (that is β€” even files that aren't @_implementationOnly are rewritten as much as possible to work as if @_implementationOnly was used). I have just merged this one.

  • On platforms other than Linux, we force all Foundation files to import CF only through @_implementationOnly. I merged this one too.

  • Once this lands today, port maintainers can start removing the CoreFoundation module (again: on platforms other than Linux where this module was available but never worked.)

In the future:

  • We cover needs that CF has API for but the cross-platform subset of Foundation doesn't with experimental or new definitive API, or implementation of missing API (such as NSMapTable/NSHashTable.)

  • Linux has a warning period about removal, and then a hard removal at some point.

  • This only applies to Linux specifically; Darwin continues to have access to CF going forward.

  • There's no change to the open-source nature of CF or the fact it underpins Foundation under this plan β€” only its availability for direct use by outside code.

Before I commit to this, I wanted to ask:

Does your code rely on Core Foundation outside of Darwin? (Most importantly, do you do so on Linux?) And, if so: what parts of Core Foundation do you rely on, if you can share?

Please let me know; I want to only commit to this course of action if we know that your needs will be supported going forward.

cc @compnerd

ETA: All changes marked β€˜Today’ in the timeline are now in master.

12 Likes

Core Foundation?

Sorry: we force all Foundation files to import the CoreFoundation module via @_implementationOnly.

2 Likes

Just did a quick look in some open source projects we use and found these 2 CoreFoundation uses on Linux:

I won't pretend to be an expert on these though so there could definitely be easy replacements.

1 Like

That entire file in ReactiveX shows both some of the issues we've had over the years (such as incorrect importing of CF sources β€” in fact, that file does not compile right now because we have fixed the import issue where CF constants were not being imported as enums or OptionSets), and also some of the areas we know Foundation has issues (all of the things they use except for CFRunLoopWakeUp() have either existing API, or experimental API replacements β€” look for "For XCTest use only".)

DBL_DECIMAL_DIG is from <float.h> β€” CF re-exports stdlib headers, and we never formalized which stdlibs get exported how, but you're correct that we may need to make sure that importing Foundation re-exports certain symbols, though which set thereof is up for discussion.

3 Likes

To answer the question: Yes, my code relies on Core Foundation outside of Darwin (and also on Darwin). In particular, I use Core Foundation to access IBM Code Page 437 string encoding (also known as DOS Latin US).

1 Like

Thanks for bringing this up; I made note of the issue.

I use kCFErrorDescriptionKey from <CoreFoundation/CFError.h>, because there isn't an equivalent NSDescriptionErrorKey in <Foundation/NSError.h> (FB6935101).

2 Likes

There are a couple of places where I use CFRunLoop instead of RunLoop because the only way I know of to stop a run loop is by using CFRunLoopStop. There doesn't seem to be an equivalent API for RunLoop.

1 Like

Hello
Great write-up, answers a lot of questions about source of some warnings in my code :) Shame I noticed it so late
just like @Peter-Schorn, i'm using CFRunLoopStop. Also I am using CFRunLoopSourceContext1 and CFRunLoopSourceCreate + CFRunLoopSource to watch epoll file descriptors. And I also use CFRunLoopAddCommonMode to add custom run loop mode
Yes, all on linux :)

Looks like we can't use #import <CoreFoundation/CoreFoundation.h> on non-Darwin platform but it is actually there with the toolchain (toolchain/swift/CoreFoundation/CoreFoundation.h).

Is this an expected behavior? And can we depend on the macro's CF provide on non-Darwin platform?

I don't think it's a good idea to resurrect 2-year old threads, especially in addition to the other thread you've already created. As moderators have previously suggested, it's better to link to the old thread instead of bumping the old one. I'll answer the questions here though for posterity, but wouldn't mind if moderators lock this thread to redirect the discussion to the new one.

Yes, CoreFoundation is a private API on non-Darwin platforms, it may go away or change without notice in future versions of Swift.

No, as it's a private API that should not be accessed anywhere outside of swift-corelibs-foundation.

Note that it is unavailable on Windows, and so, if you use import CoreFoundation, you have implicitly made your package non-portable.

2 Likes