What’s next for Foundation

Date type is in Essentials, but without TimeZone type in I18n,
its string representation can't be constructed,
so it can't be displayed on applications.

How use cases are intended?

It can measure elapsed time,
but standard Clock API is sufficient for that.

Date is always expressed in UTC, so TimeZone is not necessary for pure computational and algorithmic usage, such as dealing with unix timestamps and parsing ISO8601 dates.

It looks like all of the types used for formatting dates in a human-presentable way, like TimeZone, Calendar, Locale etc, are (appropriately) in the FoundationInternationalization package.

6 Likes

Great news. I'd love pure-Swift Foundation.
Let me give some bike-shedding.


As String Manifesto says, we must rename it first rather than only changing APIs.


Do "NS-prefix collection types" include NSString? I believe NSString is also no longer useful nowadays.


  • AttributedString

Current SwiftFoundation is not completely independent from compiler/stdlib because some types like this require @_spi feature.
Will such a situation continue?


It's a good point. We can remove Date type itself.

1 Like

I saw several replies refer to a Lock type, but unless they are talking about NSLock, I cannot find any references to this in documentation or in code.

Further, unless I'm missing something, I don't feel like structure concurrency can fully replace everything offered by (NS?)Lock. Perhaps even a little with OperationQueue and Timer. I admit I don't understand RunLoop enough to make a judgement there, but expect that it might be similar.

To be clear, I use and prefer structured concurrency whenever possible. It just seems that it doesn't quite offer a complete solution. Yet, at least.

edit: I appear to just be echoing @Jon_Shier's sentiments :slight_smile: What’s next for Foundation - #3 by Jon_Shier - I'm excited by the announcement otherwise, though!

4 Likes

Tons of great responses here, thank you everyone! I'll try to consolidate some replies here.

First, some high level clarifications on the API surface.

Darwin Foundation (what we will call 'Foundation framework' to distinguish it) will be a superset of the package, because it will include all of the implementations we cannot remove or refactor as part of this work (stuff written in Objective-C or which we decide to omit from the package for some reason). Clearly we can't just yank the rug out from all existing apps on iOS or macOS. Therefore, the main task here is to refactor the code that we choose to have a common implementation into Swift and make that the core instead of C or ObjC.

We've prototyped this approach and are pretty sure it will work out well. A reimplementation of Calendar in Swift is 1.5x to 18x as fast as the C one (calling from Swift in various synthetic benchmarks like creation, date calculation). And it's completely compatible with the existing C and ObjC entry points, too.

We can choose to omit existing methods from included types in the package, but we can't remove it from Darwin for compatibility reasons. We could implement those methods partially in ObjC (available only on Darwin) and partially in Swift (available everywhere), or just bring that API along to Swift so we have a more unified implementation. I think it will be a case-by-case decision, but we are likely to choose the latter for maintainability and usability reasons.

We can also choose to add new methods to these types, where we think it will add value (e.g. the suggested improvement of hex utilities to Data). That is the purpose of developing an open contribution process. New methods will be available on all platforms, unless we decide it's irrelevant on some for a good reason.

The process can also be used to deprecate existing API. I do want to note that Foundation has a high bar for deprecation. It can be very disruptive to client apps when they upgrade the SDK. We generally only deprecate when:

  • The API is actively harmful for some reason,
  • After giving plenty of notice and,
  • having replacements available for a significant amount of time.

This work is going to take some time, so we will start with the contents of Essentials and Internationalization and see how it goes as we progress through all of those API. Networking and XML will come later.

Now, about some of the concrete types:

Stream, Port, Timer

Stream, Port, and Timer are inherently tied to RunLoop, which is itself inherently thread-based. Given the move away from concrete threads with the introduction of tasks in structured concurrency, it doesn't seem like these API are the right model to encourage going forward. I'm still open to discussion on that point.

It may be the case that we could use a new API for structured concurrency that adjusts the fire date like Timer and RunLoop do. I'm not sure.

NSCoding

NSCoding is inherently tied to NSObject and keyed archiving to the Objective-C runtime for class names. I don't see a lot of value in attempting to reimplement it in Swift. The existing swift-corelibs-foundation repository will continue to exist (perhaps with a different module name?) in the future, if there is a compatibility requirement to use it on non-Darwin platforms.

Lock

This one is controversial for sure. In fact, Foundation needs it internally. However, it's unclear to me if we should bring it forward into the package as a public type or not. I'm open to discussion.

Date

We litigated this one extensively in the previous discussions about Duration and the conclusion there was to keep it in Foundation. We'll honor that decision by keeping it in Essentials. Date itself is just a Double, really, so it can be used independently of any dependency on ICU. Formatting it, or using it with Calendar or TimeZone does currently require ICU support and therefore those APIs will be in the Internationalization module.

Decimal

I think we may end up needing this for JSONSerialization, I'm not sure. If so it'll be in Essentials. Decimal suffers more than some other types from a poor translation into Swift's ideas around numerics, so it may deserve some additional attention.

Forward/backward compatibility and module names

We have some work ahead of us to figure out how we transition the platform to the new package. Since Foundation is shipped with the toolchain on Linux and the SDK on Darwin, calling the new package Foundation will result in some conflicts. We've decided not to rename it (because it's always going to be Foundation on Darwin), so I think we will need some kind of mode switch. I could imagine, for example, Swift 6 mode renaming the existing swift-corelibs-foundation module to get it out of the way. Another option is to provide a compiler flag for both Darwin and Linux like --no-sdk which prevents the import of anything from the toolchain on Linux or the SDK on Darwin, so the packages you included are the only option and the potential for a conflict is removed.

I think we may choose to have the package simply called FoundationPreview as a very first step, so we can give ourselves some space to resolve these questions.

38 Likes

Love to see ‘LocalizedError’ in the list of types that need an update. I’m hoping that means the ‘errorDescription’ will finally become a non-optional ‘String’ to be actually useful. And that it will also gain an ‘errorCode’ API so we can build (nested) error types which are uniquely identifiable and can even represent an ‘error call stack’ if nesting is used.

I’m planning to write a blog post to elaborate more on this topic and will link it here, but for now I’m just happy about this part of the announcement!

8 Likes

+1 Less namespace pollution is great. I’d prefer to see the essentials module further broken up. Maybe: FoundationTypes, FoundationFiles, FoundationSerialization, FoundationOS

A high-precision, low-cost time API.
Darwin's mach_absolute_time() or what is called hrtime on other platforms is standard and useful for performance measurement.

3 Likes

Personally I hope very much that locks, atomics and threads APIs will be added to the standard library. I think the current concurrency story pushed to developers is lacking and this is having a highly detrimental effect.

13 Likes

yes. atomics please

1 Like

This might be an overgeneralized/simplistic argument, but if Foundation has a need for it that isn't provided for by async/await, wouldn't there be legit uses for it outside of Foundation as well?

16 Likes

I think we should probably save redesigning Foundation's API for once the process for proposing changes is ready to go, rather than all in one forum thread ahead of time. The interesting question for a lot of things (probably including locks) will be less "should this be available?" and more "which library should this go in?".

19 Likes

If we cut the module into smaller pieces, we may want to need "submodule" feature.

import Foundation.Foo
import Foundation.Bar
import Foundation.Baz

// or

import Foundation.*

I don’t know if it’s desirable or not.

4 Likes

Nothing should be removed, (especially the items proposed for removal) . The rest of this proposal is good.

1 Like

Structured concurrency is still in its infancy and suffers a lot in terms of performance. Regardless of that removing well established classes for the purpose of forcing the still-in-infancy “new” way feels more ideologically motivated.

4 Likes

Forgot to mention the "leeway" functionality that optimises system power consumption by coalescing timers' firing.

This is quite serious. Not all code can be (or afford being) async/await-able. If RunLoop concept in its whole is a no-no in the new foundation consider some simplified version of it, e.g. an explicit "make this call every now and then when you can" model that would be useful for clients outside of structured concurrency.

Locks (various flavours) and the mentioned above atomics are very important for lower level code. "Is it useful for foundation itself?" could be a good litmus test on what to include. IMHO Swift should be generic enough to implement Swift compiler itself, a new implementation of Foundation, a device driver, a realtime function, etc.

4 Likes

The proposal is to use a flat module structure. Submodules are not needed. I assume the Foundation module would just re-export the modules like is done in other places:

@_exported import FoundationEssentials
@_exported import FoundationInternationalization
@_exported import FoundationNetworking
...etc...

Hopefully there will be some modernization of FileManager. I'm a fan of moving FileManager APIs to use a FilePath struct similar to swift-system. It would be great to see FileManger drop the delegate pattern. Callbacks might be fine for the features that currently require a delegate.

5 Likes

I guess we could add timer behavior with a new API like:

Task {
    while let cancelToken = await for(timeInterval) {
        doSomething()
        if magic {
            cancelToken.cancel()
        }
    }
}

Canceling the Task would also cancel the timer. I don't know if such an implementation would allow the same power-saving features timer had, but it would be a more general approach for timers.