What’s next for Foundation

Development Core Libraries
82

Considering that the standard library's string manipulation functions are quite anemic, a larger set of string manipulation functions are needed in many (most?) scenarios involving strings. If the NSString extension methods are to be removed from Foundation, something else should come in its place.

2 Likes
83

I didn't expect to see so much love for RunLoop here. :grinning:

I'm aware of the differences in capabilities between what async/await is capable of now and RunLoop. There are three reasons I put it on the exclusion list:

  1. It seems like, in the future, async/await is likely to gain more functionality like RunLoop has then RunLoop becoming async/await friendly. That said, a common ER for RunLoop is some kind of integration with custom actors, especially for the main run loop.
  2. The difficulty of porting it to Swift; it has a very C-centric interface (e.g. run loop sources). Also, the follow-up part of getting the related types like NSStream, NSTimer, etc. to have more modern Swift API.
  3. Questionable value on non-Darwin platforms. To be 100% clear - RunLoop is not going anywhere on Darwin platforms. It's the fundamental concurrency primitive for all of the UI frameworks main thread interactions.

If we can address those points, I'm willing to consider putting it back on the list.

4 Likes
84

I agree with this. I think the standard library should keep string functions to a minimum, but Foundation should expand on it.

85

alas, this is not how the NIO developers see it, at least with respect to other NIO types that have become de-factor cross-platform standards.

i don’t agree with this. the sheer size of Foundation and the amount of un-used APIs that must be brought along just to perform basic string processing tasks (like left-padding a number!) make it a non-starter for server-side use cases.

4 Likes
86

It's worthwhile noting as an aside that RunLoop is such a key and integral part of Foundation today and yet the actual code backing it -- to put it in the nicest possible terms -- shows signs of it having grown organically over a number of years and duplicates a lot of Dispatch functionality. As noted, RunLoop already shares a load-bearing backdoor with Dispatch to work and therefore in order to get Foundation to work on a new platform you have to get Dispatch to work on that platform, which seems to have its own problems.

I'd be quite happy were RunLoop to become a separate, nonportable library for nonportable Swift code, and so to add perhaps a contrary note to the RunLoop love: I'm quite supportive of this going away from Swift Foundation proper.

4 Likes
87

Sure. I think I overstated that more than I meant to. I think there is a balance. If anything is pretty esoteric it should be in Foundation. Any common string manipulation functions belong in the standard library.

EDIT: I hope moving anything that really should be part of the standard library to the standard library is part of this proposal. I'd need to look through Foundation more carefully, but there are probably some things that are in the wrong place.

2 Likes
88

I don't want to derail the thread, I will only briefly state that this is incorrect. There's no such requirement for target platforms. Swift still supports armv7, IIRC there are community-maintained builds for that 32-bit platform.

Additionally, Wasm (capitalized this way per the WebAssembly spec) is not comparable to other platforms in terms of bit width. It's fully stack-based, so it has no registers, but it fully supports 64-bit integers and floats right from the beginning. Thus, "Wasm64" is a misnomer, as it's not a new variant of the platform, but only an extension of the existing spec.

Wasm doesn't support I/O in the same way aarch64 or x86_64 don't support I/O. These matters are handled on a higher level by operating systems (POSIX, WASI etc), not instruction sets.

8 Likes
89

Yeah. I actually read up on some of the history of this after posting that. The NIO devs think NIO is too complex (requires a powerful OS behind it) to be a good portable implementation. Some targets might be impossible to implement it. I'm still of the opinion that runloops or other event handling abstractions (other than use of modern concurrency) don't belong in Foundation except maybe in an Obj-C compatibility module.

1 Like
90

One easy rule of thumb to apply is “does it take a Locale?”. If yes, Foundation, if no, probably stdlib. There’s cases this doesn’t cover but it’s a nice guideline.

14 Likes
91

perhaps this is just my lack of experience with internationalization, but is locale relevant for:

  1. left-padding a number with 0s,
  2. right-padding a number with 0s, or
  3. stringifying (not rounding!) a Double to n digits of precision?
92

3 yes, the other two no, unless I'm forgetting something. Note that I'm not claiming current stuff follows this rule of thumb, just that it's useful for thinking about where new things should go.

7 Likes
93

As long as these APIs are the only access methods for certain functionality in iOS (see also my personal use case which I have explained in What’s next for Foundation - #46 by mickeyl), Stream continues to be a great abstraction. Now with the next Foundation ditching Stream while at the same time swift-coreutils-foundation blocking contributions, I'm wary of the future.

1 Like
94

Aren’t there different numerical systems that could use characters other than 0? Or would that be handled by the font?

95

oh, yeah, maybe so. Let's not get too bogged down in this one example please :slight_smile:

1 Like
96

I think this is the best possible way forward with functionality of "run loop". Making it async is not scalable just because conceptually it does not fit. Appending to what I pointed as an example: dedicated actor representing "render context" accepting render requests (i.e. consuming events) and producing notifications about rendering done (i.e. generating events) is literally a replacement of render loop.

The only concerning things for me personally are:

  • prioritization of "event sources" like CF/NS run loop source allowed. conceptually this would be "place this event to the top of the queue", tho it sounds a bit odd
  • main thread is not thrown away on non-Darwing platforms. the concept of main thread is well established practice for every person making apps for Darwin platforms. keeping this concept everywhere simplifies greatly ability to switch development targets
3 Likes
97

I must say I didn’t understand a lot of that but I appreciate the thought you’ve put into it.

My concern remains though: I’m not looking for a way to “hijack” the default main run loop and install my own function which also blocks for the lifecycle of the program. Instead, I need a way to manually drain jobs that have been enqueued to the main queue.

To fulfil this purpose, we currently call into CFRunLoopRun(kCFRunLoopMain, 0.001) from a callback provided to Android’s Choreographer API, which is the equivalent of Apple platforms’ DisplayLink.

The flow looks like this:

  1. Swift code enqueues work onto the main queue via Task { @MainActor … } or DispatchQueue.main.async
  2. Android Choreographer calls into our custom swift render function, which renders via OpenGL and then calls CFRunLoopRun as above, which processes the jobs.

There is no place in our Swift code where we can reasonably block, because that would block Android’s existing main thread Looper. We want Swift’s main thread to be that same thread to prevent synchronisation issues. By blocking that thread, we’d hang the entire app and be terminated by the system.

Also, on Android, there is no @main function in the first place. Our native code runs (necessarily) as a shared library, again, because we have to share the process with the JVM and all the managed code running there. We do get a callback when our Swift shared library is loaded, but we certainly can’t block in it.

So what I’m asking for is hopefully actually pretty simple. To remove RunLoop we just need something like MainActor.processTasks(maxAllowedProcessingDuration: .milliseconds(10)). I don’t think the customisation point described @etcwilde’s post will solve our issue, unfortunately. CC: @Douglas_Gregor

2 Likes
98

There is a hook in the Concurrency runtime called swift_task_enqueueMainExecutor_hook which should allow you to forward MainActor jobs to an existing service like Android's main-thread queue. When the queue invokes you, run the job using swift_job_run, passing swift_task_getMainExecutor() as the executor.

That would probably be a reasonable default behavior on Android. It's not a configuration that has gotten a lot of contributions, but that's not because they'd be unwelcome.

8 Likes
99

I am, perhaps, unreasonably happy about this.

100

This is a pretty important if subtle point. The UI frameworks themselves make heavy use of {,NS,CF}RunLoop, and a lot of existing code would break if its timing shifted relative to work the frameworks already schedule against the main thread’s runloop.

2 Likes
101

Thanks, I’ll have to look into this further to understand how to use it exactly and whether it indeed solves our problem.

If I could see an example of existing code doing this (eg. in dispatch_main, or CF’s equivalent) I would feel more confident about using it going forward. I will have another look at those functions.

To be clear, this would essentially involve calling into private APIs of the Swift runtime? I’m hoping the fact that it’s called “…_hook” means its use is somewhat sanctioned and will not disappear in a future release. You mention contributions in that direction - are additions to Swift-accessible public API in scope for that?

Finally, I assume that solution would not drain jobs enqueued via DispatchQueue.main, only Swift native concurrency, is that correct?