With the new official Android SDK, binary sizes are becoming a bigger issue for Swift and may be blocking some adoption, and one of the reasons is especially the use of import Foundation throughout the package ecosystem. This is due to the size of the ICU package in Foundation, and there are ongoing discussions on how we could improve that situation on platforms, such as Android.
Several popular packages, that people would like to use on Android, still make use of import Foundation. From what I could tell, library authors generally want to use import FoundationEssentials, but because of Swift leaking the public API of imports, this cannot be done in a non-source-breaking way. Therefore, requiring the release of a new major version.
Also, improving this situtation would of course also have benefits for other platforms, such as Linux, not just Android. But it is especially important for Android adoption imo.
Here are some of the issues/PRs I could find of this issue in popular repositories:
I am just creating this thread to bring the situation to the surface again and opening a discussion. Particularly, I am curious if the authors of the above libraries would be OK with cutting new major releases because of this change. And if not, why not?
(Obviously if this was for a project I'm getting paid for to work on and maintain, forking everything would be a given and then making those changes myself would be easy)
Thanks for surfacing this again, Mads. As you mention, the linked thread does discuss some potential solutions to getting the ICU dependency size down (externalizing the ICU data so it can be thinned and not duplicated across the Android architectures, or adding a FoundationAndroidICU substitute library that uses JNI upcalls for its i18n needs in order to share the system ICU data, etc.) But it isn't something that is going to be solved in the short term.
You are right that this is hampering adoption: it is the number one reason our users cite for sticking with Skip's "Lite" mode (where a transpiled Hello World apk is 17MB) rather than the "Fuse" mode that uses the Swift SDK for Android (where a compiled Hello World apk is 124MB). Solving this is a high priority for us.
But in the short term, eliminating unnecessary dependencies on FoundationInternationalization would be a great help. I understand why this causes a source-breaking change (and thus forces burning a major release under the policy of many projects):
#if canImport(FoundationEssentials)
import FoundationEssentials
#else
import Foundation
#endif
ā¦ however, I wonder if perhaps this might pass muster:
#if os(Android) // and os(WhateverElse), like MUSLā¦
import FoundationEssentials
#else
import Foundation
#endif
It might require the call site to add an extra import sometimes, but given that this would only be for platforms that weren't (officially) supported until now, maybe this is a change that wouldn't need to be considered "source breaking"? After all, adding Android support sometimes requires adding extra imports anyway (see Exploring the Swift SDK for Android | Swift.org), so adding Android platform support to a package isn't expected to always just work "out of the box".
I think the goal here is the right one. I think itās going to be very difficult to achieve without the tooling in place to make it a usable developer experience
A smart way of using the Unicode data of the host system either for Android, Linux, or Windows and so reducing the usage of the ICU data and so bringing down the deployment size of Swift programs would be great. Maybe someone can explain whether this is not possible or just difficult or not efficient enough?
The problem is, when you need one thing from Foundation that is not in FoundationEssentials, then you have it. The division of packages would need to be more fine-grained to help in any circumstances, up to the point that maintenance and usage becomes difficult. A better solution in many cases might be link-time optimization (LTO). To my understanding there is some functionality, but not ready for prime time, so maybe needs more focus?
For Android, the problem is that Android's icu4c is quite minimal and doesn't expose nearly enough API for FoundationInternationalization's needs. There's no date or number formatting or parsing, no calendar logic, etc. Plus, it is only available from Android 12 (API level 31), which is on only 75% of devices. And the ICU data file format is unstable and isn't forward or backward compatible, so even if we could locate it and try to load it directly from the file system, we wouldn't be able to reliably parse it. Different API levels use different versions of ICU.
That's why I suggested making a shim library that passes the icu calls through JNI to the Java ICU4J packages (like android.icu.text and android.icu.number), which is much more complete than icu4c and is available since API 24 (95% of devices). However, there is no guarantee that it would have sufficient API coverage for Foundation's needs, it wouldn't contain Apple-specific ICU functions that are relied on, and it could very well introduce unacceptable performance bottlenecks. It might work, but it would take a lot of work to find out.
We aren't the only ones facing this issue on Android. Flutter apps also need to bundle their own i18n data files, but they include tooling to thin out their data files for just the locales the app supports, which greatly reduces the size of the data file. We'd need to create our own equivalent tool (perhaps based on the ICU Data Build Tool | ICU Documentation). I had a draft PR to explore the possibility of externalizing the data file, but it never garnered much attention (and then got accidentally closed).
Anyway, there's your answer for Android Other platforms might have similar reasons.
FWIW, I personally think we should take those source-breaks and live with the potential of breaking a few builds. Releasing new majors of packages is an even more breaking change as we have seen with the gRPC 2.0.0 release. We already have a few packages in the ecosystem that made the transition from import Foundation to import FoundationEssentials. I will bring the topic to the Ecosystem Steering Group in our next meeting to discuss further.
I completely agree. Iām convinced projects that prioritise importing FoundationEssentials will happily accept a source break in return. Conversely, projects that donāt care will almost certainly already have an āimport Foundationā somewhere in their code and therefore wonāt even notice the source break.
Just as a little sidenote, I think that Foundation is incredibly good. I understand that you wouldnāt want to use it in some basic protocol service endpoint and that it may be a pain there.
But for anything UI, whether Wasm things, Android apps or server side templates, I would always want full Foundation to be available.
I don't really understand the Android issue. Don't Java/Kotlin apps have a way to do localization related things? If so, why not just delegate that task to the host system? I mean not in Foundation using the system ICU but the developer using the system ICU like they would in a plain Java/Kotlin app and then not using the whole Foundation, just FoundationEssentials.
And then it's the same problem as other platforms where some common packages import Foundation for no reason and then don't remove that import because that would be a breaking change, so the developer has to maintain their forks with Foundation removed.
Iād actually also push on the side of ājust do itā for theseā¦ Itās a looming problem ever since the Foundation split was introduced but weāve not benefitted from it as much as we could have because these compatibility concerns.
It is true that the problem of relying on a āleakedā Foundation import is likely to hit people who were doing so unknowingly, but on the other hand, the solutions are not that hard of a liftā¦ Iād personally be supportive to take a lanient approach to guaranteed source compatibility here and just go for it, and address the breaks wherever necessary.
That absolutely does seem like the most useful medium/long term goal to strive for - hope we can figure out the when and how to get progress towards that. If this is a high priority as you say, we could figure out a road towards this solution Iām sure.
Of course, but as Marc explained above, Foundation relies on ICU for other APIs also, which Android doesn't really provide as a system C library:
For those who want to use the full Foundation beyond Essentials, that choice then bloats up Swift apps on Android, as we then ship the entire ICU data array to support that. We need to do some work to thin that out, but it will always add some weight.
I recently did an experiment using OpenAPIGenerator in an ElementaryUI wasm web app.
Even in full Swift, a "hello world" can be stripped down to <2 MB (which may be workable in certain situations where Embbeded Swift is too limiting). Adding the OpenAPIRuntime (which uses Foundation) gets you a 19 MB binary (optimized, compressed) - which is essentially unusable on the web.
Now, I do not know how much of this is due to Foundation, or what effect depending on FoundationEssentials instead would have, but I want to echo the concern that binary sizes matter - this is a serious issue for Swift adoption.
I am very much in favor of this. The way I see this:
leaving it as it is -> every package that uses Foundation remains a toxic binary size killer
having entire package ecosystems make a major jump -> very disruptive and realistically not going to happen
"just do it" -> a few people might need to add an import Foundation in their code that they should have had anyway, but the ecosystem can start "healing"
Other platforms might have similar reasons.