In @kateinoigakukun's thread on filtered ICU data, @tinaliu listed "a dynamic ICU build to support limited locales" and asked whether the challenge is packaging the data in a flexible way (t/81581). In my ICU data thread, @xwu pointed out that per-locale subsetting in the ongoing Swift work would have a larger impact than another flavor of ICU (t/90098). This doesn't replace that work: it applies the same locale list to the data that is still in ICU, as an opt-in, with the default unchanged. I measured what it saves and what breaks.
What I built. swift-foundation-icu's data with 10 languages (en, es, fr, de, it, pt, ru, ja, zh, ar and their regional locales), rebuilt with genrb and ICU's parent rules. Keeping every locale gives back the current data byte for byte. This rebuild was only for measuring; a maintainable version would use ICU's own localeFilter when the data is built from Apple ICU's sources.
Results
- Data: 34.0 → 17.6 MB. A minimal Android app with
import Foundation(arm64): 65.7 → 49.4 MB installed, 24.0 → 18.9 MB gzipped. With onlyen, the data is 14.1 MB; each further language adds 0.15–0.3 MB, exceptzh(1.8 MB). - The kept locales give the same output as with the full data: swift-foundation's tests on macOS, swift-corelibs-foundation's tests on Linux, and a probe of formatting, names, sorting and case mapping in 21 locales, and a smaller probe on an Android emulator.
- Every test that fails uses a dropped locale. A dropped locale doesn't crash: it formats like
en_US_POSIXor root, with no error.Locale.availableIdentifierslists only the kept locales (1034 → 316).
Consistency. On Apple platforms an app always has every locale, so an app with a list would format a dropped locale differently there. That's why it should be the app's choice, and why the fallback matters.
One list for both kinds of data. The generators for the Swift date and list tables already take --locales and --fallback (swiftlang/swift-foundation#2330). The same list could select the ICU data, and a dropped locale could fall back by the rules in utils/cldr-locale-fallback.swift, so the ICU and Swift paths agree.
Questions
- Is a locale list chosen when Foundation is built a direction you'd take now (a CMake option for the SDK, or the SwiftPM data build @icharleshu described in t/78399), or should it wait for more of the Swift data work?
- Who should choose the list: whoever builds the SDK, or the app? The app choosing needs the data outside the library, which @compnerd was wary of in t/78399.
- What should a dropped locale fall back to: a locale the app picks (like
--fallback), or root?
The rebuild script, logs and per-language sizes are here: swift-foundation-icu/locale-subset at locale-subset · niktishakov/swift-foundation-icu · GitHub. I can turn it into an ICUExport step if this is the way to go.