Hello community,
I've written a proposal that builds on the previous Package Manager Resources proposal that was recently accepted to support localized resources. I'm planning to work on an implementation in parallel so people can play with it.
Please let me know what you think!
David.
“Localizations” and “locales” are not the same thing. Stick to the word “localization”.
developmentRegion should not have a default value. It will be wrong more often than it is right if users can get away with ignoring it.
However, does it even have a reason to exist? How will Foundation know about it unless SwiftPM starts managing an Info.plist file? (Which doesn’t seem to be part of the proposal, and for good reason.)
To simplify creating a
Localefor the new initializer parameter, thePackageDescriptionmodule will add static factory methods toLocalefor each currently valid identifier.
That list is literally infinite. You’ll need a better qualifier for which ones are worth a static helper.
Also, Foundation’s localization APIs can process arbitrary localizations, not just the ones it can provide metadata about. If something really is provably invalid, you can throw an error, but don’t throw just because you don’t recognize it. Even private use identifiers like he-Qnqd (Hebrew with Nikud) are perfectly valid.
Oh and localizations and locales also spell their identifiers differently. "en_US" from the proposal happens to be invalid. It should be "en-US".
(Locale uses POSIX locale identifiers, whereas the localization API uses IETF language tags.)
I've tried to be careful to use the word "locale" sparingly and only when it made sense. Can you pinpoint me to where you think it doesn't?
If it doesn't have a default value, we need to make it non-optional then, and I think it's a bad idea to force that on all the packages that don't have any localized resources.
I forgot to mention it in the proposal, but SwiftPM does need to generate the resource Bundle's Info.plist. That's how localization works in Bundles.
I was planning on adding them all (930). Do you see an issue with that?
Very good point. I'll remove that diagnostic.
I was purposefully sticking to the spelling of the Locale so that: Locale.en_US == Locale(identifier: "en_US").
The word “locale” should not be used here at all, because locales do not come into play at all. Open Xcode, create a new project and use the UI to create add a localization. Every multi‐component localization it suggests uses a hyphen, not an underscore. The resulting subdirectory name also has a hyphen, not an underscore. This is because they are IETF language tags, and they have absolutely nothing to do with Foundation’s Locale type or its POSIX identifiers. Many simple identifiers can be easily converted back and forth by swapping hyphens for underscores and vice versa, but more complex ones do not map at all.
To be abundantly clear, the underscored examples currently in the proposal are incompatible with Foundation, and will not always work properly with the macOS System Preferences. [Edit: I did some experimenting and it appears macOS attempts to handle both types of identifiers if I manually set them arbitrarily with the defaults command line tool. I presume this is for legacy or compatibility reasons. But both System Preferences and Xcode generate IETF‐style hyphenated identifiers. And the POSIX identifiers break down when they get more complicated, while the IETF ones work no matter what.]
The static constants are shown on an extension in the proposal. Is that supposed to be an extension to Foundation’s Locale type? If so, it is the wrong type for the job. SwiftPM should either roll its own Localization type, or just use strings like the Foundation API (such as here, here, or here).
Yeah, I guess that would be annoying for unlocalized packages. Then I propose making it optional, nil by default and then throw an error if the manifest forgets it but still tries to use resources.
Interesting. To what extent is Info.plist understood on other platforms? Handling of Info.plist probably needs to be flushed out more. A lot more goes into Info.plist than localization. How should the mix of user‐defined parts and SwiftPM‐defined parts be juggled?
In the most common scenario, SwiftPM will be making a resource bundle, and the client application will have its own development region specified, thereby overriding anything specified by the embedded bundle. It only comes into play in the rare case that the resource bundle does not support a localization used by the top level application, and then no option is really all that good.
Actually, I just looked at the source code, and it does not look like it comes into play at all (whether intentionally or by accident). Bundle’s preferredLocalizations(from:) calls CFBundleCopyPreferredLocalizationsFromArray, which forwards to _CFBundleCopyLocalizationsForPreferences, which forwards to _CFBundleCopyPreferredLanguagesInList, with devLang as simply NULL. That last function then ignores it and falls back to US English.
Since it does nothing anyway, can we drop the idea of a development region and punt it along with Info.plist until the point were SwiftPM is trying to handle top‐level bundled applications (if ever)? In a main bundle like that is the only case were CFBundleDevelopmentRegion would have any effect. (Finder might also use it as a fallback if the bundle had a localized display name, but (a) it is embedded inside another bundle and (b) SwiftPM isn’t attempting to support CFBundleDisplayName yet.)
I think that will only make the dangerous conflation even worse in developer’s minds. CoreFoundation’s own source code already testifies to how widespread the confusion is. Xcode’s GUI displays full English names of each of its suggestions, and those could be used if we really want constants. But to be honest, I’m not really convinced that 930 constants are worth their weight when the user needs to know the code anyway to create the directory.
I also assume the declaration will need to match the directory exactly. What happens if you have "en_US" in the manifest, but "en-US.lproj" in the file system? Or "ar" and "ara.lproj"? Or "cze" and "ces.lproj"? (All three pairs are semantically equal.) How much responsibility for such things does SwiftPM want to take on? If we just take a string, then we don’t have to worry about it and can let it be the user’s problem. But if the actual string identifier is hidden behind a constant name, it will make debugging such situations all the more confusing.
I understand now! Thanks for making it clear.
I will modify the proposal to use standard IETF Language tags.
I think it makes sense to give developmentRegion a type of LanguageTag, a struct that wraps a String. This allows users to specify custom language tags if they need to, but also allows us to attach static values for common language tags. In that regard, I think we should provide all ISO 639-1 and ISO 639-2 language codes for convenience. The list is not too long.
Makes sense.
I think you may be looking at the wrong API. The preferredLocalizations(from:) set of functions don't take into account the development region, but the Bundle.url(forResource:withExtension:) set of functions, which load resources, call CFBundleCopyResourceURL, which calls _CFBundleCopyFindResources, which calls _CFBundleCopyLanguageSearchListInDirectory, which ends up using the devLang as a fallback.
And SwiftPM resources are loaded through the Resources bundle, so we can generate an Info.plist there without messing with user-defined values.
I've updated the proposal in consequence.
When I find a new Swift Package, I normally look at the Package.swift first. I'd find it extremely helpful to know what the project's development region is.
Also making it required allows SwiftPM / Foundation to potentially provide the functionality of using it to set the Locale.current rather than using the host system (EDIT: during debug builds). Which would make testing much more reliable rather than having to make sure that I am setting the Locale even if I'm not testing a particular Locale case (like when testing on CI)
Thanks for looking into it. I was foolishly trying to answer at 3 a.m. Not only was my brain not at peak capacity, but reading my post now it looks like it came out much more tersely than I would have liked. For that I apologize.
Sure.
Would it be much trouble to move developmentRegion to the target level, or allow it to be overridden at the target level? I work with several Xcode projects where different components of the same project have different development regions. It would be a shame to have to split them into multiple separate packages.
(To those reading and thinking, Why?!?: They are related to language training, linguistic analysis and the like, and so naturally not all components have the same set of localizations available, to the point of being completely disjoint at times).
I re‐read the proposal and it looks good now. 
Glad to see this addressed!
I have mixed feelings about how much of this should be handled in the package manager.
Localization is an extremely complicated and difficult thing to handle. I don't think anyone would disagree that localizing resources should be possible, but a Package Manager seems like the wrong place for things like LanguageTag to be defined. If there's a need for better compile-time support for localization, I would expect it to belong in Foundation alongside the rest of Localization support to avoid maintenance/API overlap.
It's also concerning on the side of UX complexity. IME managing Localization well relies heavily on tooling. If there will be a specific folder/file structure required for this support (which, maybe should be left up to the user anyway?) I would want the ability to CRUD Localizations from swift package. That would mean the available tools wouldn't have to each re-implement those capabilities, lessening the chance of different/incompatible formats arising.
All that said, I don't know what it's like to work with Bundles-from-Packages yet. E.g. If Xcode's localization doesn't work with the current (accepted) implementation, this would probably become a higher priority (for me) and I would care less about the extra features. If it does work, I would prefer this proposal to function as a good starting point with the goal of a robust set of features to handle Localization well.
Anyway, thanks @hartbit for pushing this forward!
LanguageTag is just a small wrapper over String and exists only to improve the PackageDescription API: its not meant to be exhaustive. By the way, how would you imagine better compile-time support for localization in Foundation, which is a runtime library?
While I agree that tooling support for localized resource would be very useful, I don't think we should introduce it in the same proposal as this one. I think its better to take small steps.
Don't worry, I appreciate the feedback!
If we allow it to be defined at the target level, I think we should also keep it at the package level, and have the target level work as an override, as you suggest. Does anybody else have any options on this subject?
The main things I would like to have is constants/enums for Locales. Although the support a Locale is determined at runtime, most use cases are for known codes and it's easy to also have an option for a wildcard. Anytime
Top-of-my-head example (plenty of necessary info/functionality missing)
enum Locale {
case es(ESLocale) // Spanish
case other(languageCode: String, regionCode: String)
}
enum ESLocale {
case es // Spain
case mx // Mexico
case ar // Argentina
case other(regionCode: String)
}
Improvements outside of that are mainly related to wanting compile-time checks/info for Bundle resources.
I would really like SPM to be better at this kind of thing than Xcode. Thinking through this a bit more, it seems like a good starting point to add basic support to the manifest. I just hope there will be more done in the future to give it more utility.
As a big user of SwiftGen, I whole heartedly agree! But this is not specific to localizations: it makes sense for all resources. And I really think it makes more sense to bring it up in a follow-up proposal to give it center stage.
I've updated the proposal to add a few diagnostics and remove others. I've also finished an implementation that matches the current proposal: Localized resources by hartbit · Pull Request #2535 · apple/swift-package-manager · GitHub
I'll try to generate toolchains for people to try out.
Just a quick update to let everyone know that after an offline discussion I was convinced that providing a set of common language tags is not a great idea as its inherently biased. I've revised the proposal and implementation to remove them, and make LanguageTag conform to ExpressibleByStringLiteral (as it should always have been), and we're back to using IETF language tags.
I think I'm mostly +1 on this now.
My primary remaining critique is it seems like developmentRegion belongs in a target. That is where the resources are declared anyway and would allow different defaults in targets within the same package. That's especially useful for testTarget.
My lesser concern is it seems like developmentRegion is better expressed by defaultRegion or defaultLocalization. That's the definition in the documentation and IMO feels more descriptive of it's actual purpose. developmentRegion feels ambiguous to me and I had to look it up to understand how it's actually used. Also as far as I can tell it becomes the default at runtime, which means development is kind of misleading because it indicates it's ignored for "production". There may be legacy reasons the key should remain the same, but this is probably the best chance to change the API before it's set in stone.
The problem is that the developmentRegion in Xcode can only be defined at the project level. I'm worried that if we allow defining it at the target level, it will break the Xcode integration. Do you have more information on this @NeoNacho?
I agree with you, but developers on Apple platforms already know this value as the "Development Region". There's a potential confusion cost to call it differently.
The default entry in the template for each bundle has CFBundleDevelopmentRegion set to $(DEVELOPMENT_LANGUAGE) for convenient centralization in the GUI, but it is not uncommon to set the value to something else. This is often done even when the project is uniform, since it is far easier for scripts and linters to check or automate the .plist entry than it is for them to try to understand the .pbxproj format.