Pitch: Unicode Equivalence for Swift Source

So it would be a mere bug fix then? Or was that not what Chris’ answer meant and the core team’s answer still pending?

@xwu, is there already some implementation associated with the previous broader discussions that could be used as a starting point to factor out the targeted bug fix? Or would it be better to start from scratch?

With the proviso that we'd want to be able to run an implementation through as much compatibility testing as we could, yes, consensus on the Core Team was that this would just be a bug-fix.


No the previous discussion pre-dated the need for an implementation.

Thanks for clearifying, I didn't know this.

So fixing this bug and switching to any NF* internally is a source breaking change, no matter which NF* is used. The fact that duplicated characters are the reduced to the same identifer should then be mentioned in Swift Book/Lexical Structure/Identifiers

The only instance where anything would be source‐breaking is if code is already trying to use equal text—such as (decomposed) and é (composed)—as two separate identifiers. The compiler would begin considering them equal. Such usage is highly unlikely in practice, no matter how exotic the human language of the developer.

Even if a user’s code has been written in one normalization form (or a random hodge‐podge) and the compiler decides to think internally in the other form, the user will see no observable difference. All the same identifiers still reference all the same things.

The issue is only on the ABI level. If an already compiled binary used one normalization form and the compiler starts mangling names only in the opposite normalization form, then no new module could link against any existing symbol whose name differs between the normalization forms. (Since ASCII is static across normalization, users of ASCII code would never notice.)


I am less familiar the compiler than I am the other parts of the project written in Swift. I have heard mumblings about ICU vs no ICU in the compiler and I don’t know what the actual reality is. Is there easy access to a string normalization function inside the compiler, or will it require modification to even get access to that ability?

(It may take some time before I get to this. If someone beats me to it, I won’t hold it against you. :wink:)

I think module names are affected by this too.

I just saw this other post...

...and experimented with variations of the user’s instructions.

Creating a project and entering the name in either NFD or NFC results in the same thing: Xcode will have NFD in its UI for the project settings and the directory and process name are NFD too. But the mangled class reference in the interface builder file is NFC. And right now they aren’t considered matching. I’m not an interface builder expert though, so there may be more going on in that case.

If we are to encourage the usage of Unicode identifiers, there are a few places where normalization (or the lack thereof) can pose a problem:

  • Interface Builder: outlets, actions, class names, because the Objective-C runtime does not normalize names.

  • Generated files for modules, libraries, frameworks: not all filesystems normalize Unicode and/or use Unicode-aware name lookups. Thinking specifically of Linux vs. macOS here. To counter that, build tools should normalize names before creating files, and later use the normalized name when reading the files it generated. (I haven't tested, but I presume this is a problem.)

  • String identifiers in Cocoa and elsewhere. For instance: notifications names. Swift will see two names as being the same, but NSNotificationCenter will not on platforms where comparison is done using NSString. Note that comparison is done using String in swift-corelib-foundation, not NSString, so results will probably not match Apple platforms.

Generally, anywhere a string is used as an identifier might be a problem if not all comparisons are normalized ones. Everyone should be testing for this, but I bet no one is.

My personal advice for programmers would be to stick to ASCII if possible when choosing identifiers because it is not vulnerable to normalization issues. It might be a long time before the entire ecosystem can handle things harmoniously, and you probably don't want subtle bugs to creep in because one API see two strings as identical while another doesn't.