Note: all of this behavior is handled behind a resilience barrier, so it can be kept up to date with any changes in the OS and ObjC conventions.
Hah, yes, tagged pointers can have associated objects. Pointer identity is derived from the bit pattern of the pointer itself, so that means that all "copies" of a tagged pointer string with the same contents have the same identity, and they all share the same associated objects. Since this identity is tied to just some bits rather than a managed memory address, such associated objects become effectively immortal.
When we bridge in tagged pointer NSStrings, we eagerly unpack them into our small string to hit all our fast-paths. When we bridge them back out to ObjC, e.g. to check for associated objects, we form tagged pointer NSStrings, whereupon they automatically regain their associated objects due to their identity-sharing.
I'm currently investigating this. It's certainly true that there will be cases where it's slower, but I think the situation is better than it might seem at first:
ASCII strings can be fast-pathed easily, and account for most things like OS-provided dictionary keys
Many significant chunks of non-ASCII text will be user facing attributed strings, which aren't currently Swift strings and aren't typically random-accessed like this anyway
Many of the rest are network content that's natively in UTF8, so is currently being transcoded into UTF16 first (if non-ASCII) by NSString
Most high performance use of NSString requires using bulk contents accessors anyway: -UTF8String, -cStringUsingEncoding:, -getCString:encoding:, CFStringGetCStringPtr, and various higher level APIs that use these internally. I'm in the process of making sure that the bulk accessors are as fast as possible. For example, relative to the initial UTF8 String landing, my tests show -UTF8String as ~14x faster and -getCString:encoding: as ~7x faster when passed NSASCIIStringEncoding or NSUTF8StringEncoding. In a lot of cases that should be beating pre-UTF8 Swift String too
A more challenging problem is the tendency of NSString APIs to take a UTF16 NSRange with the assumption that creating a 0..<length NSRange for the "just do the whole string" case is a very cheap operation. If that appears to be a significant problem in practice we'll certainly be open to talking with the Foundation folks about ways to improve the situation.
If you have specific examples of String<->NSString interop that are important to you that are regressed (or were already slow), please feel free to ping me about them. There's quite a bit of stuff I'm still investigating, but it's always good to be aware of the landscape of remaining problems.
PCMPxSTRx is no longer faster than equivalent "simple" vector instruction sequences for straightforward comparisons (this had already been the case for a few years when that article was written, which is curious). It can be used productively (with some care) for some other operations like substring matching, but that's not as much of a heavy-hitter. There's a bunch of string stuff that will benefit from general vectorization, and which is absolutely on our roadmap to tackle, but using the PCMPxSTRx instructions specifically isn't a source of wins on the most important operations.
Optimizing validation is something that we’ll get to eventually, but in the Swift model it only ever happens once, and the cost is linear, so there are some other optimizations that are higher-priority (and yes, I know Daniel).
