Ah, you're using Swift 5.4! The code you're reading in IntegerParsing.swift was rewritten (by me) in March, so you'll need macOS 12/Swift 5.5 to see the results I posted. The previous implementation was indeed over 10 times slower for Substring, one of the reasons I rewrote it.
My main preoccupation during the rewrite was making sure code size didn't regress; improving the performance of Substring was so obviously happening that I neglected to include a benchmark. However, it'd probably be wise to add one to the standard library repository so that future revisions don't cause an unintended regression--that's a good starter issue I think. I'm happy to point you in the right direction if you're interested; if not, perhaps you could file a starter bug at bugs.swift.org?