Trying to get a flat Set (actually just a Set<Int>)

Well, it is one of the three fastest ones. x3, x4 and x5 are about the same (around 0.07 s in a release build on my machine), they are faster than x1, x2 and x6. But as demonstrated here, the optimizability is highly context dependent, in ways that makes it very hard, if not impossible, to build an intuition for performance in current Swift.