This, or something like this, has come up a few times before. As I recall, being able to express "types conforming to T but not U" in a generic context would make type inference undecidable (and Turing complete to boot).
Swift's type system is already Turing complete, but negation would make it Turing complete even if the existing system constraints were resolved.