They do affect Swift code generation.
You can see this in action by building for wasm32-unknown-none-wasm triple in the embedded mode with the latest nightly toolchain. Architecture-specific Clang flags will affect Swift code generation. For example, -Xcc -mmultivalue will allow generated Wasm functions to return Swift tuples on Wasm stack as Wasm tuples, instead of storing the tuple in Wasm linear memory and returning an address it. Same for other flags, you can control whether your generated Wasm code contains SIMD or atomic instructions: Clang command line argument reference — Clang 18.0.0git documentation
This works the same way for other architectures. I'm only using Wasm as an example here as I've spent enough time looking at disassembly of produced Wasm binaries with different combinations of these flags.
-Xcc flags are handled by ClangImporter, but they are also passed to the LLVM context that produces machine code for given LLVM IR, whatever that IR was generated from: Swift or any of the C family languages.