It goes in your Package.swift, where you define your .target. You can pass the swiftSettings parameter.
I don't know what the cause of this issue is or whether it's fixed in Swift 5.6. You can try the Swift 5.6 development snapshots on Linux to see.