Running swift on command line from downloaded toolchain fails

I would add a bug for this, but I'm not sure if I am doing this correctly.

If I download one of the macOS development snapshots, and then run the swift build using a full path to the swift exe from this snapshot in the terminal. I get the following errors. Interestingly it says each symbol is implemented in /usr/lib/swift/libswiftCore.dylib or another dylib in the same folder. These files don't exist on my computer.

objc[97138]: Class _TtCs27_KeyedEncodingContainerBase is implemented in both /usr/lib/swift/libswiftCore.dylib (0x7ff84d19d518) and /Library/Developer/Toolchains/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-25-a.xctoolchain/usr/lib/swift/macosx/libswiftCore.dylib (0x110797a68). One of the two will be used. Which one is undefined.
objc[97138]: Class _TtCs27_KeyedDecodingContainerBase is implemented in both /usr/lib/swift/libswiftCore.dylib (0x7ff84d19d6d0) and /Library/Developer/Toolchains/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-25-a.xctoolchain/usr/lib/swift/macosx/libswiftCore.dylib (0x110797c20). One of the two will be used. Which one is undefined.
objc[97138]: Class _TtCs19__EmptyArrayStorage is implemented in both /usr/lib/swift/libswiftCore.dylib (0x7ff84d19c170) and /Library/Developer/Toolchains/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-25-a.xctoolchain/usr/lib/swift/macosx/libswiftCore.dylib (0x1107875b8). One of the two will be used. Which one is undefined.
...
objc[97138]: Class _TtCs9MainActor is implemented in both /usr/lib/swift/libswift_Concurrency.dylib (0x7ffb4e744eb0) and /Library/Developer/Toolchains/swift-5.7-DEVELOPMENT-SNAPSHOT-2022-04-25-a.xctoolchain/usr/lib/swift/macosx/libswift_Concurrency.dylib (0x10f316298). One of the two will be used. Which one is undefined.
zsh: segmentation fault   build

I also tried adding the toolchain .../usr/bin to $PATH and just calling swift as described on the swift.org site. This gave me the same results.

The downloaded toolchain works fine in Xcode.

I have ascertained this is an issue just with 5.7 and trunk development snapshots, so have added a bug Running swift-build from downloaded toolchain outside of Xcode crashes · Issue #58658 · apple/swift · GitHub