That’s OK for parsing trusted packages on your local machine. But crawling thousands of packages would mean that a server would have to run all the untrusted Package.swift (on Linux there is not sandbox support, so your proposed idea would mean arbitrary code execution). Also, there are a lot of manifests out there with #if ... #endif blocks which cannot be parsed correctly.
Rust TOML package manifests are easily parsed without requiring to run them through any complex compiler. I seriously think that choosing Swift as a manifest language was a terrible mistake and I hope they realize that.