The swift-tools-version annotation is used to determine the version of PackageDescription library that should be used to evaluate the manifest file. We need this information before the manifest can be evaluated. This feature allows SwiftPM to evolve the PackageDescription library without breaking the existing package ecosystem. For e.g., if you update your package's tools version, your clients on older tools version will continue to use the latest version of the older manifest until they update their tools. Also note that this feature is different from version-specific manifest file which determines which manifest to use for a particular (marketing) version of Swift. We never really had a feature that allows using newer manifest features without breaking compatibility for older clients.
I like the #if compiler idea but we would need a way for manifests to declare a list of compatible tools versions so older tools can figure out which PackageDescription version to use if they don't have the latest one. Maybe, something like:
// swift-tools-version: 4.2, 5.0