Internal imports by default is very problematic for generated code

after working on an internal code generation framework for the past few months, i’ve experienced a lot of friction with the new SE-0409 Access Level on Imports feature, and the upcoming Swift 7 “Internal Imports by Default” mode.

specifically:

  1. when a module is public imported, and no public API uses symbols from that module, the compiler generates a warning.
  2. when a module internal imports a module that is public imported elsewhere in that module, the compiler also generates a warning.

this is really problematic for code generation tools, which cannot easily track what generated API in which file uses symbols from what module, and cannot really determine the appropriate import ACL to use to avoid compiler warnings.

the second behavior also means there is a lot of potential for “spooky action at a distance”, that is, if someone adds a public import somewhere in a module, it can cause a deluge of warnings to appear for all the internal imports that live in that module. if those internal imports are written by a code generation tool, this can be a very difficult refactoring task, all to clean up spurious compiler warnings.

6 Likes

Hey @taylorswift, IMO, those warnings are rightfully generated to ensure that (1) the user's codebase doesn't over-claim the access levels of imported dependencies, and (2) the user's codebase's claims about imported dependencies remain consistent. Does your code generation tool have a post-processing phase to clean up warnings like these?

no, it does not, and it would actually be near-impossible do perform this accurately without at the minimum compiling the project, reading the IndexStoreDB symbol resolutions for all the source tokens, and attributing them to their original modules.

there are no tools in existence today that can accurately lint Swift module imports, although there are a few that (falsely) claim to do so.

I'm a bit surprised that this would be a warning. In the following setup:

  • A.swift does public import M because A exposes types from M in its public APIs
  • B.swift does internal import M because B only exposes types from M in internal APIs used elsewhere in the module

What's wrong with that? The internal import in B seems like the right way for the author to say "I'd like to know in the form of a compiler error if I ever add anything to B that uses M in a public API".

Is this behavior different for library evolution vs. non-resilient because of the additional dependency pruning aspect? I know that you get warnings about inconsistent use of @_implementationOnly import vs regular import for that reason.

But the situation above being a warning feels like it's more aggressive than it needs to be.


EDIT: @taylorswift Do you have an isolated reproducer? I can't reproduce it myself, the following works without warnings:

// A.swift
public import Foundation
public func foo() -> Data { Data() }
// B.swift
internal import Foundation
internal func foo2() -> Data { Data() }
$ swiftc -version
swift-driver version: 1.127.14.1 Apple Swift version 6.2 (swiftlang-6.2.0.19.9 clang-1700.3.19.1)
Target: arm64-apple-macosx15.0

$ swiftc -parse-as-library -c -enable-experimental-feature AccessLevelOnImport -enable-upcoming-feature InternalImportsByDefault -enable-library-evolution *.swift
# ok

$ swiftc -parse-as-library -c -enable-experimental-feature AccessLevelOnImport -enable-upcoming-feature InternalImportsByDefault *.swift
# ok
8 Likes

Are you sure you're receiving warnings about conflicting internal import and public import? I'd only expect you to see warnings on bare import declarations (that are implicitly internal) after you introduce a public import.

sorry, it seems i was mistaken, what seems to have happened in our code base was somebody had added duplicate @_spi imports of the same module, in the same file (not a different file), without any explicit ACLs, and this output ended up in a module which had AccessLevelOnImport on but not InternalImportsByDefault, as we are in the process of incrementally adopting the latter.

i didn’t notice the duplicated module because it had the @_spi prefix, and our team has gotten into a bad habit of using the same naming scheme for @_spis as we do for our actual frameworks.

however, the first problem (warning on public import when not used by any public API) is still a challenge for code generation.

never mind, it turns out i wasn’t crazy, there really is a compiler error (not just a warning) for “inconsistent” import visibility between different files:

/Users/runner/actions-runner/_work/package-data-model/package-data-model/Local/OrdoInternals/Sources/Internals/Autogenerated/Autogenerated+Buffers/AggregateDefinition+Buffer.generated.swift:9:8: 
error: ambiguous implicit access level for import of 'Foundation'; it is imported as 'internal' elsewhere
import Foundation
       ^
internal 
/Users/runner/actions-runner/_work/package-data-model/package-data-model/Local/OrdoInternals/Sources/Internals/Autogenerated/Autogenerated+Buffers/AggregateDefinition+Buffer.generated.swift:9:8: 
note: silence these warnings by adopting the upcoming feature 'InternalImportsByDefault'
import Foundation
       ^
/Users/runner/actions-runner/_work/package-data-model/package-data-model/Local/OrdoInternals/Sources/Internals/Autogenerated/Autogenerated+TypeBridges/PropertyValue+UnionBridge.generated.swift:9:1: 
note: imported 'internal' here
internal import Foundation
^

the problem is we can’t actually enable InternalImportsByDefault yet for this module because, well, we’re in the process of migrating it so that we can enable it.

so it feels like we are missing an incremental migration path here.

right so the current situation is we have a (large, about 70% autogenerated module) that does not have InternalImportsByDefault enabled, but we want to enable it on our roadmap.

if i have a bare import (which is implicitly public import), and there exists an internal import in some other file, i get a compiler error, because InternalImportsByDefault is not enabled.

if i enable InternalImportsByDefault as the compiler asks, and adjust the code generator to emit public import instead of import in these sites, then i get tons and tons of Public import of 'FooModule' was not used in public declarations or inlinable code everywhere.

we cannot stay where we are, because of the ambiguous implicit access level compiler error. and we cannot move forward (e.g., adopt InternalImportsByDefault), because of the tsunami of compiler warnings. so we basically cannot use internal import at all with code generation.

Am I missing something or does it seem that you may need to scan sources for how a model is imported and replicate that…? Unless I’m misreading what the error is about.

Maybe I am missing something but why don't you emit internal import once you turn on the InternalImportsByDefault feature. In general, if you could reduce this into a simple reproducer repo that would help a lot.

because if the generated code exposes any public API that uses symbols from that module, then it will result in a compiler error. similarly, if no such public API is generated, it will result in a compiler warning.

the code generator consumes Swift code as input, and rewrites it, so it does not know the provenance of any of the symbols in the code it transforms. it cannot possibly know that from AST information alone.

i’m not sure what you mean by that?

1 Like

If I understand this right you basically have to match the import kind generated by the source generator to the import kind that the actual sources – if so, you’d need to scan all the modules sources to determine the import kind to use and then do that, no?