One thing I have mentioned previously in terms of finer grained access control is for Swift to clone and modify Rust's module system as "small modules", with a new access control level external.
That is add a mod concept which in some manner groups files together in a new unit.
All old "large" modules contain a single implicit "small" module.
New "large" modules can contain several "small" modules
This means changing the meaning of internal to be within the smallest containing mod is not source breaking (I think), thus external could be a new access control level which is either equivalent to the old internal (all files in the "large Module") or just all files in the/any parent mod.