I don't remember the exact location, but there have been several contributions from active members of the community who stated they would be fine with just two levels of access control — but I haven't seen this standpoint discussed at all.
Most likely, that is because it contradicts the perceived spirit on swift-evolution, but who knows, maybe this opinion isn't as isolated as we think?
To be honest, I don't expect that such a drastic simplification would be seriously considered, but I think it's a good exercise to take such an extreme perspective.
[If anyone would like to participate in this experiment, you may prefer to skip the rest for now to avoid bias ]
[personal results start here]
I don't need access control as a tool to enforce my opinion onto others, so I'm not generally opposed to a compiler-orientated model, where access limitation would only be used to allow better optimisation.
What I don't want to loose is a way to document intensions which helps users of an API — but this could be something different than a compilation error that tells me I'm trying to call a method that I'm not allowed to use.
In my day to day routine, the language level actually isn't that important:
When I work in an environment that is new to me, a method that is missing in autocompletion is as protected as a private member, and it's similar for overrides.
So I would be fine with a set of annotations that are ignored by the compiler, but used by the IDE. Actually, such an approach could have saved me from stupid errors in the past, because the compiler only knows if I'm allowed or forbidden to do something (anyone else ever had fun with UITableView.didDeselectRowAtIndexPath? Alphabetical order is a really inconvenient sometimes, and there is no way for the compiler to detect such problems reliably).