My understanding—and keep in mind that I'm not working on move-only stuff or coroutines—is that atomic anything will require compiler support when not explicitly working with pointers or global stored properties. Even with modify, Swift's formal model is still move-in/move-out with assumed/enforced exclusivity, not by-address. (modify gets it to move-in/move-out rather than copy-in/copy-out, but not all the way to by-address. That's still considered an optimization.)
@John_McCall would have the most definitive answer to this question.