This problem is well-understood, but the solution isn't to drop the Sendable requirement in the current language, it's to make it so that the compiler can infer that it isn't needed at all. See [Pitch] Safely sending non-`Sendable` values across isolation domains for an example of how the compiler might grow this capability.
I agree with you that this usage pattern is probably safe, so long as we are excluding behaviours that are unsafe in async code anyway, but I'd still be reluctant to remove the annotation at this time.