We can't remove these conversions, because it would break source compatibility.
Note that gmm(pairFn) is ambiguous only with -swift-version 4, because we allow the splatting conversion in both directions. If I recall, this was due to an oversight in the old implementation.
In -swift-version 5, only one direction is allowed.
If you look at CSSimplify.cpp you'll see that now both conversions are explicitly codified because the new representation is quite different between the two arguments and single argument of tuple form.
It would be nice if ranking would prefer the version without the conversion if possible. Ranking is itself a bit of a mess right now, and needs some focused redesign.