In this case, you're right that there are fewer comparisons to do. If there was a type annotation on the left side, the compiler would indeed need to make sure that the type inferred on the right is convertible to the type in the type annotation.
Leaving out the type annotation isn't always faster, though. For example, when there are overloads involved, a type annotation can help the compiler find the correct overload faster, which allows it to prune subsequent overloads that obviously won't work and avoid extra checking.