Also as an aside its worth pointing out that this "specificity" is a global property. For a given expression, a solution is a self-consistent assignment of overload choices to each declaration reference in the expression. There might be several valid solutions for a single expression. The type checker picks a solution among the set of valid solutions by find a solution that ranks higher than all other solutions. When a highest-ranked solution does not exist, we diagnose an ambiguity error.
Solution ranking takes into account both information recorded during constraint solving (the "score") and actual properties of the solutions under comparison.
In particular, we don't look at each overload set in isolation to pick a most specific overload and take them together, because doing so could produce a solution that is not correct or not most specific under the rules of the language.
And Jordan is right that the rules need to be a) simplified b) carefully studied and c) written down. Not necessarily in that order :)