Sticking with some form of Borrow here is...fine. If we do so, I'd prefer Borrowed by analogy with Optional, as to me it just sounds more natural to say that you've got "a borrowed foo" rather than "a borrow foo." (I'd stick with MutableBorrowed rather than MutablyBorrowed though; it's fine to say that something is a mutable, borrowed foo.) I did check the dictionary and (excluding an unrelated and very strange definition) it appears there's no real English language precedent for nouning or adjectiving this particular verb.
I must admit, though, I can't subscribe to your two objections to Ref. Indeed, it would seem that they're mutually exclusive: as Swift doesn't promise that it'll never stack promote a value of a reference type, your first concern would apply also to the "very important concept" we already have that's the core of your second concern.
As to the first objection on its own: Semantically, I do see this type as an indirection, and I think we ought to teach it that way, in no small part because I can't see clear to a tidier way of explaining what a Mutable(Ref|Borrow) would be—both the title of this proposal and the analogy that I really like about being a MutableSpanOfOne reinforce the interpretation. That its immutable counterpart may not be implemented with an actual indirection behind the scenes is (a) not a behavior that we'd want users to rely on—as I understand it, whether there's actual indirection or not relies on type layout that isn't itself guaranteed in the surface language and could vary by architecture, etc.; and (b) of a kind with how Swift optimizes notional semantics generally: for instance, values are notionally copied all over the place without actual copies; it's admittedly not great for beginner learning, but by the time that users will be reaching for this type, the idea that there could be notional indirection without actual indirection should be old hat.
As to the second objection: It seems to me actively useful rather than wrongheaded to draw parallels between var x: MutableRef<SomeValueType> and var y: SomeReferenceType. They're clearly not the same thing, but you can also tell at a glance that they're not straining to pretend to be; rather, the semantics rhyme to an extent that parallels how much the terminology echoes each other. I kind of like that a user armed only with the knowledge of how value and reference types differ from each other and who knows that, for example, Int is a value type can start to get some purchase on what a MutableRef<Int> is and might do just by composing that knowledge.