U+FFFD � REPLACEMENT CHARACTER is ideal for this, yes—however it is entirely possible that a user might paste U+FFFD in from somewhere else.
If it is important to you that the character not be inputable, it is best to use a “noncharacter”, the full list of which is here: Q: Which code points are noncharacters?. It doesn’t matter which you pick (although probably don’t use U+FFFE or U+FFFF)—noncharacters are explicitly reserved for internal application usage, so you can assign them whatever meaning you want. In the extremely rare case that a user inputs the noncharacter you choose into the textfield, you should replace that character with U+FFFD to ensure the noncharacter retains your specialized usage.