Doesn't a class with only let properties follow value semantics?
No because those constants could be other objects that can mutate. As long the the whole object never mutates than the object has value semantics (e.g. UIColor).
That depends on how value semantics is defined and it actually hasn't been precisely defined yet. Informally most people use a definition that accepts immutable classes as having value semantics.
However, I think there is a good case to be made that value semantics implies locality of reasoning. The possibility of weak and unowned references to any instance of any class in Swift means that really isn't a way to guarantee locality of reasoning about code that works with class instances. We will need to consider this fact very carefully if / when support for value semantics is added.
It's also worth mentioning that when you dig into the details it becomes quite difficult to specify value semantics precisely at the type level rather than the operation level. I spent quite a bit of time trying to do this last summer and was unsuccessful.