As rog points out, if you don't want to watch the screen, you're not
using a visual editor.

hjkl does solve the hand motion issue, but a strong reason for that
mapping was ambiguity between tty escape sequences, and vi(1)
commands. Basically it couldn't tell the difference between the
user pressing escape then A, and a cursor key generating the very
same sequence.

It tried timing out after escape waiting for another character, but
at 9600 baud, users often generated ambiguous sequences within
the timeout. It would never work.

As a result, all users learnt hjkl for their own sanity, and configured
their termcaps not to tell vi(1) about cursor sequences just in case
it was stupid enough to try to tell the difference.