[9fans] Re: protocol botch

[Russ Cox <russcox@gmail.com>]

The story is this.  I'm CC'ing 9fans so that it
ends up in the archives for anyone else
who has trouble.

Auth/keyfs serves the keys from /adm/keydb.
It consults nvram to learn the encryption key
for the database (it uses the p9sk1 key for
bootes to do this), but that is all.  Bootes has
its own entry in the key database that had better
have the same key as in the nvram.

Factotum initializes itself using nvram, either
pulling out the p9sk1 key or a secstore key (and
then grabbing a whole file of keys from secstore).

The p9sk1 key for bootes then lives in many places:
1. it is in the nvram on the auth server machine,
    used to encrypt the auth db.
2. it is listed in bootes's entry in the key db.
3. it is listed in the secstore file.
4. it is in the nvram of any machine that doesn't use secstore.
5. either via #3 or #4, it gets loaded into the server factotum.

If #1 is not right, auth/keyfs will not be able to decrypt the db.
It is up-to-you to keep #2 through #4 in sync.
Auth/debug checks that the password you type
into auth/debug matches what is in the auth db (#2 above).
It doesn't check the others.  In particular, it doesn't 
check what ends up in factotum, which would indirectly
check #3 and #4.  It could, but it doesn't.

In order for cpu to work between two machines, the 
factotums on the two machines (#5) must agree with the
auth server (#2) about the passwords in question.

The big-hammer approach to making this work is to 
go to the auth server and run "auth/changeuser bootes"
and "auth/changeuser glenda" and make extra
sure that you type the passwords correctly.
Then reset the keys in factotum using

echo 'key proto=p9sk1 user=whoever dom=my.dom.net \
    !password=my-password' >/mnt/factotum/ctl

on the terminal and the cpu server (setting the right user
and password combo on each).  Now you should have
reasonable confidence that the auth db and the two
factotums are in sync, and cpu should work.

The factotum debug logs you've been posting show that
all is well until factotum contacts the auth server, at which
point you get a protocol botch.  This protocol botch happens
because the factotum and the auth server are speaking at
each other using different encryption keys, so each appears
to be speaking random garbage to the other.

Looking at the logs you've posted, the first botch happens
when the client hands the ticket to the cpu server, suggesting
that the client and the auth server agree about the client
password, but the auth server and the cpu server do not
agree about the server password, so that would be the first
password I tried resetting.

Russ