All I need were these steps. The first two should be obvious but they tripped me up.
Make sure gcc AND g++ are installed.
unset CC
unzip musl-cross-make-master.zip
cd musl-cross-make-master
make TARGET=i686-linux-musl OUTPUT=$(pwd).install -j6 install
So this builds 32-bit executables that use musl libs. It kills two birds with one stone. That's great.
But it appears to expect musl libc to be installed in /lib for dynamically linked executables. That's not an option for me. Is there a way I can override this?