Is it possible to separate the syntax extension from the code
generation part? syntax extension is always fragile, code generation
API is still pretty useful. In my opinion, the main meat lies in code
generation part.

Not sure, but even if, I guess it would not be easy.

<:sexp< type t = bla bla >> maybe more robust, but it requires
camlp4of as the basic preprocessor instead of camlp4o

If there were only one preprocessor (sexplib) this would work, but having more than one (e.g. also bin-prot) does not seem straightforward. It's not clear how multiple and independently developed code generators can be composed, especially safely without interfering with each other.