(Resurrecting this thread because I stumbled upon it while rereading
A Formalized Interpreter, and I believe I have something new to add.)
As I understand it, using `□A` to mean "syntax for A", an infinitary type theory has rules like `(A → □B) → □(A → B)`. (Note that this is exactly what HOAS does, which explains why it's so easy to write an interpreter for HOAS without running into the semisimplicial types coherence issues.)
> Are there other more serious problems with an infinitary type theory?
I think the answer to this is "it depends". In
"An Order-Theoretic Analysis of Universe Polymorphism", Favonia, Carlo Angiuli, and Reed Mullanix have a consistency proof for a system with rational-indexed universes (and no explicit universe level quantification). However, infinitary rules give access to internal universe quantification (consider the function `λ i. "Type"ᵢ`). I believe HOAS-like internal-level quantification rules out any "fractal-like" scheme of universes (such as the rationals), because we can write an interpreter for "terms using universes i with 0 <= i <= 1" into terms that use universes between 0 and 2 (because we have enough universes to do that), and then we can embed terms with universes between 0 and 2 back into terms with universes between 0 and 1 (divide universe indices by 2), and this loop gives inconsistency by Gödel / Löb.
So at the very least, having infinitary limits rules out some of the more exotic universe level structures.