Thinking out loud:
Most VFD's that I've dealt with were largely text displays - normally you'd see an 8051 or similar driving a mess of shift registers. Essentially, one would jam an ascii byte over GPIO and toggle a latch to update the display. I'm assuming it will be somewhat similar for this display, even though it supports bitmaps.
With that in mind, I'd be tempted to write this completely in userspace, likely as an fs that overlays /dev/draw. It might be more hassle than it's worth, but it would be rather fun to run the fs and fire up games/catclock as a PoC.
Cheers,
Steve