Just a quick note. Today I was able to bootstrap the lexer-generating parser. That is, the parser can now generate a lexer which it can use to generate an identical lexer. At one pass of the optimization phase, the lexer flies up to about 5,500 states then finally optimizes down to about 50. I have no idea why it's producing all those temporary states. The hand-coded lexer generator uses the same backend calls and doesn't seem to have any difficulties.
I suspect I'll be revisiting this at some inconvenient time.