TAS Must Die, Chapter 10

I started cleaning up some cruft I left behind in the code. Details, nothing worth mentioning.

Now I use the lexers I generate in two different places. One use is by P1, the parser that produces lexers as its output. P1 interprets common regular expressions and thus uses a lot of operators:

variable: [a-z]([a-z]|[0-9])*

for example. [, -, ], |, (, ), and * are all operators of some sort. Now sometimes you'll want to interpret one of these characters as a common boring character, not as an operator. In this case, I escape it with a backslash:

copyright: \(c\)

for example.

The other use of the lexer is just to read a bunch of iscript. I don't need a lot of escape-type processing. But it would be nice if the lexer ignored whitespace, devoured extraneous new lines, and handled windows newlines.

I've tried a few times to create a class that did both tasks well. It was a failure. They are pretty different. So I tried creating a base class that did very little except implement a pushback stack so I could unread tokens. It's helpful sometimes. I then subclassed two different TokenReaders, onefor the parser and it's crazy operators, and one for more mundane iscript processing.

My result, while it works, is not satisfying. It's clunky and brittle.

So combining it all was terrible, subclassing was unsatisfying. Within the next few nights I'll implement a Decorator design pattern. The gist of this is that you have two classes that implement some interface. You wrap one object in with another. You query the latter in your application. It invokes the wrapped object as necessary, and applies some type of transformation or filter.