From: Jonathan Fine on
Wes James wrote:
> On Wed, Feb 24, 2010 at 5:03 AM, Jonathan Fine <J.Fine(a)> wrote:
>> Hi
>> Does anyone know of a collection of regular expressions that will break a
>> TeX/LaTeX document into tokens? Assume that there is no verbatim or other
>> category code changes.
> I'm not sure how this does it, but it might help:

Thanks, Wes. I'm already using PlasTeX

It handles changes of category codes, which makes it over the top for
what I want to do. In addition it is a fairly large complex
application, and sadly it's not at all easy to use just a part of the
code base.

There's been more discussion of this thread on comp.text.tex (which is
where I set the follow-up to).