1. Parser and Scanner Generation: An Introduction
Using SLK and Flex++
Copyright © Curt Hill

2. Introduction
There are a variety of parser generators and scanner generators
The standards for UNIX seem to be lex and yacc
Yacc seems to have replaced something earlier
The GNU versions are flex and bison
We will use flex++ which is somewhat more parameterizable
We will also use SLK which seems to accept a better set of languages
Copyright © Curt Hill

3. Warning This process is complicated
This presentation overviews the process
This is followed by more detailed coverage of Flex++ and SLK
Neither program is really easy
Copyright © Curt Hill

4. Process 1 Craft the BNF Feed this into your parser generator
The parser generator should generate several files
Some of these will compile into the later pieces of the parser
Others will give information to help you debug your grammar
One should give you the reserved words of the language
Copyright © Curt Hill

5. Debugging Grammar?
Typically what happens is that some construct was left out
The non-terminal is never on the left-hand side of a production
When this is the case the parser generator declares it to be a reserved word
Thus we look at the reserved words to see if they are all really valid
Copyright © Curt Hill

6. Makefile
Since we are likely to go through this process several times we use Make to create our executable
Unlike all previous makefile we will use Flex++ and SLK just as if they are compilers
Our source code will need to be newer than our SLK and Flex++ input
Copyright © Curt Hill

7. Tagging
SLK has a parameter that allows all files to be prefixed by some letters
This allows files with the same function but for different languages to be distinguished
For example if we were working on a C and a separate C++ parser we might use prefixes of C_ or CPP_
In this class we will use our initials
One parser generator will be quite enough
Copyright © Curt Hill

8. Process 2 Start creating the input to Flex++
As previously mentioned the parser generator generates a file that will list the terminals
In SLK this is the XXXKeywords.txt file or the XXXConstants.h
Where XXX would be your initials
Copyright © Curt Hill

9. Process 2 Design your token data type
I suggest a class
It should have an enumeration or integer that is the token type
It should have a string to represent identifiers or number tokens
It should record the line and column number where token was found
It should have the usual constructors
Possibly one with all of the above
Copyright © Curt Hill

10. Process 3 Start the scanner definition section
Name the scanner
Set whatever definitions make sense
The header define gets includes
Determine what the scanner will return:
Integer
Enumeration
Object – this is the recommended one
Copyright © Curt Hill

11. Process 4 Start the rules section section of the scanner generator
For each terminal create a rule
Punctuation and reserved words are easy
Numbers, names are somewhat harder
Comments hardest
Tokens should return an initialized object
Copyright © Curt Hill

12. Process 5
There are certain things that a parser generator cannot determine
These are often called semantic routines
One of these is what to do if there is an error in the input file
Next we design the error handling
This is the XXXError class
Copyright © Curt Hill

13. Process 6
Another thing that cannot be pre-determined by the parser generator is what to do for semantic action
BNF cannot:
Check types
Generate code
Instead, we specify a function that does something at various steps
The writing of these functions is what this step is about
Copyright © Curt Hill

14. Process 7 Once all this is done the parser should be makeable
Of course, at this step we need to have various legal and illegal programs to test the parser
This will require various refinements
Anywhere from the grammar to one of the semantic routines
Copyright © Curt Hill

15. Finally Next we consider the SLK and Flex++ programs in more detail
We will then look at a completed project
An assignment would be next
Are you excited yet?
Copyright © Curt Hill