The purpose of this post is to set the stage for a later discussion about TDOP parsing. This post will introduce top-down and bottom-up parsing, then combine the two methods to introduce left-corner parsing. Also, the top-down parsing language (TDPL) will be briefly mentioned as its semantics relate to TDOP.

Traditional Parsing Methods

Before getting into TDOP, it's important to have at least some background in non-TDOP parsing methods. This is because TDOP can be understood as a combination of several different parsing methods.

Parsing is a language acceptance problem. That is, a parser is a function that accepts or rejects a string. If a parser accepts a string then we say that string is in some language. The opposite is said of rejection. A string in this case means a sequence of zero or more symbols. In the English language, symbols are Latin/alphabetic characters. In the C programming language, symbols are reserved words, variables, literals, and punctuation (e.g. void, "foo", >, etc.).

Typically, a parser accepts the language generated by a context-free grammar (CFG). CFGs are a formalism for describing some languages. The following is an example CFG that generates simple arithmetic expressions:

E → "(" E ")"
E → A

A → M "+" A
A → M "-" A
A → "-" A
A → M

M → N "×" M
M → N "÷" M
M → N

N → "0"
N → "1"
  ⋮
N → "10"

Note: ignore the unusual placement of the parentheses and the right-associativity of the operators described by the grammar.

The name to the left of the → is called a variable or a non-terminal. Something in quotes is called a token, or terminal. Both terminals and non-terminals are considered symbols. Terminals can be thought of as the letters of one's language.

The → itself is a relation which says that non-terminal on the left-hand side can generate the language on the right-hand side. This combination is called a production.

Note: the rest of this article will focus on parsing strings from left-to-right. The following examples detailing various parsing methods assume that our parsers alway guess correctly. Finally, we assume that our grammars are ε-free. That is, the right-hand side of a production is never empty (with one exception).

Top-Down Parsing

As its name implies, top-down parsing proceeds top-down. In the case of the above expression grammar, the "top" starts off as E. The action of going "down" involves one of two things:

1. Replacing a non-terminal with something that it is related to (the right-hand side of →).
2. Consuming a terminal.

Right-hand sides of productions contain both terminals and non-terminals. In replacing a non-terminal with ones of its right-hand sides, we set up expectations about the structure of later parts of the string. For example, suppose we want to parse "(2 × 3)". Parsing will proceed as follows:

If—as a side-effect of parsing a string—one wanted to build a parse tree, then the order of constructing nodes in the parse tree would be as follows:

prev	reset	next

Top-Down Parsing Language

Brief mention needs to be given to the top-down parsing language (TDPL). The TDPL formalizes the behavior of many top-down parsers. A key difference between a TDPL grammar and a CFG is that productions are totally ordered in a TDPL grammar.

For example, if the productions of the above CFG were totally ordered according to their text order, then a parser cannot try the second production (E → A) without first failing to parse according to the first production (E → "(" E ")").

Bottom-Up Parsing

We can characterize top-down parsers as making "global" decisions. Their expectations about the future structure of the as-of-yet unseen parts of the string are evidence of this. On the other hand, bottom-up parsers operate "locally". That is, they make decisions based only on the structure of the part of the string that they have already seen.

The consequence of local decision making is that bottom-up parsers discover sub-structures of the parsed string before they discover super/structures. In theory, a bottom-up parser has no expectations about the remainder of the string to be parsed. In practice, common bottom-up parsers implicitly make use of top-down information.

Bottom-up parsers typically perform two main actions: shift and reduce.

1. Shifting is similar to consuming to the extent that our cursor into the string being parsed moves forward by one symbol. This is equivalent to removing the first symbol of the input string.
   
   Unlike top-down parsers, bottom-up parsers do not maintain a sequence of expectations. Instead, they operate on a partially parsed substring of the input string.
   
   Shifting involves taking the first symbol from remainder of the input string and appending it to the end of the partially parsed string.
2. Reducing operates on a suffix of the partially parsed string. A reduction involves taking a suffix of the partially parsed string, matching it against the right-hand side of a production, and then replacing it with the left-hand side of a production (non-terminal).

For example, suppose we want to parse "(2 × 3)". Parsing will proceed as follows:

If—as a side-effect of parsing a string—one wanted to build a parse tree, then the order of constructing nodes in the parse tree would be as follows:

prev	reset	next

Left-Corner Parsing

Left-corner parsing (LC) is a parsing technique that makes decisions based on top-down and bottom-up information.

In the case of the bottom-up parser above, it appears that we were lucky that the sequence of shifts and reductions ended up reducing the entire string to an E. Strictly speaking, the goal of the above bottom-up parser was exactly that: reduce a string to E. If our expression were very long, then it wouldn't be clear until near the end of a bottom-up parse that our parser might have a chance of reaching its goal of E.

An LC parser attempts to satisfy multiple goals, including the end goal of reducing the string to E. An LC parser predicts substructures present in the remainder of the string, and attempts to parse those sub-structures bottom-up. But the prediction step sets up expectations about the structure of unseen parts of the string, which is a top-down approach.

In fact, LC parsers alternate between bottom-up and top-down parsing. Alternation is possible because an LC parser maintains a list of goals (analogous to our top-down expectations), a list of predictions, and a partial parse of the input string (as in a bottom-up parser). An LC parser operates on its input string and these three lists in the following way:

1. Repeat:
   1. If the head of the goal list is a terminal, then consume the terminal and shift the first symbol of the remainder of the input string onto the end of the partial parse. If the goal terminal does not match the first symbol of the string then reject.
      
      If the head of the goal list is a non-terminal, then attempt to reduce a suffix of the partial parse to the to the goal non-terminal. If such a reduction is possible, then remove the non-terminal from the head of goal list and update the partial parse accordingly.
      
      This step is repeated until the goal list remains unchanged.
   2. If β is the last symbol of the partial parse, then find a production of the form "α → β γ" where γ is a string of zero-or-more symbols. β is said to be a left corner of α. Left corners can be both terminals and non-terminals. If we weren't restricting ourselves to ε-free CFGs, then left corners do not necessarily appear immediately following the "→"!
      
      Place γ and α on the head of the goal list, so that the first symbol (if any) of γ is our next goal.
      
      If the goals list is changed then return to the step 1.1.
   3. If neither of the previous two steps changed the goals list, then shift a symbol from the remainder of the input string onto the end of the partial parse.
      
      If no such symbol can be shifted, then reject the string. Otherwise, return to step 1.2.
2. Stop when the goal list is empty.

For example, suppose we want to parse "(2 × 3)". Parsing will proceed as follows:

If—as a side-effect of parsing a string—one wanted to build a parse tree, then the order of constructing nodes in the parse tree would be as follows:

prev	reset	next

Compared to the other two methods, this seems like a lot of work for nothing! Also, there is some amount of magic happening: recall that we are operating under the assumption that every action taken will be the correct one. In practice, one constructs a table and "cheats" when deciding which actions to take.

Summary

Top-down and bottom-up parsing were covered to set the stage for left-corner parsing and the TDPL, which provide context for the behavior of TDOP parsers. My next post will go into TDOP and how it relates to left-corner parsing and the TDPL.

Optimizations

So far I have implemented the following optimizations. This post will focus on the last optimization, symbolic interpretation (labeled EVAL).

CP

Copy propagation

CF

Constant folding (with local constant propagation)

LICM

Loop-invariant code motion

DCE

Dead code elimination (with unreachable code elimination, block merging, and local constant de-duplication)

CSE

Common subexpression elimination

EVAL

Symbolic interpretation (based on abstract interpretation)

These optimizations were arranged into the following pipeline, where dashed edges are followed when a pass changes something and solid edges are followed when no changes are made:

SimpleSUIF

This project uses Stanford's SimpleSUIF compiler infrastructure. SimpleSUIF's intermediate representation (IR) is a linked list of instructions, including such things as basic arithmetic, bitwise operators, memory/constant load/store, and calling/branching operations. The IR is register based, with three register classes: machine, pseudo, and temporary. For our purposes, machine registers are never used. Temporary registers represent single-definition and single-use registers, where both the definition and use (if any) must reside in the same basic block. Temporary registers often hold loaded constants. Pseudo registers behave like general purpose registers. Finally, all registers are typed.

One quirk of how we use SimpleSUIF is that there is no apparent way to access the IR for an arbitrary function within the same compilation unit. As such, interprocedural optimizations such as function inlining and compile-time execution are not possible. This was unfortunate as there was one particular test case that would have benefitted from interprocedural optimization.

Test case

Below is one of the functions in the test case of interest. Two lines are striked out because the dead code elimination optimization pass regards them as useless.

float f1(float b, float c){
   int i;
   float j, k;

   j = c;
   for(i = 0; i < 2; i++) {
      k = b * i;
      j += k;
   }

   return k;
}

Looking closely at this example, it is clear that only the initialization of i to 0, the last iteration of the loop, and the value of b are important to the output of f1. However, this is difficult to tell from the perspective of the IR without running through the program. With more information (e.g. about loop induction variables or loop dependencies), we might be able to make smarter decision, but only in some really restricted cases. Unfortunately, it's not clear how one should go about "executing" this program in the absence of a particular value for b. This is where symbolic interpretation comes in.

Symbolic interpretation

Symbolic interpretation is similar to local value numbering in that we operate on concrete and symbolic values. For simplicity, I restricted this optimization pass to a subset of the provably pure functions. Because information about other functions was absent, I considered a pure function to be any function that does not:

• Load from or store to a memory location.
• Call any functions. Note: this constraint can be relaxed in the case of a recursive function call. The test cases I focused on did not include recursive function calls; however, this method can easily be extended to apply to that case.
• Copy from one memory location to another memory location.

Thus, a function is considered pure if it depends only on constants, local variables, and function arguments, and performs no operation that could generate a side-effect.

The following control-flow graph (does not include some edges because I am lazy with SVG) is an interactive symbolic executor of the SimpleSUIF-like IR representing the above function. Below I describe how each step of the evaluator is performed.

The symbolic interpreter behaves similarly to something that performs a combination of constant folding and constant propagation, with the exception that when an operation is performed on an expression containing a symbol, a new symbol is generated.

For example, if one performs an ldc operation to load the constant 0 into register t6, then we can assign to t6 the value 0. If a copy (cpy) operation is performed, then the value of the right-hand register is assigned to be the new value of the left-hand register. For example, cpy r3 = t6 assigns to r3 the value 0.

Sometimes a register is used before it is defined. For example, r1 in mul t8 = r1, r3 is never defined in the above code. This is because r1 represents one of the arguments to the function. In this case, r1 is given a new symbolic value that is distinct from every other symbolic value. In the above simulator, the symbolic value assigned to r1 is named r1. The purpose of being able to identify the "origin" of a symbol value will be useful for code generation.

When a symbolic value participates in an expression, as in mul t8 = r1, r3, a new and unique symbolic value is generated that represents the expression. If any of the components of the expression are constants (known at compile time) then we want to store those constants as part of the symbolic expression. For example, in the first iteration of the loop, t8 is assigned the symbolic expression r1 * 0. In the second iteration of the loop, t8 is assigned the symbolic expression r1 * 1.

Something not touched on in this example is a branch that depends on a symbolic value. In this case, we cannot follow the branch as we don't know in which direction it will go at runtime. We are concerned with cases in which we can statically determine the direction of the branch.

Code Generation

The focus of symbolic evaluation has been to end up with some symbolic or constant expression for each register. In fact, for this optimization, only the returned register (r5) ends up being useful. If the returned register contained a constant value then the function is necessarily constant, and so the function's code can be replaced with a ldc followed by a ret.

In the case that the returned register is a symbolic expression, we can walk the expression tree and output for each subexpression the instructions needed to compute that subexpression. The leaves of the expression tree will be symbolic register values (named according to their register) or constants.

Using the above expression tree walking strategy, the symbolic expression of r5 can be converted to the following sequence of instructions:

ldc t1 = 1
mul t2 = r1, t1
ret t2

Here we have generated new registers to hold temporaries, but left symbolic registers alone. This new sequence of instructions takes the place of the old, larger sequence of instruction.

Often I would pop in to Sheng's office on the third floor of Middlesex College at The University of Western Ontario. On his desks were towers of books and papers; it baffled me that they never fell. In his office, we would talk--sometimes for hours--about his past students and what they were up to, about parsing techniques, finite automata, regular languages and their operations, and object-oriented programming.

I prefaced each of our e-mail correspondences with the far too formal "Prof. Sheng Yu". Goodbye Prof. Sheng Yu; you will be missed.

Well, I've finally submitted my undergraduate thesis project's final report. My project was to develop the newest version of Grail+. Grail+ is a set of command line tools for manipulating non-deterministic finite automata (ε-NFAs), non-deterministic pushdown automata (ε-NPDAs), and context-free grammars (CFGs). Grail+ is built on top of the Formal Language Template Library (FLTL), a library that I developed for representing and symbolically manipulating CFGs, ε-NFAs, and ε-NPDAs. Over the past several months I've worked hard and built Grail+ and the FLTL from the ground up. Together, they represent around 12,000 lines of C++.

My report can be found here. The report is 49 pages long. For anyone reading this blog, the most interesting part of the report is the implementation discussion. Unfortunately, I had to leave a lot out of the report as it is already quite long. As such, the API described in the report is incomplete and some of the interesting discussions were cut short.

I have licensed Grail+ under the MIT License. I am interested in collaborating with others to continue the development of the project. The source code of Grail+ can be found here.