1. 25 September 2002BU CAS CS520 1 Parsing Combinators Hongwei Xi Boston University

2. 25 September 2002BU CAS CS520 2 Calculator: grammar (I) expr ::= integer | (expr) | ~expr | expr + expr | expr – expr | expr  expr | expr / expr This grammar is ambiguous: For instance, the expression 1 + 2 / 3 has more than one parse trees.

3. 25 September 2002BU CAS CS520 3 Calculator: grammar (II) expr ::= term | expr + term | expr – term term ::= factor | term  factor | term / factor factor ::= integer | (expr) | ~factor We have now resolved ambiguity in the previous grammar

4. 25 September 2002BU CAS CS520 4 A Calculator: grammar (III) expr ::= term expr’ expr’ ::= /* empty */ | + term expr’ | – term expr’ term ::= factor term’ term’ ::= /* empty */ |  factor term’ | / factor term’ factor ::= integer | (expr) | ~factor

5. 25 September 2002BU CAS CS520 5 Calculator: parsing and computing fun expr () = $term && $expr’ wth (fn (t, k) => k(t)) and expr’ () = PLUS >> $term && $expr’ wth (fn (t, k) => (fn t’ => k (t’ + t)) || MINUS >> $term && $expr’ wth (fn (t, k) => (fn t’ => k (t’ - t)) || succeed (fn t => t) … …

6. 25 September 2002BU CAS CS520 6 Parsing Combinators (I) type pos = Pos.T ‘t stream = (‘t, pos) Stream.T (‘a, ‘t) Parser = pos * ‘t stream -> ‘a * pos * pos * ‘t stream infix 4 >> << infix 3 && infix 2 -- ## infix 2 wth suchthat return guard infixr 1 || exception Fail of pos

7. 25 September 2002BU CAS CS520 7 Parsing Combinators (II) fun succeed x (pos, ts) = (x, pos, pos, ts) fun fail (pos, ts) = raise (Fail pos) fun done x (pos, ts) = if S.isEmpty ts then (x, pos, pos, ts) else fail (Pos.rightEdge pos, ts) fun any (pos, ts) = if S.isEmpty ts then fail (pos, ts) else let val ((x, pos), ts) = Stream.uncons ts in (x, pos, pos, ts) end

8. 25 September 2002BU CAS CS520 8 Parsing Combinator: -- fun (p -- q) (pos, ts) = let val (x, posx, pos, ts) = p (pos, ts) val (y, posy, pos, ts) = q x (pos, ts) in (y, Pos.union (posx, posy), pos, ts) end

9. 25 September 2002BU CAS CS520 9 Parsing Combinator: ## fun (p ## q) (pos, ts) = p (pos, x) handle Fail err1 => q err1 (pos, ts) handle Fail err2 => raise (Fail (Pos.max (err1, err2)))

10. 25 September 2002BU CAS CS520 10 Parsing Combinators: && and || fun p && q = p -- (fn x => q -- (fn y => succeed (x, y))) fun p || q = p ## (fn _ => q)

11. 25 September 2002BU CAS CS520 11 Parsing Combinator: !! fun !! p (pos, ts) = let val (x, posx, pos, ts) = p (pos, ts) in ((x, posx), posx, pos, ts) end

12. 25 September 2002BU CAS CS520 12 More Parsing Combinators fun p wth f = p -- (succeed o f) fun p suchthat g = p -- (fn x => if g x then succeed x else fail) fun p return x = p -- (fn _ => succeed x) fun p guard g = p ## (fn err => let val _ = g err in fail end) fun satisfy g = any suchthat g fun literal t = satisfy (fn t’ => t = t’) fun opt p = p wth Some || succeed None fun p q return x) fun p >> q = p -- (fn _ => q) … …

13. 25 September 2002BU CAS CS520 13 A Parser for Simple Types (I) signature PARSER = sig datatype tp = TpInt | TpBool TpFun of tp * tp val parseString: string -> tp end

14. 25 September 2002BU CAS CS520 14 A Parser for Simple Types (II) structure Parser :> PARSER = struct … … val tpInt = Token.litWord “int” val tpBool = Token.litWord “bool” val LPAREN = Token.litWord “(“ val RPAREN = Token.litWord “)” val ARROW = Token.litWord “->” fun tp () = $tp0 && $tp’ wth (fn (t, k) => k(t)) and tp’ () = ARROW >> $tp wth fn t => (fn t’ => TpFun (t’, t)) || succeed (fn t => t) and tp0 () = tpInt return TpInt || tpBool return TpBool || LPAREN >> $tp << RPAREN … … end

15. 25 September 2002BU CAS CS520 15 Try it yourself! A parser for untyped -calculus implemented using parsing combinators can be found on the course’s homepage Also, some helpful material can be found in Lawrence Paulson’s book on ML; or read a paper by Hutton Try it yourself and ask questions!