Introduction to Language Processing Technology Natawut Nupairoj, Ph.D. Department of Computer Engineering Chulalongkorn University

Outline Level of Programming Languages. Language Processors. Specification of Programming Languages.

swap(int v[], int k) { int temp; temp = v[k]; v[k] = v[k+1]; v[k+1] = temp; } swap: muli $2, $5, 4 add $2, $4, $2 lw $15, 0($2)... Assembler C Compiler Level of Programming Languages High level: C / Java / Pascal Low level: Assembly / Bytecode Machine Language

High-Level Language Characteristics Expressions: a = b + (c – d)/2; Data types:  Integer, character, boolean.  Record, array. Control structures:  Selective.  Iterative.

High-Level Language Characteristics Declarations:  Identifier can be constant, variable, procedure, function, and type. Abstraction:  Object-oriented concept.  Concern only what, not how. Encapsulation:  Object-oriented concept.  Information hiding.

Language Processors Program that manipulates programs express in some programming languages. Example:  Editor.  Translator / Compiler.  Interpreter.

Translator Translate a “source” program into an “equivalent” “object” program. Translator source program object program error messages C C++ FORTRAN Java VB Assembly C Bytecode p-code

Tombstone Diagrams Ordinary program Program P Written with Language L L P Java Sort x86 Sort x86 Web Browser

x86 Web Browser Tombstone Diagrams Machine M Machine M x86 SPARCx86 SPARC x86 Web Browser

Tombstone Diagrams Translator L S  T S is translated to T Translator is written with Language L C Java  x86 x86 Java  x86 C++ Java  C

Tombstone Diagrams Compilation x86 C  x86 x86 Sort C x86 Sort

Tombstone Diagrams Cross Compilation x86 C  SPARC x86 SPARC Sort SPARC Sort C

Tombstone Diagrams x86 Java  C x86 C  x86 x86 Two-stage compilation C Sort Java Sort x86 Sort

Tombstone Diagrams x86 C  x86 x86 Compiling a compiler C Pascal  x86 x86 Pascal  x86

Tombstone Diagrams Interpreter SLSL Interpret source S x86 Written in language L Basic x86 Basic x86 SQL SPARC Basic Sort

Tombstone Diagrams Abstract machine = hardware emulator  interpreter for low-level language. x86 C  x86 x C 370 x x86 = 370 HW1 370 HW1

Tombstone Diagrams Java  Portable environment: write-once-run-anywhere.  Interpretive compiler. M Java  JVM JVM M JVM = Bytecode

Tombstone Diagrams x86 JVM x86 SPARC JVM SPARC JVM Sort JVM Sort x86 Java  JVM x86 JVM Sort Java Sort

Tombstone Diagrams Bootstrapping  Compiler L that is written on L language. Full bootstrap  Start from nothing. Half bootstrap  Start from other machine. NNP C  NNP

Tombstone Diagrams Full Bootstrap NNP C sub C sub  NNP NNP C sub  NNP NNP C sub C  NNP NNP C  NNP NNP C sub  NNP NNP C sub  NNP NNP C sub  NNP

Tombstone Diagrams NNP C C  NNP NNP C  NNP NNP C  NNP

Tombstone Diagrams NNP C sub C sub  NNP NNP C sub  NNP NNP C sub C  NNP NNP C  NNP NNP C sub  NNP NNP C  NNP NNP C C  NNP

Tombstone Diagrams Half Bootstrap x86 C  x86 x86 C C  NNP x86 C  NNP x86 C  NNP x86 C C  NNP NNP C  NNP x86 C  X86 x86

Specification of Programming Language Specification  Syntax Define symbol and structure of the language. Grammar.  Contextual constraints Constraints beyond grammar. Rules of the language: scope rules, type rules, etc.  Semantics Meaning of program: its behaviors when run. How to translate a sentence S of the language L to a machine code on M

Syntax Context-free grammar  Terminals.  Non-terminals / Variables.  Start symbol.  Production rules. Usually being expressed with BNF notation.

BNF Notation Backus-Naur Form. Given production rule: N   N   Can be written as: N ::= 

Example: Mini-Triangle Program ! This is a comment. It continues to the end-of-line. let const m ~ 7; var n: Integer in begin n:= 2 * m * m; putint(n); end Terminals beginconstdoelseendif inletthenvarwhile ;::=~() +-*/<> =\

Mini-Triangle Syntax Program::=Command Command::=single-Command |Command ; single-Command single-Command ::=V-name := Expression |Identifier ( Expression ) |if Expression then single-Command else single-Command |while Expression do single-Command |let Declaration in single-Command |begin Command end

Mini-Triangle Syntax Expression::=primary-Expression |Expression Operator primary-Expression primary-Expression ::=Integer-Literal |V-name |Operator primary-Expression |( Expression ) V-name::=Identifier Declaration::=single-Declaration |Declaration ; single-Declaration single-Declaration ::=const Identifier ~ Expression |var Identifier : Type-denoter

Mini-Triangle Syntax Type-denoter::=Identifier Operator::=+ | - | * | / | | = | \ Identifier::=Letter | Identifier Letter |Identifier Digit Integer-Literal ::=Digit | Integer-Literal Digit Comment::=! Graphic* eol Letter::=a | b | … |z Digit::=0 | 1 | 2 | … | 9

Syntax Tree Ordered tree with  Internal nodes: non-terminals.  Leaf nodes: terminals.  N-tree of G is a syntax tree with N as the root.

Mini-Triangle Syntax Tree Expression ::= primary-Expression | Expression Operator primary-Expression primary-Expression ::= Integer-Literal | V-name | Operator primary-Expression |( Expression ) V-name ::= Identifier …