1. Natural Language Processing (NLP)
Natural Language Processing (NLP) refers to AI method of communicating with an intelligent systems using a natural language such as English.
Processing of Natural Language is required when you want an intelligent system like robot to perform as per your instructions, when you want to hear decision from a dialogue based clinical expert system, etc.
The field of NLP involves making computers to perform useful tasks with the natural languages humans use. The input and output of an NLP system can be −
Speech
Written Text

2. Components of NLP
There are two components of NLP as given −
Natural Language Understanding (NLU)
Understanding involves the following tasks −
Mapping the given input in natural language into useful representations.
Analyzing different aspects of the language.
Natural Language Generation (NLG)- It is the process of producing meaningful phrases and sentences in the form of natural language from some internal representation.
It involves −Text planning − It includes retrieving the relevant content from knowledge base. Sentence planning − It includes choosing required words, forming meaningful phrases, setting tone of the sentence.Text Realization − It is mapping sentence plan into sentence structure.

3. Difficulties in NLU NL has an extremely rich form and structure.
It is very ambiguous. There can be different levels of ambiguity −
Lexical ambiguity − It is at very primitive level such as word-level.
For example, treating the word “board” as noun or verb?
Syntax Level ambiguity − A sentence can be parsed in different ways.
For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted a beetle that had red cap?
Referential ambiguity − Referring to something using pronouns. For example, Rima went to Gauri. She said, “I am tired.” − Exactly who is tired?
One input can mean different meanings.
Many inputs can mean the same thing.

4. NLP Terminology
Phonology − It is study of organizing sound systematically.
Morphology − It is a study of construction of words from primitive meaningful units.
Morpheme − It is primitive unit of meaning in a language.
Syntax − It refers to arranging words to make a sentence. It also involves determining the structural role of words in the sentence and in phrases.
Semantics − It is concerned with the meaning of words and how to combine words into meaningful phrases and sentences.
Pragmatics − It deals with using and understanding sentences in different situations and how the interpretation of the sentence is affected.
Discourse − It deals with how the immediately preceding sentence can affect the interpretation of the next sentence.
World Knowledge − It includes the general knowledge about the world.

5. Steps in NLP There are general five steps −
Lexical Analysis − It involves identifying and analyzing the structure of words. Lexicon of a language means the collection of words and phrases in a language. Lexical analysis is dividing the whole chunk of txt into paragraphs, sentences, and words.
Syntactic Analysis (Parsing) − It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words. The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.

6. Steps in NLP Cont..
Semantic Analysis − It draws the exact meaning or the dictionary meaning from the text. The text is checked for meaningfulness. It is done by mapping syntactic structures and objects in the task domain. The semantic analyzer disregards sentence such as “hot ice-cream”.
Discourse Integration − The meaning of any sentence depends upon the meaning of the sentence just before it. In addition, it also brings about the meaning of immediately succeeding sentence.
Pragmatic Analysis − During this, what was said is re-interpreted on what it actually meant. It involves deriving those aspects of language which require real world knowledge.

7. Steps in NLP Cont..

8. Implementation Aspects of Syntactic Analysis
There are a number of algorithms researchers have developed for syntactic analysis, but we consider only the following simple methods −
Context-Free Grammar
Top-Down Parser
Context-Free Grammar-It is the grammar that consists rules with a single symbol on the left-hand side of the rewrite rules. Let us create grammar to parse a sentence − “The bird pecks the grains”
Articles (DET) − a | an | the
Nouns − bird | birds | grain | grains
Noun Phrase (NP) − Article + Noun | Article + Adjective + Noun
= DET N | DET ADJ N
Verbs − pecks | pecking | pecked
Verb Phrase (VP) − NP V | V NP
Adjectives (ADJ) − beautiful | small | chirping

9. Implementation Aspects of Syntactic Analysis cont..
The parse tree breaks down the sentence into structured parts so that the computer can easily understand and process it. In order for the parsing algorithm to construct this parse tree, a set of rewrite rules, which describe what tree structures are legal, need to be constructed.
These rules say that a certain symbol may be expanded in the tree by a sequence of other symbols. According to first order logic rule, if there are two strings Noun Phrase (NP) and Verb Phrase (VP), then the string combined by NP followed by VP is a sentence. The rewrite rules for the sentence are as follows −

10. Implementation Aspects of Syntactic Analysis cont..
S → NP VP
NP → DET N | DET ADJ N
VP → V NP
Lexocon −
DET → a | the
ADJ → beautiful | perching
N → bird | birds | grain | grains
V → peck | pecks | pecking
The parse tree can be created as shown −

11. Implementation Aspects of Syntactic Analysis cont..

12. Implementation Aspects of Syntactic Analysis cont..
Now consider the above rewrite rules. Since V can be replaced by both, "peck" or "pecks", sentences such as "The bird peck the grains" with wrong subject-verb agreement are also permitted.
Merit − The simplest style of grammar, therefore widely used one.
Demerits
They are not highly precise. For example, “The grains peck the bird”, is a syntactically correct according to parser, but even if it makes no sense, parser takes it as a correct sentence.
To bring out high precision, multiple sets of grammar need to be prepared. It may require a completely different sets of rules for parsing singular and plural variations, passive sentences, etc., which can lead to creation of huge set of rules that are unmanageable.

13. Implementation Aspects of Syntactic Analysis cont..
Top-Down Parser-Here, the parser starts with the S symbol and attempts to rewrite it into a sequence of terminal symbols that matches the classes of the words in the input sentence until it consists entirely of terminal symbols.
These are then checked with the input sentence to see if it matched. If not, the process is started over again with a different set of rules. This is repeated until a specific rule is found which describes the structure of the sentence.
Merit − It is simple to implement.
Demerits
It is inefficient, as the search process has to be repeated if an error occurs.
Slow speed of working.

14. Parsing

15. Parsing
A grammar describes the strings of tokens that are syntactically legal in a PL
A recogniser simply accepts or rejects strings.
A generator produces sentences in the language described by the grammar
A parser construct a derivation or parse tree for a sentence (if possible)
Two common types of parsers:
bottom-up or data driven
top-down or hypothesis driven
A recursive descent parser is a way to implement a top-down parser that is particularly simple.

16. Top down vs. bottom up parsing
The parsing problem is to connect the root node S with the tree leaves, the input
Top-down parsers: starts constructing the parse tree at the top (root) of the parse tree and move down towards the leaves. Easy to implement by hand, but work with restricted grammars. examples:
Predictive parsers (e.g., LL(k))
Bottom-up parsers: build the nodes on the bottom of the parse tree first. Suitable for automatic parser generation, handle a larger class of grammars. examples:
shift-reduce parser (or LR(k) parsers)
Both are general techniques that can be made to work for all languages (but not all grammars!).

17. Top down vs. bottom up parsing
Both are general techniques that can be made to work for all languages (but not all grammars!).
Recall that a given language can be described by several grammars.
Both of these grammars describe the same language
E -> E + Num
E -> Num
E -> Num + E
E -> Num
The first one, with it’s left recursion, causes problems for top down parsers.
For a given parsing technique, we may have to transform the grammar to work with it.

18. Parsing complexity LL parsers LR parsers
How hard is the parsing task?
Parsing an arbitrary Context Free Grammar is O(n3), e.g., it can take time proportional the cube of the number of symbols in the input. This is bad! (why?)
If we constrain the grammar somewhat, we can always parse in linear time. This is good!
Linear-time parsing
LL parsers
Recognize LL grammar
Use a top-down strategy
LR parsers
Recognize LR grammar
Use a bottom-up strategy
LL(n) : Left to right, Leftmost derivation, look ahead at most n symbols.
LR(n) : Left to right, Right derivation, look ahead at most n symbols.
Both are used in real compilers (LR parsers are more common)

19. Top Down Parsing Methods
Simplest method is a full-backup, recursive descent parser
Often used for parsing simple languages
Write recursive recognizers (subroutines) for each grammar rule
If rules succeeds perform some action (i.e., build a tree node, emit code, etc.)
If rule fails, return failure. Caller may try another choice or fail
On failure it “backs up”

20. Top Down Parsing Methods: Problems
When going forward, the parser consumes tokens from the input, so what happens if we have to back up?
suggestions?
Algorithms that use backup tend to be, in general, inefficient
Grammar rules which are left-recursive lead to non-termination!

21. Recursive Decent Parsing: Example
For the grammar:
<term> -> <factor> {(*|/)<factor>}*
We could use the following recursive descent parsing subprogram (this one is written in C)
void term() {
factor(); /* parse first factor*/
while (next_token == ast_code ||
next_token == slash_code) {
lexical(); /* get next token */
factor(); /* parse next factor */ }

22. Problems Some grammars cause problems for top down parsers.
Top down parsers do not work with left-recursive grammars.
E.g., one with a rule like: E -> E + T
We can transform a left-recursive grammar into one which is not.
A top down grammar can limit backtracking if it only has one rule per non-terminal
The technique of rule factoring can be used to eliminate multiple rules for a non-terminal.

23. Bottom-up parsing Goal of parser : build a derivation
Top-down parser : build a derivation by working from the start symbol towards the input.
Builds parse tree from root to leaves
Builds leftmost derivation
Bottom-up parser : build a derivation by working from the input back toward the start symbol
Builds parse tree from leaves to root
Builds reverse rightmost derivation

24. Bottom-up parsing
The parser looks for a substring of the parse tree's frontier...
...that matches the rhs of a production and
...whose reduction to the non-terminal on the lhs represents on step along the reverse of a rightmost derivation
Such a substring is called a handle.
Important:
Not all substrings that match a rhs are handles.

25. Bottom-up parsing techniques
Shift-reduce parsing
Shift input symbols until a handle is found. Then, reduce the substring to the non-terminal on the lhs of the corresponding production.
Operator-precedence parsing
Based on shift-reduce parsing.
Identifies handles based on precedence rules.

26. Example: Shift-reduce parsing
STACK
ACTION $
Shift
Grammar:
$ id1
Reduce (rule 5)
1. S  E
2. E  E + E
3. E  E * E
4. E  num
5. E  id
$ E
Shift
$ E +
Shift
$ E + num
Reduce (rule 4)
$ E + E
Shift
$ E + E *
Shift
$ E + E * id2
Reduce (rule 5)
Input to parse:
id1 + num * id2
$ E + E * E
Reduce (rule 3)
$ E + E
Reduce (rule 2)
$ E
Reduce (rule 1)
Handles:
underlined
$ S
Accept

27. Shift-Reduce parsing A shift-reduce parser has 4 actions:
Shift -- next input symbol is shifted onto the stack
Reduce -- handle is at top of stack
pop handle
push appropriate lhs
Accept -- stop parsing & report success
Error -- call error reporting/recovery routine

28. Augmented Transition Networks
An augmented transition network (ATN) is a type of graph structure used in the operational definition of formal languages, used especially in parsing relatively complex natural languages, and having wide application in artificial intelligence. An ATN can, theoretically, analyze the structure of any sentence, however complicated.

29. Components of an ATN
ATNs build on the idea of using finite state machines to parse sentences. ATNs have states to mark the progress in processing a string. Transitions correspond to individual words or phrases from a syntactic type. W. A. Woods in "Transition Network Grammars for Natural Language Analysis" claims that by adding a recursive mechanism to a finite state model, parsing can be achieved much more efficiently. Instead of building an automaton for a particular sentence, a collection of transition graphs are built. A grammatically correct sentence is parsed by reaching a final state in any state graph. Transitions between these graphs are simply subroutine calls from one state to any initial state on any graph in the network. A sentence is determined to be grammatically correct if a final state is reached by the last word in the sentence.

30. Examples of ATNs NP verb S2 S1 S NP1 det NP noun pronoun NP2 name
The top ATN specifies the structure of a sentence.
The bottom ATN corresponds to its NP transition and would be used to process a noun phrase as subject of the sentence.

31. A more complex example of an ATN for a sentenceNP S S1 S4 NP
verb NP S2 S3
jump
jump
The jump transitions are for optional components
and consume no input.
ATN for parsing
Mary gave me a new picture
Mary sings.
etc

32. Parsing with an ATN
Set the ATN pointer to [S] and the source pointer to the 1st word of the sentence. The ATN pointer points to the current node.
Select an arc out of the current node which satisfies any associated test and, if the arc is labeled with a word category, the current word must be a member of that category.
Execute any actions associated with the arc and:
If the arc is a word category, update the current position in the source sentence by one word and change the current node to the destination of the arc
If the arc corresponds to another ATN, push the starting node of the ATN onto the ATN pointer
If the arc is jump, chage the current node to the destination of that arc
If the arc is done, pop the current node off of the ATN pointer and set * to the value returned by this node. If the ATN pointer is now empty and all of the text has been processed, return *. If the ATN pointer is empty and there is remaining text, fail. Otherwise, return to step 2.

33. Example parse : Mary gave me a new picture
ATN Pointer
Source Pointer
Variables and Values S
NP S
NP2 S S1 S2
NP S2
NP2 S2 S3
NP S3
NP1 S3
NP2 S3 S4
Mary
gave me a
new
picture
Name = Mary
* = (NP (Name Mary))
Verb = (V (Verb give))
Pronoun = me
OJB = (NP (Pronoun M))
DET = a
ADJS = (new)
NOUN = picture

34. A more complex example of an ATN for a sentenceNP S S1 S4 NP
verb NP S2 S3
jump
jump
The jump transitions are for optional components
and consume no input.
ATN for parsing
Mary gave me a new picture
Mary sings.
etc

35. Example parse : Mary gave me a new picture
ATN Pointer
Source Pointer
Variables and Values S
NP S
NP2 S S1 S2
NP S2
NP2 S2 S3
NP S3
NP1 S3
NP2 S3 S4
Mary
gave me a
new
picture
Name = Mary
* = (NP (Name Mary))
Verb = (V (Verb give))
Pronoun = me
OJB = (NP (Pronoun M))
DET = a
ADJS = (new)
NOUN = picture