1. CCSB354 ARTIFICIAL INTELLIGENCE (AI)
CHAPTER 12
NATURAL LANGUAGE PROCESSING
(NLP)
Textbook (Chapter 13, & especially pages 558 & 588)
Instructor: Alicia Tang Y. C.

2. Language is a complicated phenomenon, involving processes as varied as the recognition of sounds or printed letters, syntactic parsing, high-level semantic inferences,etc.

3. What is Natural Language Processing (NLP)?
Natural language gives computer users the ability to communication with the computer in their native language.
This technology allows for conversational type of interface. A general NLP system is not yet possible, especially in recognising and interpreting written sentences.

4. Types
Natural Language Understanding: Investigate methods of allowing the computer to comprehend instructions given in ordinary English.
Natural Language Generation: Strives to have computers produce ordinary English language so that people can understand computers easily.

5. NLP Software Tools
 INTELLECT
 SPOCK
 BBN Parlane
 NaturalLink

6.  Parser  Lexicon  Understander  Knowledge base  Generator
Component of NLP
 Parser
 Lexicon
 Understander
 Knowledge base
 Generator

7. Stages in Producing an Internal Representation of a Sentence

8. love kiss ?? lips jungle object agent object Contextual
Knowledge Interpretation
Expanded representation:
possess
pet:cheetah
experiencer
object
love
Person: tarzan
Person:jane
agent
kiss
object
instrument
lips
location
jungle
location ??

9. Block Diagram of a Natural Language Understanding Program of the Syntactic-Semantic Analysis Type
Input
text
String
Parser
Understander
Generator
Output
Lexicon
Knowledge
base

10. Block Diagram of a Computer Language Translation System
Generalised
Intermediate
Form
(GIF)
Rule-based
Processor
Input
Program
Parser
Rule-based
Processor
Lexicon
Output Program in
Target Language

11. NLP Database Expert Systems: where do they stand?

12. EASE OF COMMUNICATION WITH DIFFERENT TYPES OF INTEGRATED SYSTEMS
Oh no!
Well..not bad!
Oh yeah!!
EASE OF COMMUNICATION WITH DIFFERENT TYPES OF INTEGRATED SYSTEMS

13. Levels of analysis for natural language
Prosody
it deals with the rhythm and intonation of language
Phonology
it examines the sounds that are combined to form language.
Morphology
it concerns with the components that make up words.
E.g. the effect of prefixes (non-, un-) and suffixes (-ing, -ly)

14. Levels of analysis for natural language
Syntax
this involves the study of the rule for combining words to form legal sentences
Semantics
it considers the meaning of words, phrases and sentences
Pragmatics
this is the study of the ways in which language is used and its effects on the listener
World Knowledge
this includes knowledge of the ‘physical’ world, the world of our social interaction

15. Syntax Specification and parsing using Context-free Grammars
1. Sentence  noun-phrase verb-phrase
2. Noun-phrase  noun
3. Noun-phrase  article noun
4. Verb-phrase  verb
5. Verb-phrase  verb noun-phrase
6. Article  a
7. Article  the
8. Noun  man
9. Noun  dog
10. Verb  likes
11. Verb  bites
Specification and parsing using Context-free Grammars
the rules listed below define a GRAMMAR for simple transitive sentences

16. Syntax Rules 6. to 11. are terminals
1. Sentence  noun-phrase verb-phrase
2. Noun-phrase  noun
3. Noun-phrase  article noun
4. Verb-phrase  verb
5. Verb-phrase  verb noun-phrase
6. Article  a
7. Article  the
8. Noun  man
9. Noun  dog
10. Verb  likes
11. Verb  bites
Rules 6. to 11. are terminals
they define a lexicon for the language
terms that describe high level linguistic are called nonterminals (sentence, verb, noun-phrase, etc)

17. Syntax
A derivation of the sentence “the man bites the dog” (syntactically correct but semantically wrong, we shall see it later on)
String Apply Rule No.
sentence
noun-phrase verb-phrase 3
article noun verb-phrase 7
the noun verb-phrase 8
the man verb-phrase 5
the man verb noun-phrase 11
the man bites noun-phrase 3
the man bites article noun
the man bites the noun
the man bites the dog done
Parsing algorithms fall
into two classes:
1. Bottom-up
2. Top-down

18. Draw a parse tree for the sentence “the man bites the dog”

19. Top-down Parsing Parse Tree S Unrecognised I/P the man bites the dogVP A N
the
man V
bites
the
dog

20. Bottom-Up Parsing Parse Tree the Unrecognised I/P
the man bites the dog
man bites the dog
bites the dog
the dog
dog A
man N NP
bites V
the A
dog N NP VP S

21. Transition Network Parsers
A transition network parser represents grammars as a set of finite-state machines (i.e. transition networks), like this:
Noun-phrase
Verb-phrase S S
initial
final
Example: sentence (S)

22. Transition Network Parsers
Example: Noun-phrase
Noun
Article S S
initial
final
Noun
Example: Article
Read page 563, text a S S
initial
final
the

23. The Chomsky Hierarchy and Context-Sensitive Grammars
A context free grammar allows rule to have a single nonterminal on their left-hand side.
Context-free grammar is not powerful enough to represent rules of natural language syntax
the context-sensitive languages form a proper superset of the context-free counterpart

24. The Chomsky Hierarchy and Context-Sensitive Grammars
Here, one or more symbols on the left-hand side of a rule are allowed that makes it possible to define a context in which that rule can be applied.
This ensures satisfaction of a global constraints such as number agreement and other semantic checks.
The semantic error in earlier example could be detected in context-sensitive grammar if a non- terminal, act_of_biting is added to the grammar, preventing the sentence “man bites dog” to be valid.

25. The Chomsky Hierarchy
To correctly express grammatical structure of a language (e.g. English), rules are needed.
We can classify grammars according to the kinds of rules that appear in it.
Having done that, we can classify the language into families according to the kinds of rules that are needed to express its grammars.
One such means of classifying grammars in this manner is called Chomsky Hierarchy

26. Grammar Hierarchy The Grammars for the language Type 0 Type 1 Type 2

27. Grammar Hierarchy Type 0 Name of Grammar: Transformation Grammar
Form of Rules: anything  anything
Computational Power: General Turing Machine
String Characteristics: Any form

28. Grammar Hierarchy Type 1 Crossing dependencies E.g. Ali1 helps4 Ahmad2
Name of Grammar: Context-sensitive
Form of Rules: A B C  A D C
Computational Power: Linear Bound Automata
String Characteristics: an bn cn
Crossing dependencies
E.g. Ali1 helps4 Ahmad2
to teach5 Aida3 programming6.
a1 a2 a3 b3 b2 b3

29. E.g. Ali1 who studies in UNITEN2 that offers2 quality programmes
Type 2
Name of Grammar: Context-free
Form of Rules: A  B C D ….
Computational Power: Push Down Stack Automata
String Characteristics: an bn 2 2 2 1 1 1 1 1 1
Nested dependencies
E.g. Ali1 who studies in UNITEN2
that offers2 quality programmes
is1 graduating
a1 a2 a b3 b2 b1

30. They have been used for grammars of Morphology
Type 3
Name of Grammar: Right Linear
Form of Rules: A  x B (x in terminal category)
Computational Power: Finite State Automata
String Characteristics: a* b*
They have been used for grammars of Morphology

31. NATURAL LANGUAGE PROCESSING
Problems with NLP :
Ambiguity
multiple word meanings
the pitcher is angry
the pitcher is empty
Inaccuracy
Incompleteness

32. How people overcome natural language problems?
Imprecision
I’ve been waiting for you for a long time.
The king ruled the kingdom for a long time.
Unclear antecedents
Ben hits Bill because the sympathized with mary.
How people overcome natural language problems?
Context Familiarity Expectations

33. SPEECH RECOGNITION Advantages: Context Analysing Speech
most natural method
ease of access
speed
manual freedom
remote access
Context
Isolated Word Recognition (IWR)
Connected Word Recognition (CWR)
Continuous Speech Recognition (CSR)
Analysing Speech
Syllable Phonemes Allophones

34. Speech Component: A Holistic Look
I/O Processing Other data
Speech
recognition
Match with other
sound frequencies
Freq. spectrum
She likes ice-cream
Word Sentence
Syntactic
Analysis
Grammar of
language
(dictionary)
Semantic
Analysis
Meaning of each
word (thesaurus)
She like ice-cream
Sentence Structure
x. Likes(x,ice-cream)
Partial Meaning
Pragmatics
Context of
utterance
likes(siti, ice-cream).
Full Sentence Meaning

35. Exercises:

36. Question: Consider the grammar defined by the following BNF:
S  NP VP
NP  <name> | <det> <noun> |PP
VP  <verb> | <verb> NP NP | <verb> NP PP | VP NP
PP  <prep> NP
<name>  “Ben" | “Ann"
<noun>  "morning" | “ice-cream"
<verb>  "gave" | "saw"
<det>  "the"
<prep>  "to" | "in"
S – SENTENCE
NP – NOUN PHRASE
VP – VERB PHRASE
DET – DETERMINER
PP - PREPOSITION

37. Answers: a) Ben gave Ann the ice-cream.
Draw parse trees in this grammar for the following sentences.
a) Ben gave Ann the ice-cream.
b)   Ann gave the ice-cream to Ben.
S  NP VP
NP  <name> | <det> <noun> |PP
VP  <verb> | <verb> NP NP | <verb> NP PP | VP NP
PP  <prep> NP
<name>  “Ben" | “Ann"
<noun>  "morning" | “ice-cream"
<verb>  "gave" | "saw"
<det>  "the"
<prep>  "to" | "in"
Answers: