1. ELN – Natural Language Processing Giuseppe Attardi
Introduction to NLTK
ELN – Natural Language Processing
Giuseppe Attardi

2. Installing NLTK Download NLTK data Download and Install
Download NLTK data
>>> import nltk
>>> nltk.download()

3. NLTK

4. NLTK Suite of classes for several NLP tasks
Parsing, POS tagging, classifiers…
Several text processing utilities, corpora
Brown, Penn Treebank corpus…
Your data was divided into sentences using ‘punkt’

5. NLTK Text material Tools Resources Raw text Annotated Text
Part of speech taggers
Semantic analysis
Resources
WordNet, Treebanks

6. Linguistic Tasks Part of Speech Tagging Parsing Word Net
Named Entity Recognition
Information Retrieval
Sentiment Analysis
Document Clustering
Topic Segmentation
Authoring
Machine Translation
Summarization
Information Extraction
Spoken Dialog Systems
Natural Language Generation
Word Sense Disambiguation

7. Part of Speech Tagging
Task: Given a string of words, identify the parts of speech for each word.
A man walks into a bar.
Det Noun Verb Prep Det Noun

8. POS Tag Usage Surface level syntax. Primary operation Parsing
Word Sense Disambiguation
Semantic Role labeling
Segmentation
Discourse, Topic, Sentence

9. How to do it? Learn from Data. Annotated Data: A man walks into a bar.
Det Noun Verb Prep Det Noun
Unlabeled Data:
A man walks home.
The pitcher issued four walks.

10. POS probabilities Det Noun Verb Prep Adj A 0.9 0.1 man 0.6 0.2 walks
man
0.6
0.2
walks
0.8
into 1
bar
0.7
0.3

11. ‘import nltk’
You will need to import the necessary modules to create objects and call member functions
import ~ include objects from pre-built packages
FreqDist, ConditionalFreqDist are in nltk.probability
PlaintextCorpusReader is in nltk.corpus

12. Exercise 1. Run examples from Chapter 1 of NLTK book:

13. Exercise 2. Run examples from Chapter 3 of NLTK book