Download presentation

Presentation is loading. Please wait.

Published byBeatrix James Modified over 4 years ago

1
Statistical Methods and Linguistics - Steven Abney 1998. 09. 24. Thur. POSTECH Computer Science NLP Lab 9425021 Shim Jun-Hyuk

2
2 CS730B - Statistical NLP Contents o Introduction o Linguistics Review under Statistical methods Language Acquisition Language Change Language Variation o Language Structure and Performance Language Property Grammaticality and Ambiguity v. Performance Non-Linguistic Factors for Performance Grammaticality and Acceptability Grammar and Computation The Frictionless Plane, Autonomy and Isolation Holy Grail

3
3 CS730B - Statistical NLP Contents o How Statistics Helps Disambiguation Degrees of Grammaticality Naturalness Structure Preferences Error Tolerance Learning on the Fly Lexical Acquisition o Objections Are Stochastic Methods only for engineers? Did not Chomsky debunk all this ages ago? o Conclusion

4
4 CS730B - Statistical NLP Introduction o Linguistics m Computation Linguistics Performance Practical Application little concerned with human language processing Rationale by the Statistical Method m Theoretical Linguistics Competence Theoretical Research with grammars and structures concerned with human language processing o Objectives m Theoretical Background of Statistical analyses m Review in the view of Linguistics m Importance of Weighted Grammar

5
5 CS730B - Statistical NLP 1. Linguistics Review under Statistical Models (1) o Objective m Linguistics Issues in terms of population of grammar m General population of grammar can be usefully examined by the Statistical Models o Language Acquisition (LA) m Probabilistic(stochastic) or weighted grammar in Children’s LA Process m Co-existence and decay in grammars m Algebraic(Non-stochastic) grammar as supplementation

6
6 CS730B - Statistical NLP 1. Linguistics Review under Statistical Models (2) o Language Change m Change in Probability of Language Construction EX) Rule, Parameter setting m Not “Abrupt”, but “Gradual” m Statistical Co-existence and Decay “ Adult monolingual speaker ” - finally the grammar is stochastic in community o Language Variance m Dialectology Arbitrary continuum of language made by geographic distance Contact Frequency and intelligibility m Typology EX) Language Feature, Conditional Probability distributions m Statistical Modeling using the stochastic grammar

7
7 CS730B - Statistical NLP 2. Language Structure and Performance (1) o Language m Algebraic Properties l Idealization - Adult monolingual Speaker l theoretical syntax - Linguistics Data l Structure judgments for competence m Statistical Properties l Stochastic Model - Performance data l adjustments on structure-judgement data for “performance effects” l grammaticality and ambiguity judgments about the sentences as opposed to structure

8
8 CS730B - Statistical NLP 2. Language Structure and Performance (2) o Grammaticality and Ambiguity v. Performance m Example The a are of I The cows are grazing in the meadow John saw Mary Ambiguity Problem under Grammatical structures m Genuine ambiguities and Spurious ambiguities Problem Is not ungrammatical but undesired analyses case1 - elided sentence case2 - rare Usage The Problem is how to identify the correct structure form the possible. Can be solved by the use of weighted grammars in computational linguistics

9
9 CS730B - Statistical NLP 2. Language Structure and Performance (3) o Non-Linguistic Factors for Performance m Perception is the problem of Performance and It needs Non-Linguistic Factors with Grammaticality m Grammaticality and Acceptability perceptions of grammaticality and Ambiguity - Performance data What is “ Performance data ” - find some choice of words and context to get a clear positive judgment (Acceptability) m Grammar and Computation The Problem how can we compute the linguistic data simply and absolutely Competence v. Computation m Autonomy of syntax - not same as isolation and not be reduced to semantics m Holy Grail The larger picture and ultimate goal of Generative linguistics is to make sense of language production, comprehension, acquisition, variation, and change

10
10 CS730B - Statistical NLP 3. How Statistics Helps (1) o Disambiguation ( 모호성 해소 ) m Describing an algorithm to compute the correct parse among the possible m correct parse - the parse that human perceive m various statistical methods exist m 예 ) “John walks” - Context-free grammar with weights of rules o Degrees of Grammaticality m Gradations of acceptability m Degrees of error in speech production m Measure of goodness is a global measure that combine the degrees of grammaticality with naturalness and structural preference m By parameter Estimation, we can get the measure of “ degrees of grammaticality”

11
11 CS730B - Statistical NLP 3. How Statistics Helps (2) o Naturalness m plausibility - in the sense of selectional preferences m collocational knowledge - “how do you say it” m statistical method are applied to collocations and selectional restrictions o Structural Preference m One of the parsing strategies m longest-match preference m make an important role in the dispreference for the structure o Error tolerance m Detecting the error in sentences and select the best analysis m Primary motivations for Shannon’s noisy channel model

12
12 CS730B - Statistical NLP 3. How Statistics Helps (3) o Learning on the Fly m much like the error correction m to admit a space of learning operations assigning a new part of speech to a word adding a new subcategorization frame to verb, etc o Lexical Acquisition m the absolute richness of natural language grammars and lexica m primary area of application for distributional and statistical approaches to acquisition m Example of distributional Approaches acquisition of Part-of-Speech Collocation selectional restriction and ETC.

13
13 CS730B - Statistical NLP 4. Objections to Statistical Methods o Are Stochastic Models only for Engineers? m Are the stochastic models practically always a stopgap approximation? m With a complex deterministic system and the initial conditions we can compute the state at all time m In fact, more insight and successful than identifying every deterministic factors o What Chomsky really proves? m syntactic Structures (1957) Chomsky : grammatical( s ) P n ( s ) > E no choice for “n” and “E” P n ( s ) : best n-th order approximation to English Shannon ’ s MM : grammatical( s ) lim (n oo) P n ( s ) > E n increase, then erroneously assigned non-zero probability decease m Handbook of Mathematical Psychology (1963)

14
14 CS730B - Statistical NLP 5. Conclusion o Statistical method m weighted grammars, distributional induction methods m relevant to Linguistics o Performance v. Competence m Performance is not a goal but a useful tool of Computational Linguistics m Competence is needed to understand the algebraic properties of language m Algebraic methods are inadequate for understanding the human language m The Age of Computational Linguistics using Statistical Technology

Similar presentations

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google