1 A Chart Parser for Analyzing Modern Standard Arabic Sentence Eman Othman Computer Science Dept., Institute of Statistical Studies and Research (ISSR), Cairo Univ. 5 Tharwat St., Orman, Giza, Egypt Khaled Shaalan Computer Science Dept., Faculty of Computers and Information, Cairo Univ. 5 Tharwat St., Orman, Giza, Egypt Ahmed Rafea Computer Science Dept., American University in Cairo 113, Sharia Kasr El-Aini, P.O. Box 2511, 11511, Cairo, Egypt.

2 Outline Why Arabic Is Difficult to Parse? Arabic Chart Parser Architecture Arabic Morphological Analyzer and Lexicon Arabic Unification Based Grammar The Proposed Chart Parser Conclusion

3 Why Arabic Is Difficult to Parse? Length of the sentence and the complex Arabic syntax Omission of diacritics (vowels) in written Arabic “altashkii” Free word order nature of Arabic sentence Presence of an elliptic personal pronoun “alDamiir almustatir”

4 Arabic Chart Parser Architecture Grammar rule Input sentence Stem with features Inflected word Stem with features Parse tree Stem Parser Lexicon Morphological Analyzer Arabic Grammar

5 Arabic Morphological Analyzer and Lexicon Morphology - using augmented transition network (ATN) technique. -The morphological analyzer consists of three modules: Analyzer module, lexical disambiguation module and features extraction module.

6 Prefix= ‘ ال ’, Suffix =’ ين ’, Stem= {noun (‘ طلب ’…), verb (‘ طلب ’…)} Prefix= ‘ال’, Suffix =’ين’, Stem= {noun ('طلب', undefined, male, single, no, [quiescence],[noun, irrational],[])} noun ('طلب', defined,male, dual_or_plural,no, [accusative_or_genitive], [noun,irrational ],[]) Analyzer noun ('طلب',undefined, male, single, no, [quiescence],[noun, irrational],[]). verb('طلب', neutral,past, [male],single,[accusative],trans_1_obj,[rational, neutral],['طلب']). noun ('طلب',undefined, male, single, no, [quiescence],[noun, irrational],[]). verb('طلب', neutral,past, [male],single,[accusative],trans_1_obj,[rational, neutral],['طلب']). Disambiguatio n Add features A Morph Example: الطلبين (alTalabayn) الطلبين

7 Lexicon Three morphological categories for Arabic words: noun, verb, and particle Two types of features in the lexicon: syntactic features that eliminate syntactic ambiguity lexical features that eliminate lexical ambiguity features are stored in the lexicon and can be modified during the sentence analysis No. of entries: 5000

8 The Verb Entry form: verb (Stem, Voice, Tense, [Subject Gender, Object Gender], Number, End case, Transitivity, [Subject rationality, Object Rationality], Infinitive) Syntactic features: Voice: passive / active Tense: past / present [Subject gender, Object Gender]: [Male/Female, Male/Female] Number: singular /dual / plural [End Case, Agent]: [accusative / nominative / genitive, subject / object / proagent], Transitivity: intransitive / transitive_1_obj / transitive_2_obj This feature gives the grammar the ability to predict the max number of agents expected after the verb that helps in distinguishing between passive and active voice of verbs that do not change their form in both cases.

9 The Verb Entry (Cont’d) Lexical features: [Subject Rationality, Object rationality]: [rational/irrational, rational/irrational] This feature helps in determining that an agent is the proagent for a verb but not its subject by comparing the rationality feature of this verb with the agent feature. Infinitive: [infinitive form] since we did not store the passive form of the verb in the lexicon this feature is needed when the morphology decides that the verb in the passive voice.

10 The Noun Entry form: noun(Stem,Definition,Gender,Number,Adjectivability,End case,[Category, Rationality], irregular plural) Syntactic features: Definition: defined/undefined/neutral Gender: male/female Number: Singular /dual / plural, End case: [indeclinable/quiescence/accusative/nominative/genitive, without_noon: to indicate that the noun does not take suffix “ ن ” in case of dual or plural which means that the noun must be in an annexation form] Irregular plural: [broken plural form of the irregular noun]

11 The Noun Entry (Cont’d) Lexical features: Adjectivability: yes: if we can get the adjective form by adding the suffix “ ى ” /no otherwise [Category, Rationality]: [category can be any noun type like: adjective, infinitive, demonstrative noun … etc., rational/irrational]. The category is needed because some noun types are not allowed to occur in a certain sentence position like the adjective in the position of subject.

12 The Particle Entry Particles: A particle has the following form: Particle (Stem, Category). The only feature represented here is the Category: preposition, conjunction…etc.

13 A UBG for Arabic Implemented using SICStus Prolog 3.10 Each grammar rule has the form rule(LHS,RHS):- constrains Constraints used to: 1. Ensure the agreement bet. LHS and RHS 2. Reduce the syntactic ambiguity. 3. Reduce the semantic ambiguity

14 The Grammar Arabic sentence Either nominal sentence or verbal sentence Either simple or compound. the simple sentence does not have a complementary that could occur at the end of the sentence. No. of rules: 170 The rules are collected into 22 group. Each one represents a grammatical category such as: Object, Subject, Defined, conjunction form, substitution form etc. This categorization is designed in such a way it helps in maintaining the grammar in a modular way

15 A Grammar Rule Example The following rule defines the verbal sentence that contains only verb phrase. This verb phrase could be just a verb or a verb preceded by a particle. Some constraints should be satisfied in order to apply this rule during the course of parsing the sentence: The verb must not be ditransitive (i.e. taking two objects): If the morphological analysis of the verb could not tell us whether it is in passive or active voice, we have to check the transitivity of the verb: If it is intransitive, the verb must be in active voice and the (Cat) feature which hold the information about the agent will be either connected or absent agent If the verb is transitive and takes only one object, it must be in passive voice and the pro-agent will be either connected or absent. Note that when the verb has the same lexical form in both the active and passive voices, like the verb “ زرع ”, we used semantics features to determine the correct voice.

16 Its implementation rule( simple_verbal_sentence(Stem, Time1,Gen,Num,Cat), [verb_phrase(Stem, Time,Gen,Num,Trans,_,Agent)]):- ((\+var(Agent),Agent\==[])-> Agent=[Sub|Obj];Sub=Agent), Trans\==trans_2_obj, (Time==neutral,\+var(Trans)-> (Trans==intrans-> Time1=active, (\+var(Obj),Obj\==[]-> Cat=connected_subject;Cat=Sub); Time1=passive, (\+var(Obj), Obj==[]-> Cat=Sub; Cat=connected_pro_agent)) ;( var(Trans)->true ; (Time==active-> Trans=intrans, Time1=active, (\+var(Obj),Obj\==[]-> Cat=connected_subject_objct; Cat=Sub); Trans=trans_1_obj,Time1=passive, (\+var(Obj),Obj\==[]-> Cat=connected_pro_agent_objct; Cat=Sub) ))).

17 The Proposed Chart Parser Combines the advantages of both top- down and bottom-up parsing algorithms. predictive & avoids any reduplication of work

18 Chart Parser Algorithm Initializing the chart For every rule of form root->C 1 C 2 ………. C k add an arc labeled root-> o C 1 C 2 ………. C k using the arc introduction algorithmintroduction algorithm Parsing Do until there is no input: If the agenda is empty look up the interpretation of the next word and add it to the agenda. Select a constitute C from the agenda. Using the arc extension algorithm combine C with every active arc on the chart. Any new constituent are add to the agenda.arc extension For any active arcs created in step 3 add them to the chart using the arc introduction algorithm

19 A Chart Parsing Example verbal_sentence(Stem,Time,Gen,Num,Cat) verb_phrase(Stem,Time,[S_Gen|O_Gen],Num,Trans,[ S_rat,O_rat],Agent) تروى START Parse(‘ تروى الارض جيدا ’, verbal_sentence) simple_verbal_sentence (Stem,Time,Gen,Num,Cat) agent(Stem,Cat,Age nt_type,Rat) circumstantial(Stem,Gen,Num) verb(Stem,Time,Tense,Gen, Num,[End_case Agent],Trans,Rat, infinitive) pro_agent(Stem,Gen,Rat) الارضجيدا defined(Stem,Gen,Num,End _case,[Cat,Rat])

20 Conclusion This paper has been concentrated on issues in the design and implementation of a chart parser for Arabic. We acquired the Arabic grammar rules of irab (‘ إعراب ’) and the effects of applying these rules to the constituents of the Arabic sentence. The grammar rules encode the syntactic and the semantic constrains that help in resolving the ambiguity of parsing Arabic sentences. This will have a positive impact on applications such as machine translation because the target sentence will be produced from a structure that represents the intended meaning of the source Arabic sentence.

21 Arc extension algorithm: To add a constituent C from position P1 to P2: IF there is any active arc of the form X->C o C...…... C k from position P0 to P1 then 1- add a new active arc X->C 1 …. C o ………. C k from position P0 to P2. 2-Insert C in the chart from position P1 to P2. 3-for any active arc of the form X->C 1 …. o C from position P0 to P1 add a new constituent of type X from P0 to P2 to the agenda else take next constituent in the agenda

22 Arc introduction algorithm To add an arc root->C 1 …. o C j ………. C k ending at position i: for each rule in the grammar of form C j -> X 1…….. X k recursively add the new arc C j -> o X 1…….. X k from position i to i.