1. 1 Polynomial Time Probabilistic Learning of a Subclass of Linear Languages with Queries Yasuhiro TAJIMA, Yoshiyuki KOTANI Tokyo Univ. of Agri. & Tech.

2. 2 This talk… Probabilistic learning algorithm of a subclass of linear languages with membership queries learning via queries + special examples → Probabilistic learning Use translation algorithms representative sample → random examples equivalence query → random examples

3. 3 Motivations A simple deterministic grammar (SDG) has at most one rule for every pair of ⇒ learning algorithm for SDG from membership queries representative sample ⇒ for linear languages CFLs SDLs Regular Linear (Tajima et al. 2004)

4. 4 Linear grammar A context-free grammar is a linear grammar if every rule is of the form : nonterminal : terminal Any linear grammar can be written in RL-linear s.t. if every rule is of the form and

5. 5 has only left linear rules ( or right linear rules) Strict-deterministic linear grammar An RL-linear is a Strict-det linear if, for any pair of rules or Ex) for some a,B,c,D

6. 6 Deterministic linear grammar A linear grammar is deterministic linear (DL) if every rule is of the form or and Theorem : Theorem(de la Higuera, Oncina 2002) : DL : identifiable in the limit from polynomial time and data

7. 7 MAT learning (Angluin1987) learner hypothesis teacher target language membership query counter example yes or no hypothesis equivalence query

8. 8 PAC learning (Valiant 1984) PAC : Probabilistic Approximate Correct : probability distribution target concept example learning algorithm hypothesis is PAC

9. 9 If a hypothesis is consistent with Equivalence query ⇒ PAC learning algorithm (Angluin[1987]) × × If there is a consistent hypothesis ⇒ PAC learnable examples consistent with examples

10. 10 Probabilistic learning with queries Learning algorithm Example oracle target language Membership query Yes or No hypothesis

11. 11 Representative sample for a Strict-det : Strict-det : representative sample (RS) for some All rules are used to generate Q

12. 12 Example : then is a representative sample (RS)

13. 13 Rule occurring probability : a target grammar : a probability distribution on for an example : error parameter : confidential parameter : the size of target grammar’s rules For every rule, define

14. 14 is a rule occurring probability s.t. appears in the derivation of an example is an probability that and is used in the derivation

15. 15 Let Suppose The set of m-examples contains a set of RS with the probability Proof: “Any rule doesn’t appear in derivations of m-examples” occurs RS

16. 16 We can conclude that 1.Equivalence query can be replaced by random examples 2.Representative sample can be replaced by random examples

17. 17 example oracle membership oracle learning algorithm membership query equivalence query representative sample query response nega m-random examples posi n-random examples probabilistic learning algorithm with queries consistency check

18. 18 Learning algorithm via queries and RS while (finish == 0) begin make nonterminals from make rules and hypothesis if (equivalence query for responds “yes”) output, finish = 1 else update by the counterexample end

19. 19 Making nonterminals then : a nonterminal = an equivalence class contains (u,v,w)

20. 20 Making rules Make all rules as follows except for not consistent with query results Select a hypothesis randomly

21. 21 a set of Strict-det (not bounded by a polynomial) SD Exact learning of strict-det Strict-det is polynomial time exact learnable via –membership queries, and –a representative samples (RS) c.f. [Angluin(1980)] for regular sets RS Possible rules The learning algorithm overview: SD Chose one randomly, Equivalence query SD The correct hypothesis Witnesses delete incorrect rule

22. 22 Conclusions Strict-det linear language can be probabilistic learnable with queries in polynomial time Future works Identification from polynomial time and data (teachability) RS → Correction queries

23. 23

24. 24 Theorem Strict-det linear languages are polynomial time probabilistic learnable with membership queries

25. 25 Simple Deterministic Languages Context-free grammar(CFG) in 2-standard Greibach normal form is Simple Deterministic Grammar (SDG) iff is unique for every and Simple Deterministic Language (SDL) is the generated language by a SDG

26. 26 Representative sample for an SDG : SDG : representative sample (RS) for some All rules are used to generate Q

27. 27 Example : then is a representative sample (RS)

28. 28 PAC learning Target language : Hypothesis language : A PAC learning algorithm outputs such that where Probability distribution : on (Valiant1984)

29. 29 Query learning of SDLs SDLs are polynomial time learnable via membership queries and a representative sample the learner the teacher membership query yes / no representative sample at the beginning representative sample : a special finite subset of (Tajima2000)

30. 30 Learning model the learnerthe teacher membership query yes / no representative sample at the beginning representative sample : a special finite subset of