Presentation on theme: "The Possibility of the Application of Neural Test Theory to SLA Research Kenichi YAMAKAWA *1, Naoki, SUGINO *2, Yuko SHIMIZU *2, Hiromasa OHBA *3, & Michiko."— Presentation transcript:
The Possibility of the Application of Neural Test Theory to SLA Research Kenichi YAMAKAWA *1, Naoki, SUGINO *2, Yuko SHIMIZU *2, Hiromasa OHBA *3, & Michiko NAKANO *4 *1: Yasuda Womens University, *2: Ritsumeikan University, *3: Joetsu University of Education, *4: Waseda University AAAL 2010 Annual Conference: March 6, 2010, Sheraton Atlanta Hotel, Atlanta, GA *The present study is supported by Grants-in-Aid for Scientific Research 1348064 (2001-2003), 16320078 (2004-2006) and 19320091 (2007-2009) from the Japan Society for Promotion of Science.
This poster explores a possibility of using Neural Test Theory in SLA research. A grammaticality judgment task aiming to explore the acquisition of unaccusative and unergative verbs was administered to 369 Japanese EFL learners. It was found that the learners could be categorized into four latent ranks. Abstract summary
Continuous ability scale Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm Tests do not have enough resolution to continuously evaluate human ability. Tests are at best capable of ranking test takers into 5 to 20 ranks. It is difficult to clarify the relationship between scores on a continuous scale and ability specification. 87 pts 85pts Problems 30pts Test scores Students language ability on a continuous scale (= test scores)
NTT is a test theory designed for evaluating achievements on an ordinal scale. Each level is called a latent rank. Latent ranks are based on the response patterns of the test takers. The number of latent ranks is determined by the data analyst. Goodness-of-fit indices are used in this determination. (Note 1) What is NTT (Neural Test Theory) ? Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm
Ordinal scale based on NTT Based on http://antlers.rd.dnc.ac.jp/~shojima/ntt/index.htm Latent rank 1 Latent rank 2 Latent rank 5 Test scores It is easy to explain the relationship between scores and abilities because the individual abilities also change in stages. This increases the tests accountability, and helps to draw a can-do chart. Ability to do X1 Ability to do X2 Ability to do X3 Ability to do X4 Ability to do X5
To explore a possibility of using NTT in SLA research (Note 2) To divide learners into different latent ranks and to attempt to draw a can- do chart 1) Subjects --- university-level 369 Japanese EFL learners 2) Materials --- a grammaticality judgment task on unaccusative / unergative verbs (Note 3) (36 items: see below) (Note 4) 3) Procedures --- judgments made on a 5-point scale (1-5) converted into 1-4 points if ungrammatical if grammatical 0 point 4 points 1 point 3 points 2 points 2 points 3 points 1 point 4 points 0 point Method Purpose
Unaccusative verbs Unaccusative verbs (appear, arrive, die, exist, fall, happen ) Category A (NP+V: 6 items) e.g. Your letter arrived yesterday. Category C (*NP+be+p.p.: 6 items) e.g. *Because of the rain, the train was arrived late. Category E (*NP+V+NP: 6 items) e.g. *Finally the waitress arrived the salad to us. Unergative verbs (cry, dance, laugh, play, sing, work) Category B (NP+V: 6 items) e.g. Her father cried at her wedding ceremony. Category D (*NP+be+p.p.: 6 items) e.g. *He was cried when he heard of his mothers death. Category F (*NP+V+NP: 6 items) e.g. *The boy hit his little sister and cried her. Six categories (A-F) in the judgment task
The learners were best distinguished on an ordinal scale based on NTT when they were ranked into four levels. Characteristics of each latent rank were described with regard to their acquisition of unaccusative / unergative verbs. Some learners in higher ranks seem to regress on an acquisition path when they judge incorrect passive sentences in Categories C and D. Conclusion Rank 1 Rank 2 Rank 3 Rank 4 Rank 4: Ability to correctly judge Category D Rank 3: Ability to correctly judge Categories E and F Rank 2: Ability to correctly judge Category A Rank 1: Ability to correctly judge Category B Rank 4 Rank 3 D) * They were laughed when … Grammatical ? ?? C) * …,the accident was happened.
Developed by Dr. Kojiro SHOJIMA and his colleagues NTT is a nonparametric test theory based on the self-organizing map (SOM) and generative topographic map (GTM). First presented at the International Meeting of the Psychometric Society in 2007 (IMPS2007) in Tokyo The statistical software (Shojima, 2008) available at http://www.rd.dnc.ac.jp/~shojima/exmk/index.htm Selected publications on NTT: 1) About NTT: http://www.rd.dnc.ac.jp/~shojima/ntt/index.htm 2) Shojima, K. (2009). Neural test theory. In K. Shigemasu et al. (Eds.), New Trends in Psychometrics (pp.407- 416). Universal Academy Press. 3) Shojima, K. (2008). Neural test theory: A latent rank theory for analyzing test data. DNC Research Note, 08-01. (Note 1) More about NTT
The present joint research has been continuing for eight years with the purpose of exploring the overall development of grammatical competence of Japanese EFL learners. We developed several grammaticality judgment tasks on unaccusative/ unergative verbs, dative alternation, psych verbs, relative clauses, wh-questions, articles, in addition to three sets of standardized test of the English grammar and a test on cue dependency, all of which were conducted on university-level Japanese learners of English. The research findings on each task/test have been published elsewhere (Yamakawa et al., 2003; Ohba et al., 2006; Shimizu et al., 2006; Sugino et al., 2006; Nakano et al., 2007; Ohba et al., 2008). Based on the data we obtained, we employed IRT (Item Response Theory), and equated the data of 1,200 learners from different tasks/tests (Yamakawa et al., 2008). We obtained the distribution of the average values of all the grammatical categories on the b-parameter (difficulty parameter) scale. To our knowledge, this was a first attempt to apply the IRT over a range of grammatical features, and revealed many characteristics of the development of their grammatical competence. In addition, we also attempted to compare the results of grammaticality judgment tasks (unaccusative/unergative verbs and relative clauses) conducted on Japanese and Korean EFL learners (Ohba et al., 2009; Yamakawa et al., 2009). Although we have obtained substantial information on the order of the difficulty of grammatical categories, we need to explore the relationship between learners acquisition stages and their mastery of grammatical categories, which is the focus of the present study. (Note 2) Summary of the background of our research
Unaccusat 1) Unacusative verbs ( e.g., happen; break) The accident happened 15 years ago. [ empty [ VP happen the accident ] ] The accident (THEME) 2) Unergative verbs (e.g., dance; cry ) Mary danced. [ Mary [ VP danced] ] Thematic role: Mary (AGENT) *Most of people are fallen in love and marry with someone. (Japanese L1) (Zobl, 1989: 204) 1) NP movement account: Analogy to the passive *The accident was happened 15 years ago. [ empty [ VP happen the accident ] ] 2)Lexical causativization account: Creation of a temporary causer *The driver happened the accident 15 years ago. (Note 3) Unaccusative verbs and unergative verbs The Unaccusative Hypothesis (Perlmutter, 1978, Burzio, 1986) L2 literature: overpassivization errors of unaccusatives Two Major Accounts of Passivized Unaccusative Verbs
Our previous studies utilizing ANOVA and IRT also found that the order of difficulty of the categories were B<A<F E<D<C. This means that, if used in the same construction, learners always find unaccusatives more difficult than unergatives, and that they start to be able to judge grammatical sentences (NP+V) as grammatical first, and then judge ungrammatical ones (NP+V+NP) as ungrammatical. Unaccusatives, if used in NP+be+p.p., are the most difficult for them. All the learners in Ranks 1-4 judged in the same order. Some learners in higher ranks may seem to regress on an acquisition path, exhibiting a U-shaped curve. Oshita (2001) argues that a U-shaped development can be observed in the acquisition of unaccusative verbs, especially when learners progress from beginners level to intermediate level. He claims that this phenomenon is caused by learners reanalysis of unaccusative verbs analogous to the passive formation rule. (Note 5) The order of difficulty of the categories (Note 6) U-shaped development: Unaccusative Trap Hypothesis (Oshita, 2001)
Nakano, M., Sugino, N., Yamakawa, K., Ohba, H., & Shimizu, Y. (2007). A Study of grammar development among Japanese university students: Intransitive verbs, transitive verbs, ditransitive verbs and logical subjects in Xcomps - Part (1). Proceedings of the 12th Conference of Pan-Pacific Association of Applied Linguistics (pp. 264-267). Ohba, H., Sugino, N., Nakano, M., Yamakawa, K., Shimizu, Y., & Kimura, S. (2006). The Development of Grammatical Competence of Japanese EFL Learners: Focusing on Relative Clause Constructions. Journal of Pan-Pacific Association of Applied Linguistics, 10, 73–87. Ohba, H., Sugino, N., Yamakawa, K., Shimizu, Y., & Nakano, M. (2008). Exploring the acquisition of English articles by Japanese EFL learners using on-line tasks. Paper presented at The Third CLS International Conference (National University of Singapore, Singapore). Ohba, H., Yamakawa, K., Sugino, N., Shimizu, Y., & Nakano, M. (2009). The acquisition of restrictive relative clauses by Japanese and Korean learners of English. Proceedings of the 14th Conference of Pan-Pacific Association of Applied Linguistics (pp. 467-470). Shimizu, Y., Yamakawa, K., Sugino, N., Ohba, H., & Nakano, M. (2006). Developing a sample-free grammatical proficiency test for SLA research. Proceedings of the 10th Conference of Pan-Pacific Association of Applied Linguistics (pp. 227–235). Sugino, N., Yamakawa, K., Ohba, H., Nakano, M., & Shimizu, Y. (2006). The acquisition of psych verbs by Japanese EFL learners: From the perspective of cue dependency. Paper presented at the 11th Conference of Pan-Pacific Association of Applied Linguistics (Kangwon University, Korea). Yamakawa, K., Sugino, N., Kimura, S., Nakano, M., Ohba, H., & Shimizu, Y. (2003). The development of grammatical competence of Japanese EFL learners: Focusing on unaccusative/unergative verb. Annual Re-view of English Language Education in Japan, 14, 1– 10. Yamakawa, K., Sugino, N., Ohba, H., Nakano, M., & Shimizu, Y. (2008). Acquisition of English grammatical features by adult Japanese EFL learners: The application of Item Response Theory in SLA research. Electronic Journal of Foreign Language Teaching (e-FLT), Vol.5, No.1: pp.13-40. Yamakawa, K., Sugino, N., Shimizu, Y., Nakano, M., & Ohba, H. (2009). The acquisition of unaccusative verbs by Japanese and Korean learners of English. Proceedings of the 14th Conference of Pan-Pacific Association of Applied Linguistics (pp. 9-14). *The present study is supported by Grants-in-Aid for Scientific Research 1348064 (2001-2003), 16320078 (2004-2006) and 19320091 (2007-2009) from the Japan Society for Promotion of Science. References Acknowledgments
*The present study is supported by Grants-in-Aid for Scientific Research 1348064 (2001-2003), 16320078 (2004-2006) and 19320091 (2007-2009) from the Japan Society for Promotion of Science. Acknowledgments