Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jin, Kyung-Ae (Korea Institute for Curriculum & Evaluation)

Similar presentations

Presentation on theme: "Jin, Kyung-Ae (Korea Institute for Curriculum & Evaluation)"— Presentation transcript:

1 Jin, Kyung-Ae (Korea Institute for Curriculum & Evaluation)

2 1. Backgrounds To enhance practical English education and improve students speaking and writing skills, various policies were adopted. A. Backgrounds National English Ability Test (NEAT) including speaking and writing as an substitute for the current College Scholastic Ability Test, which only measure reading and writing was proposed.

3 Announce a new plan to develop a new national English ability test as an assessment reform effort to improve student English ability in speaking and writing for an enhancement of practical English education. Establishment and announcement of innovative plans for English education: Nov., 2006 Establishment and announcement of a basic plan to develop the NEAT : July, 2007 Establishment and announcement of a basic plan to develop the NEAT : July, 2007 (For student user:Level 1) Provide national educational standards for English education aligned with the curriculum in elementary, middle, and high schools. (For general user:Level 2, 3) Substitute foreign tests being used for domestic purposes, assess four modalities in balance, and attempt to obtain international comparability. Introduce a new national English ability assessment emphasizing practical English skills as a substitute for the College Scholastic Ability Test. 1. Backgrounds

4 B. Pre-tests NEAT pre-tests ( ) 1 st (33 schools, 4,300 students) 2nd(43 schools, 5,300 students) 3rd(78schols, 10,600 students) Public Hearing for NEAT (10.6) NEAT pre-tests( ) 4th(34 schools, 2,900 students) 5th(169schools, 20,000 students) 66 th (16 schools, 1,900 students) Final Public Hearing (11.5)

5 2. Goals and directions Goals Directions Students will acquire basic communicative skills through high school education Level 2: basic academic English Level 3: practical English To align the contents of NEAT with the National curriculum to reform school education Provide teacher training programs with regard to the new test Students will select either NEAT Level 2 or Level based on their aptitude and future career

6 3. Test Development A. Target System Target System Number of examinees and the tests : 120,000 and 24 tests( two opportunities per students, 50,000 students simultaneous test Secure item writers and raters: 5,000 secondary teachers IBT system: Headquarter, 1,700 IBT test center( 1113) Organization: 75 personnel

7 3. Test Development B. Timetable Year Basic scheduleDevelopment and Pre-testsGeneral implementation NEAT Level 2 & Level 3 Pre-tests (3) Pre-tests (2) Pre-tests (3) General Implementa tion Implementati on (24 times, 120 examinees) Policy making on replacing CSAT Abolition of CSAT (Upon decision) (College Scholastic Ability Test) Notice two versions of the test: A B type A B type, Increase listening items

8 3. Test Development C. Development Policy Develop the test specifications for Level 2(Basic Academic English)and Level 3(Practical English) - New English Curriculum (2009 revision) is being developed aligned with NEAT New curriculum : 11, 8 notification 11, 9 textbook development 13 Applied in high school 1 st year 15 NEAT application for college entrance

9 3. Test Development Level 2 Level 2 Level 3 No grammar items Focus on communicative skills Topics on Practical English: everyday life, work Objective

10 3. Test Development A. Format Test Domains Number of Items & Test period Number of Items & Test period Level 2 Level 3 Test period Level 2 Level 3 Test period Listening Reading Speaking Writing Total min 60min 15min 35min 145min In listening and reading domains, anchor items for test equating are included

11 3. Test Development B. Topics : Practical English TopicsSubtopics Practical English 1 Everyday life: transportation, telecommunication, shopping, housing, restaurants, hospitals, beauty shops 2 Travel & leisure: reservation transportation, concerts, exhibition, sports, hobbies, cooking, hotels, public places 3 Family & school life: class, friends, birthday party, graduation, counseling, homework, class schedule, examination, library, test score 4 Work related: forms, documents, employment, salary, marketing, anouncement, notice, advertisement, manual,

12 3. Test Development B. Topics : Basic Academic English Topicssubtopics Basic Academic English 1Humanities, sociology, politics, economy, history, education 2 Science, technology, computer, information & communication, space, ocean, environment, expedition 3Art, literature, anthropology, philosophy 4 Labor, job, career, gender equality, aging society, social welfare, population, juvenile problem 5 Culture, public morality, public order, civil life, voluntary service, cooperation

13 3. Test Development A. Test objectives: Reading Reading Level 2Reading Level 3 Types% % Main idea30~35Main idea30~35 Specific Information30~35Specific Information35~40 Infer omitted information15~20Infer omitted information5~10 Discourse structure and Cohesiveness 10~20 Discourse structure and Cohesiveness 10~15 Implied meaning5~10Implied meaning10~15

14 3. Test Development More inferential and comprehensive understanding items in Level 2 than Level 3 More literal understanding items in Level 3 SectionBehavioral skillLevel 2Level 3 Listening Literal understanding60%68% Inferential understanding40%32% Reading Literal understanding32%44% Inferential understanding56%48% Comprehensive Understanding12% 8% B. Behavioral Domain: Reading & Listening

15 TextNumber of items per text Number of words per text Level 2 Level 3 Dialogue 1110 – – – 180– Monologue 190 – – – – 110 C. Number of Words (Listening) 3. Test Development

16 TextNumber of items per text Number of words per text Level 2Level 3 Single paragraph 1130 – 150(± 10%) 120 – 145(± 10%) Multiple paragraphs 2230 – 250(± 10%) 170 – 200(± 10%) 3320 – 350(± 10%) 210 – 240(± 10%) D. Number of Words (Reading) 3. Test Development

17 E. Item Types Various charts, pictures, graphs and realia are adopted to enhance authenticity of the test and align with the curriculum and textbooks Level 2Level 3 Item Type% % Appropriate answer10~15%Appropriate answer15~20% Theme, title, Main idea, Purpose, Opinion 30~35%Theme, title, Main idea, Purpose, Opinion25~30% Contents agreement, Request, Things to do 35~40% Contents agreement, Request, Things to do, Find reason 35~40% Find location, tables and information15~20% Find appropriate picture, Pictures agreement, Find location, tables and information 15~20% (1) Listening

18 3. Test Development (2) Reading Level 2Level 3 Item Type% % Theme, Title, Main idea, Purpose, Assertion 30~35 Theme, Title, Main idea, Purpose, Assertion 30~35 Literal Information(Contents Agreement) 30~35 Literal Information(Contents Agreement) 35~40 Infer blanks15~20Infer blanks5~10 Sentence insertion5~10Find order of pictures or sentences10~15 Find the order of contents5~10 Infer meaning of phrases and pronouns 10~15 Infer meaning of phrases and pronouns 5~10

19 3. Test Development (3) Speaking Level 2Level 3 Item Type Number of Items Item TypeNumber of Items Related Interviews1(4)Related Interviews1(4) Problem Solving1 1 Picture Description1 1 Presentation1Answers based on pictures1(3) Total4 4 (4) Writing Level 2Level 3 Item TypeNumber of ItemsItem TypeNumber of Items Guided Writing1 Selective Picture Description1 Complete Detailed Picture description1 Short Essay1 Writing1 Writing based on Picture Cue1 2 4

20 Listening


22 Reading


24 Speaking


26 ..

27 ..

28 4. IBT System A. Cloud Computing IBT System 1)Setting up VDI(Virtual Desktop Infrastructure) at the Central Center 2)The students log in on their monitor and take the test at the test center 3) The client in the test center down loads the test items and uploads the answers 4) Minimize effect of PC environment for providing equivalent test environment

29 .. B. Test Center 4. IBT System Year Number of test sites (for simultaneous administration) 500 rooms (21,000 students) 1,200 rooms (36,000 students) 1,700 rooms (51,000 students) AreaSeoulPusanTaeguIncheonKwangjuTaeguUlsanKyeonggiTotal Number of test sites ,705 AreaKangwonChungbukChungnamJeonbukJeonnam Keongbu k Kyeongn am Jeju Number of test sites

30 5. Score Report Score Report: Provide performance level for listening, reading, speaking and writing 1)Criterion-referenced test Excellent, Average, Basic and Below Basic Standard Setting (Bookmark Method) 2) Test equating Students scores of different tests are equated non-equivalent group design with external anchor items

31 2. Conclusions A. Survey (2009, Jin et al.) With 500 respondents, 62.1% said NEAT will enhance students speaking and writing ability 62.1% (English Education Specialists: 75%, Non-English Teachers: 68%, Parents: 61.4%, English Teachers: 54%) 68% 75% 61.4% 54%

32 2. Conclusions B. Evaluation of English Education Policy(2010, Kim, Jeon, Woo & Jin) 67.8% agree to adopt NEAT 72.5% of parents expect innovation of speaking and writing instruction at school 65.92% of parents expect that students will speak and write English better 67.8% 72.5% 65.92%

33 6. Conclusions Enhance students speaking and writing skills in addition to listening and reading through adopting NEAT Students can select different tests (Level 2 or Level 3) based on their future career and aptitude Through criterion-referenced test, the students can have clear performance objectives Positive wash back effect on school English education


Download ppt "Jin, Kyung-Ae (Korea Institute for Curriculum & Evaluation)"

Similar presentations

Ads by Google