Presentation is loading. Please wait.

Presentation is loading. Please wait.

Briefing Session on the 2012 HKDSE English Language Assessment Framework March 2008.

Similar presentations


Presentation on theme: "Briefing Session on the 2012 HKDSE English Language Assessment Framework March 2008."— Presentation transcript:

1 Briefing Session on the 2012 HKDSE English Language Assessment Framework March 2008

2 2 Background Third consultation conducted in late 2006 New public assessment of HKCEE English Language first implemented in 2007 Further consultation in the light of the 2007 HKCEE experience Questionnaire survey conducted in January 2008

3 3 2007 HKCEE Experience

4 4 SBA Implementation Schedule YearOptions for School 20071. Submit SBA marks for feedback; marks included as 15% of the subject result 2. Submit SBA marks for feedback only; marks not included in subject result 3. Not submit SBA marks; exam results to contribute 100% of subject result 20081. Submit SBA marks for feedback; marks included as 15% of the subject result 2. Submit SBA marks for feedback only; marks not included in subject result 20091. Submit SBA marks for feedback; marks included as 15% of the subject result

5 5 Implementation of SBA in 2007 No. of schools Percentage (%) No. of candidates Percentage (%) Option 1 (Yes) 1993431,87643 Option 2 (Trial) 1252220,94528 Option 3 (No) 2544421,38829 Total 57810074,209100

6 6 Moderation Results of Option 1 Schools The mean of the SBA scores is … No. of Schools Percentage (%) within the range14472.4 slightly higher than expected2914.6 higher than expected21.0 much higher than expected00 slightly lower than expected2110.6 lower than expected31.5 much lower than expected00

7 7 Moderation Results of Option 1 Schools The standard deviation of the SBA scores is … No of SchoolsPercentage (%) as expected17989.9 slightly wider than expected00 wider than expected00 slightly narrower than expected105.9 narrower than expected94.5 much wider than expected00 much narrower than expected10.5

8 8 Moderation Effect on Candidates Mark Adjustment (% of subject) No. of candidates Percentage of candidates (%) 0 (0)536517 1-3 (<1)1988162 4-6 (<2)623720 7-9 (<3)3921 31,875100

9 9 Implication of Moderation Results Mean and spread of the SBA marks submitted by the majority of schools fall within the expected range Most teachers have a good understanding of the assessment criteria and can assess their students accurately Reliability of SBA not an issue Fairness ensured by statistical moderation

10 10 Absentee Rate of School Candidates in 1996 CE English Language Speaking Exam English Language (Syllabus A) No. Sat6,623 No. of Absentees3,058 Absentee Rate31.6 % English Language (Syllabus B) No. Sat88,201 No. of Absentees19,236 Absentee Rate17.9%

11 11 Absentee Rate of School Candidates in 2007 CE English Language Speaking Exam No. SatNo. of Absentees Absentee Rate (%) Option 1 (Yes)27,9353,39810.8 Option 2 (Trial)19,3071,4667.1 Options 1 & 2 (Yes + Trial) 47,2424,8649.3 Option 3 (No)19,2983,68116.0 All Schools66,5408,54511.4

12 12 Possible Explanations 1. Students in Yes and Trial schools had more speaking practice in school because of the SBA 2. They became more confident with the public exam format, which involves similar speaking tasks – group discussion and individual response 3. Positive backwash effect of the SBA on the ability to speak English and to interact with peers in different contexts.

13 13 Effect of SBA on Speaking Exam No. Sat Speaking Exam Mean Moderated SBA Mean Option 1 (Yes) 27,80425.3827.81 Option 2 (Trial) 19,38126.00- Option 3 (No) 21,29323.51- All Schools 68,47824.97-

14 14 Effect of SBA on Speaking Exam Students in Yes schools got higher marks in the SBA than in the speaking exam Students in Yes and Trial schools got higher marks in the public speaking exam than those in No schools BUT Yes and Trial schools may just be better schools

15 15 Effect of SBA on Speaking Exam No. Sat Actual – Predicted Mean Scores t-value Option 1 (Yes) 27,8040.25-1.97 Option 2 (Trial) 19,3810.03-0.21 Option 3 (No) 21,293- 0.352.34* All Schools 68,478--

16 16 Effect of SBA on Speaking Exam Students in Yes schools did better than expected based on prediction from written papers (+0.25) Students in Trial schools did slightly better than expected (+0.03) Students in No schools did significantly worse than expected in the speaking exam (-0.35)

17 17 Congeneric Measures Model Analysis Strong evidence that the various components are measuring a single underlying ability Reliability of components.882 to.702 R > L&IS > W > SBA > S SBA (.753) more reliable than speaking exam (.702)

18 18 Correlation Matrix (by listwise case exclusion, N=28,253) Inter-paper correlations Correlation Matrix (by listwise case exclusion, N=28,253) RWL & ISSSBA R1.000.858.887.776.803 W.8581.000.852.767.797 L & IS.887.8521.000.764.796 S.776.767.7641.000.787 SBA.803.797.796.7871.000

19 19 Survey Results

20 20 Main focus of questionnaire Assessment of the modules in the Elective Part Replacing the speaking exam with SBA The need for strategic implementation of SBA in 2012 HKDSE Teachers satisfaction with the level of support provided by the HKEAA and the EDB

21 21 Results: Response Rate All Schools Part. Schools Option 1 Schools Option 2 Schools Option 3 Schools 717640199125254 422411165109130 59%64%83%87%51%

22 22 Results Responses measured on 6-point scale 3.5 adopted as minimum level of acceptability

23 23 Assessment of the Modules in the Elective Part Question All schools Option 1 schools Option 2 schools Option 3 schools 1. Assessing the modules in the Elective Part of the curriculum through the writing paper is appropriate. 3.7 2. The proposed Paper 2 structure is appropriate. 3.8 3.7 3. The proposed weighting of Part A (10%) and Part B (20%) is appropriate. 4.0 4. The question types for the modules in the Elective Part are appropriate. 3.7 5. Assessing the modules in the Elective Part of the curriculum through Part B of the SBA is appropriate. 3.94.0 3.7

24 24 Assessment of the Modules in the Elective Part Question All schools Option 1 schools Option 2 schools Option 3 schools 6. The proposed weighting of Part B (5%) is appropriate. 3.63.73.63.5 7. The amount and type of effort required of teachers to implement the SBA in HKDSE is reasonable. 2.93.13.02.7 8. The amount and type of effort required of students to complete the SBA in HKDSE is reasonable. 3.43.5 3.3 9. Students have benefited from the introduction of SBA. 3.94.13.93.6 10. The effort spent on implementing SBA has been worthwhile. 3.53.73.53.3

25 25 Replacing the Speaking Exam with SBA Question All schools Option 1 schools Option 2 schools Option 3 schools 12. School candidates need not take the public speaking examination as their speaking skills can be assessed by means of school-based assessment. 2.7 2.62.7

26 26 Teachers satisfaction with level of support provided by HKEAA and EDB (1) 13. Resources and support provided by the HKEAAAll schools 01. Handbook provided for the HKCEE English Language School-based Component 4.3 02. 2007 HKCE English Language Examination Introduction to the School-based Assessment Component (with DVD) 4.3 03. Training CDRs 1 and 24.1 04. HKCEE Chinese Language and English Language School- based Assessment (DVD distributed to parents and schools in October 2006) 3.8 05. HKCEE Chinese Language and English Language Information for Parents on School-based Assessment (leaflet distributed to parents in October 2006) 3.7 06. Statistical moderation of School-based Assessment scores (booklet distributed to schools in June 2007) 3.8 07. Group Coordinators System3.8

27 27 Teachers satisfaction with level of support provided by HKEAA and EDB (2) 08. Teachers seminars on the implementation of HKCEE English Language SBA component held in September/October 3.9 09. Assessment training sessions held in November (chaired by SBA group coordinators) 4.0 10. Inter-school sharing sessions (chaired by SBA group coordinators) 3.8 11. Professional development courses (conducted by HKU and HKIED) 3.9 12. SBA Corner on the HKEAA website (including General Information, List of Recommended Texts and FAQs) 4.0 13. The Platform for SBA of CE English Language Exam on the English Campus of HKedCity 4.3 14. The HKU website on SBA Projects3.9

28 28 Teachers satisfaction with level of support provided by HKEAA and EDB (3) 14. Resources and support provided by the EDBAll schools 01. Resource materials4.0 02. Professional development programmes4.0 03. Grants4.3 04. School-based Support Services3.8 05. Collaborative research and and development (seed) projects 3.5

29 29 Need for Strategic Implementation of SBA in 2012 HKDSE Question All schools Option 1 schools Option 2 schools Option 3 schools 16. My schools preparation plan for implementing the SBA for English Language in 2012 HKDSE is on track. 3.53.7 3.3 17. The proposed SBA framework for English Language is desirable for implementation in the 2012 HKDSE. 3.63.7 3.5

30 30 Respondents comments (1) 6. The proposed weighting of Part B (5%) is appropriate. Part B should have a heavier weighting. Both Parts A and B should each account for 10% Both Parts A and B should each account for 5%

31 31 Respondents comments (2) 7. The amount and type of effort required of teachers to implement SBA in the HKDSE is reasonable. Workload for teachers too heavy Too many assessments to do More resources needed, especially technical support More sample tasks provided

32 32 Respondents comments (3) 8. The amount and type of effort required of students to complete SBA in the HKDSE is reasonable. Reading six texts in three years and having to do four assessments in two years is too demanding. Hard to monitor time spent by candidates on SBA reading/viewing programme

33 33 Respondents comments (4) 18. State the kind of modification(s) to the English Language framework you think would be desirable for the 2012 HKDSE. Provide sample tasks Improve standardization SBA weighting should be increased/reduced Use outside examiners for SBA Keep Speaking exam Reduce the number of assessments

34 34 Proposed/Revised Assessment Frameworks Current proposed framework Revised proposed framework ComponentsWeightingRequirementsWeightingRequirements Paper 1 Reading20% Paper 2 Writing30%25% Part A10%Word limit: 20010%Word limit: 200 Part B20%Word limit: 40015%Word limit: 400

35 35 Proposed/Revised Assessment Frameworks ComponentsWeightingRequirementsWeightingRequirements Paper 3 Listening & Integrated Skills 30% Paper 4 Speaking20%For private candidates only 10%For all candidates

36 36 Proposed/Revised Assessment Frameworks ComponentsWeightingRequirementsWeightingRequirements SBA20%For school candidates only 15%For school candidates only Part A15%Three assessments; group interaction and individual presentation; report three marks 10%Two assessments; group interaction and individual presentation; report two marks Part B5%One assessment based on the modules in the Elective Part of the curriculum; report one mark 5%One assessment based on the modules in the Elective Part of the curriculum; report one mark

37 37 Rationale for the Revised Framework Teachers need to deal with HKCEE, HKALE and HKDSE from 2009 to 2012 Speaking exam retained for school candidates as the majority of teachers do not favour its replacement by SBA Weighting of SBA component reduced from 20% to 15% and weighting of Part B of Writing paper reduced from 20% to 15% so that 10% can be allocated to the Speaking exam The total number of SBA marks to be submitted reduced from 4 to 3 to address teachers concerns about workload. Weighting of SBA component reduced to 15%, which is the same as the 2007 HKCEE

38 38 Schedule for HKDSE Development Standards-referenced Reporting (SRR) Production of sample papers and marking schemes Pilot test of sample papers conducted Draft of level descriptors and annotated examples completed Briefing sessions conducted for schools & SRR requirements finalised by Dec 08 SRR booklets with sample papers disseminated to schools by Mar 09

39 39 Schedule for HKDSE Development School-based Assessment (SBA) Assessment criteria, specifications and sample tasks developed Briefing sessions conducted for schools; SBA requirements finalised by Dec 08 SBA handbook disseminated to schools by May 09

40 40 Thank You Questions?


Download ppt "Briefing Session on the 2012 HKDSE English Language Assessment Framework March 2008."

Similar presentations


Ads by Google