Presentation is loading. Please wait.

Presentation is loading. Please wait.

DIAGNOSTIC TESTING – A TALE OF TWO INITIATIVES Using diagnostic tests to assist in (i)widening access and (ii) post hoc curriculum design NAN YELD.

Similar presentations


Presentation on theme: "DIAGNOSTIC TESTING – A TALE OF TWO INITIATIVES Using diagnostic tests to assist in (i)widening access and (ii) post hoc curriculum design NAN YELD."— Presentation transcript:

1 DIAGNOSTIC TESTING – A TALE OF TWO INITIATIVES Using diagnostic tests to assist in (i)widening access and (ii) post hoc curriculum design NAN YELD

2 The Alternative Admissions Project The Exit-level Testing Project To identify talented students whose school results would not qualify them for entry To provide useful information for curriculum design To place these students into appropriate programmes of study To explore the effectiveness (for entry level curriculum design) of diagnostic information on graduating students To deliver meaningful diagnostic information for curriculum design To assess the suitability of using existing entrance selection/diagnostic tests in this attempt

3 Selected TIMSS 1999 performance indicators Benchmark Description of benchmark % students reaching benchmark Highest performing group (e.g. Singapore, Korea, Japan) Lowest performing group (South Africa, Philippines, and Morocco) 1 Top 10% Students can organise information, make generalisations, and explain solution strategies in non-routine problem solving situations. 33 % 33 % 1 % 1 % Upper quarter Students can apply their understanding and knowledge in a wide variety of relatively complex situations. ± 66 % 1 % Median Students can apply basic Mathematical knowledge in straightforward situations. ± 90 % 10 % 10 % Lower quarter Students can do basic computations with whole numbers. ± 99 % 31 % Percentage of students scoring below benchmark levels 1 % Note 1: South Africa scored the lowest of this group. 69 %

4

5

6 Some factors impacting on test development Some factors impacting on test development Issues to be considered Consequences for test development Affordability and feasibility Pen and paper mark-ability format issues Impact on system (e.g. importance of not undermining the Sen. Certificate) Tests not in school subject areas Performance on SC remains important Impact on individuals (importance of not creating another barrier to access) Writing of tests voluntary Candidates assessed on EITHER SC results OR project test results The tests need to do something different (avoid predicting the past) Develop and implement innovative ways of getting at underlying ability - scaffolding The tests must not perpetuate the revolving door tendency seen in many access projects The tests need to be able to inform curriculum development – must have some diagnostic power.

7 In a higher education context, students are required to: Make meaning from what they read; Understand and interpret conceptual and metaphorical language; Identify and track academic argument; Follow discourse structure in text; Make inferences about and extrapolate on what they read; Demonstrate familiarity with and understanding of the conventions of visual literacies, such as reading and interpreting graphs, flow-charts and diagrams; and Cope with basic numeracy demands.

8 Getting to grips with these requirements is seriously impeded by approaches to texts and epistemic practice such as: A tendency towards verbatim reproduction or plagiarism in essays A tendency to describe rather than analyse, and to offer tautologies in place of justification A tendency to focus on examples (tokens) rather than on principles (types), and the relation between them A tendency to write from a highly subjective viewpoint without depersonalising A tendency to be prescriptive or normative when asked to be analytic (Slonimsky and Shalem 2005)

9 Did the approach provide a greater range of scores (spread), so that capable students could be more clearly differentiated from weaker students? Did the approach increase the predictive validity of the test (that is, did the test correctly distinguish between weaker and stronger students)? Did the approach improve (raise) the level of stronger students scores? Assessing the effectiveness of the scaffolding approach

10 ELPT and ALPT Scores

11 Task Preparation Study Question 6.1: In note form (i.e. not full sentences), list the main points the author makes about the Japanese educational system. For this exercise, do not include comparisons or references to the American system. Question 6.2: Drawing on the text of the whole article, but using your own words, summarise the points the author makes about the Japanese system of education. Note: your summary should not include comparisons or references to the American system. Top 30% of candidates (n=114) Bottom 30% of candidates (n=114) Points made in notes and carried over to summary.62- 0.35

12

13

14 ASSUMPTIONS UNDERLYING EXIT-LEVEL TESTING PROJECT 1By and large, and with many notable exceptions, academics …. do not take seriously/ are not well informed about / are not very interested in …. information about the learning needs of incoming students. 2Academics are more likely to take information about the learning needs of their students seriously if these needs are demonstrably still present in the students they are about to graduate. 3In order to demonstrate these needs, a valid and credible (and feasible) instrument is needed.

15 1.By and large, academics …. do not take seriously/ are not well informed about / are not very interested in …. information about the learning needs of incoming students. Why might this be so? They believe it would be lowering standards to pander to needs?They believe it would be lowering standards to pander to needs? They believe its the responsibility of the schooling sector to prepare students, not theirs?They believe its the responsibility of the schooling sector to prepare students, not theirs? They assume that massification will entail higher failure rates, its only natural?They assume that massification will entail higher failure rates, its only natural? They are really only interested in students who will be majoring in their disciplines, and so its the survivors from the first-year shakedown that they are focused on?They are really only interested in students who will be majoring in their disciplines, and so its the survivors from the first-year shakedown that they are focused on? Because doing so requires a shift from an inward looking, introjected orientation, to an outward looking, projective orientation (Barnett 2000), and a willingness to negotiate curricula across departmental boundaries?Because doing so requires a shift from an inward looking, introjected orientation, to an outward looking, projective orientation (Barnett 2000), and a willingness to negotiate curricula across departmental boundaries? The outsider status of those providing the information?The outsider status of those providing the information?

16 DESCRIPTIVE STATISTICS Entering Group N = 95 Graduating group N = 67 Mean36.945.1 Standard Error 1.31.2 Median35.446.1 Mode39.939.3 Standard Deviation 13.09.5 Minimum11.827.0 Maximum71.965.2 Range60.138.2

17

18 Skill cluster descriptions (examples) Skill Assessed Explanation of Skill Area Extrapolation, application and inferencing Students capacities to draw conclusions and apply insights, either on the basis of what is stated in texts or is implied by these texts. Understanding the communicative function of sentences Students abilities to see how parts of sentences / discourse define other parts; or are examples of ideas; or are supports for arguments; or attempts to persuade Separating the essential from the non-essential Students capacities to see main ideas and supporting detail; statements and examples; facts and opinions; propositions and their arguments; being able to classify, categorise and label Understanding basic numerical concepts Students abilities to make numerical estimations; comparisons; calculate percentages and fractions; make chronological references and sequence events / processes; do basic computations Cliff, A., Hanslo, M. and Visser, A-J. (2003). Faculty of Engineering and the Built Environment: AARP tests and first-year students academic performance for the 2002/3 cycles. Unpublished report: University of Cape Town, December 2003.

19

20 REFERENCES Barnett, R. (2000). Supercomplexity and the curriculum. Studies in Higher Education, 25, 255-2665. Cliff, A., Hanslo, M. and Visser, A-J. (2003). Faculty of Engineering and the Built Environment: AARP tests and first-year students academic performance for the 2002/3 cycles. Unpublished report: University of Cape Town, December 2003. Slonimsky, L. and Shalem, Y. (2004). Pedagogic responsiveness for academic depth. In Griesel, H. (Ed.)(2004). Curriculum responsiveness – case studies in higher education. South African Universities Vice- Chancellors Association: Pretoria, 81 - 101.


Download ppt "DIAGNOSTIC TESTING – A TALE OF TWO INITIATIVES Using diagnostic tests to assist in (i)widening access and (ii) post hoc curriculum design NAN YELD."

Similar presentations


Ads by Google