Definition of Mixed Methods Evaluation Qualitative Data Quantitative Data Instruments Checklist Records Interviews Observations Documents Audio-visual materials Mixed methodology is a design for collecting, analyzing, and mixing both quantitative and qualitative data in a single study or series for studies to understand an evaluation problem(Creswell and Plano Clark, 2007).
Proposal for Objective and Valid Evaluation Multiple Choice Estimate scientific, directive, and objective performance. Take long time to develop the test sheet and also difficult to develop identical test sheets. Respondents’ position, difficult to respond. Before developing the test times, it is necessary to check the characteristics of respondents Likert Scale Easy to verify the reliability and validity of test items. Respondents’ position, Easy to respond. The same test items can be used. It’s difficult to estimate real ability because it is based on the respondent’s perception. Respondents can misrepresent about their ability Evaluation Instruments Likert Scale Multiple Choice Master Teacher Leader Teacher
Evaluation Framework of ISST Competencies Competency Factors AccessibilityCapabilityPractical UseSatisfaction Technology Communicative Pedagogical Research / Investigative Management Design Domain of ICT competence for teachers is technology, communicative, pedagogical, research/investigative, management and design.
Teacher Competencies for ICT Use in Education According to the MEN (2013), there are three levels of all teachers’ competencies: Explorer, Integrator, and Innovator. Competencies Levels ExplorerIntegratorInnovator Technology Communicative Pedagogical Research / Investigative Management Design
Development Strategies and Schedule Develop & Confirm Indicators 5.10 ~ 5. 20 Expert review 5.20 ~ 5. 30 Preliminary test with 300 samples 6. 1 ~ 6. 7 Analysis and set up the tests 6. 8 ~ 6. 15 Development of taxonomy 6. 1 ~ 6. 10 Item development 6. 10 ~ 6. 25 1 st Preliminary test With 300 samples 6. 25 ~ 7. 5 Analysis and revise the tests 7. 5 ~ 7. 10 Setting the criteria 6. 15 ~ 6. 25 2 st Preliminary test With 300 samples 7. 10 ~ 7. 20 Analysis and set up the tests 7. 20 ~ 7. 25 Setting the criteria 7. 25 ~ 7. 31 Multiple Choice Likert Scale
Development Methodology Validation of Evaluation Instruments Content validity (through experts’ review) Construct validity (through statistic program) Item response theory (through statistic program) Method for Setting the Criteria ( Standard- setting methods) Angoff & Jaeger method