1. Department of Early Childhood education
Practices for Ensuring the Integrity of Assessment in Open and Distance Learning
Salosh Govender
Department of Early Childhood education
Unisa

2. OVERVIEW OF PRESENTATION
Background the problem
Threats to assessment integrity in an Open and Distance eLearning
Assessment integrity
Need for transformatiive assessment
Rationale for transformative assessment
Student graduateness
Principles underpinning assessment integrity
Transformative assessment strategies
Key Stakeholders
Validity, Reliability, Fairness

3. Threats to assessment integrity in an ODeL Context
Absence of face-to-face (F2F) interaction,: Limited use of varied assessment methods and feedback to help students achieve the targeted learning objectives.
Diverse student profile : Lack of authentic assessment activities to meet individual needs
Limited opportunities to promote self regulated learning, online discussions, collaboration
Inconsistencies in marking among our students between assignments and the examination
Increased weighting of summative assessment in relation to formative
Assessment focused on scoring rather than improving the quality of teaching and effective learning.
It should be based on making expectations explicit and public, setting appropriate criteria and high standards for learning quality, systematically gathering, analyzing and interpreting evidence to determine how well performance matches expectations and standards, and using the resultant information to document, explain, and improve performance.
Getting these variables to work in ODeL presents mammoth challenges.
RESEARCH QUESTIONS
How can ODeL lecturers validate and authenticate students’ written work?
How can lecturers tell whether the students’ submitted work sufficiently reflects their knowledge and understanding? assessment at UNISA.
What assessment practices contribute to assessment integrity within an ODeL
What are the threats to the integrity of assessments in an ODeL context?

4. Assessment integrity WHAT IS ASSESSMENT?
Process used to identify, gather and interpret information and evidence against the required competencies in a qualification in order to make a judgement about a students’ achievement.
Can be formal, informal, non-formal
Can be of learning already done, or towards learning to inform and shape teaching and learning still to be done
ASSESSMENT INTEGRITY
Honesty and transparency in every part of the assessment process.
Assessment questions/ tasks must be based on work already covered
Students must at all times be honest about what they offer to be assessed.
Markers must understand what is offered by students for assessment and to ensure fairness in assessment.
Moderators must moderate a fair sample of examples against a fair range of cases
Core values of assessment integrity include :
Constructive alignment
Validity
Reliability
Accurate and consistent marking

5. Need for transformative assessment What is transformative assessment (TA) ?
Refers to the continuous revision and review of assessment practices so that assessment practices are aligned to institutional goals, objectives, systems, programme outcomes and student profiles with the aim of improving student learning (Popham, 2017, Black & Wiliam, 2007)
TA is driven by the extent to which academics engage in innovative assessment methods and practices – and the extent to which the stability of ICT / technology systems allows for innovation
TA focuses on assessment methodologies, quality of the assessment instruments and the integrity of the assessment (Popham, 2017)

6. The rationale for transformative assessment within our ODeL environment
Assessments are dominated by traditional methods of written assignments, tasks and tests.
Assessment methods and types does not cater for diverse student profiles (inclusion, technology savy students, etc)
New technologies create opportunities for innovative and alternate assessment tools.
Improve validity, reliability authenticity of assessment
Harness multiple stakeholder support in enacting assessment (eTutors, markers, teacher assistants, etc)

7. A key institutional goal: Student graduateness
All programmes should aspire to develop students as global graduates capable of leadership and professional practice in a global community.
Graduates should be
capable of applying their discipline in local, national and international contexts
Culturally aware and capable of respecting and acting in socially just/ responsible ways
Capable of acting in environmentally responsible ways 

8. Principles underpinning assessment integrity
Constructive alignment
Validity
Reliability
Honesty

9. Transformative assessment strategies in ODeL
Constructive alignment
Authentic assessments
Alternative assessments
Key stakeholders
Assessment for Learning
Balancing formative and summative assessments

10. Theoretical underpinning: Constructive alignment model ( Biggs, 1999)
The curriculum is stated in the form of clear outcomes which state the level of understanding required rather than simply listing the topics to be covered.
Develop an integrated assessment plan starting from the programme level exit outcomes
Requires a co-ordinated process with fellow academics involved in teaching the module.
The assessment tasks address the outcomes, so that you can assess to see if, and how well the students have learned what the outcomes state they should be learning.
All components in such an aligned system thus address the same agenda and support each other.
Ensures consistency that optimises the likelihood for students to engage in appropriate learning activities.

11. Formative Assessment
“Formative assessment is a planned process in which assessment- elicited evidence of students' status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics” (Popham, 2007)
Formative assessment is not a test but a process—a planned process involving a number of different activities.
One of those activities is the use of assessments, both formal and informal, to elicit evidence regarding students' status: the degree to which a particular student has mastered a particular skill or body of knowledge.
Based on this evidence, teachers adjust their ongoing instructional activities or students adjust the procedures they're currently using to try to learn whatever they're trying to learn.

12. Assessment for Learning
AfL Strategies
Assessment for Learning techniques in an ODeL context
Clarifying, sharing and understanding learning outcomes and success criteria
Teaching plan of the study guide. Learning units
Assessment tasks
Use rubrics
Providing feedback that moves students forward
Key component for student interaction in ODeL context
Supporting E tutors 2 and Reflective blogs 2 3 4
Markers feedback: constructive, specific and actionable
Feedback on discussion forums
Activating students as instructional resources for each other
Plan authentic assessments, to engage students
Encourage collaborative learning on discussion forums
Activating students as owners of their own learning
Self regulated learning
Self assessment activities in study guide
Rubrics to assist students to self assess
Use the discussion forum platforms to establish students needs
Rely on E tutors and markers feedback
Case based activities
Engineering effective class-room discussions, questions and tasks that elicit evidence of learning

13. Authentic Assessments
Develops a sense of personal engagement
Harnesses students' everyday experiences
Brings a "live" feel to assessments
Encourages students to focus/ reflect and evaluate their own learning
Draws on and links to real world
Links learning to practice
Addresses real issues in the workplace
Learning in and through communities
Enhances students' perception of meaning and relevance of assessment
Promotes deep approaches to learning
Positions students as active learners
Emulates complex ways of thinking
Promotes hands-on experience of study

14. Transformative assessments linked to students needs
OERS
Curriculum responsiveness
Student Reflection
Context based
Africanisation

15. A triadic relationship: Key stakeholders in ODeL
Empower E tutors and markers
Gather data from E tutors and markers Provide quality feedback to help students improve learning
Academic
eTutors
Markers
AfL
Student – self assessment
Students are empowered to take actions to improve their learning through self assessment
Provide quality feedback

16. Validity and reliability of assessment practices
Evidence showing that assessment practices fully cover the validity and reliability criteria
Evidence of consistency in knowledge, skills, attitudes and values…
Evidence of consistency across markers…
Evidence that assessment is fit for purpose: Is fair
Assessment measures intended abilities/ skills/ knowledge
Assessment
Criteria
Use of rubrics
All assessments are moderated( pre/ post moderation)
Ensures alignment with NQF levels and cognitive levels
Regular markers’ meeting
Moderation of scripts
Ensure markers have a good content knowledge: Markers complete exam papers
Marking memo and rubrics
Construct of valid estimate
Construct validity/ content validity
Assessments include questions from different cognitive levels
Assessment include questions from different cognitive levels
I moderate 10% of all scripts marked by marker
Workshop academics in the department
Member of the quality assurance committee
QA through FTA
Evidence