1. of Training Proctors and Invigilators
FIFTY Shades
of Training Proctors and Invigilators
for Standardized Test Administration
Granit ZELA
Albanian Test Team Head
STANAG 6001 Testing Workshop
Skopje 5-7 Sept 2017

2. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Agenda
Standardized test administration validity
Invigilator training goals, role, aids/activities
Invigilator responsibilities: prior, during, at end of the test
4. Test cheating: test theft, test cheating methods
Cheating in 21st century has become incredibly high-tech with a plethora of gadgets available to dishonest students.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

3. Standardized Test Administration Validity
in the Garden of Eden
The cheating
“Crib Garment”
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 3 3 3

4. Standardized Test Administration Validity
Test administration according to standardized procedures:
uniform testing conditions uniform test administration
1. The invigilator at one test centre who shows students to their seats is rude and aggressive.
2. Five venues are selected for a speaking test during the summer. One of these is next to a major road where extensive road works have just begun.
Test administration is to be done according to standardized procedures: the same test content, resources, directions, testing conditions and scoring procedures.. Consistency in language testing is about avoiding any variation in procedure that could threaten score meaning. Standardization requires that invigilators give all test sessions in the same way.
Testers constantly emphasize the importance of validity, or how we ensure that the inferences we make from test scores are meaningful and can be supported. In order to understand this we will consider two examples.
The invigilator at one test centre who shows students to their seats is rude and aggressive.
Five venues are selected for a speaking test during the summer. One of these is next to a major road where extensive roadworks have just begun.
In each case there is the very real possibility that the score awarded will not mean what we intend it to mean.
In the first example it is highly likely that coming into contact with such an individual is going to result in stress. When students arrive to take tests they are usually quite nervous at best. We try to put students at their ease so that when they come to take the test they are performing as well as they are able. If stress interferes with taking the test, the lower score that they get may well reflect this stress and not their ability on the intended construct.
The second example involves non-human factors in the test-taking process that require considerable foresight and planning. During the test there may be significant noise from the road that distracts students, something that we know to have an impact on test scores. Alternatively, the test administrator may close all the windows and increase the temperature in the room to a level that also causes discomfort. As there are five venues, we have the additional problem of fairness to all students. One group of students has been treated differently and, if they are at a disadvantage compared with others, also unfairly.
Any non-standard practice in the administration of testing is a potential threat to validity. We should therefore think in terms of the quality assurance procedures that we could put in place to protect the validity of the tests.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

5. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Training goals
Fully comprehend what is expected of them for the test administration
Be confident in roles and responsibilities as invigilators
Understand regulations for running tests
Be able to act appropriately in any given circumstances
By the end of the training, invigilators will:
T o become an invigilator one must successfully succeed in a mandatory training
Only staff who will have completed the invigilator training to a satisfactory level is to be allocated invigilation work
Invigilation training is designed to support you as an invigilator and equip you with the necessary skills and knowledge to perform to the highest level
It is tailored to fit institutions’ requirements.
This presentation is designed to train their invigilators. The trainer can decide how much information to give to invigilators. For example, if invigilators are new to invigilating we recommend showing them the whole presentation and concentrating on the key terms. However, if this is a refresher course and invigilators are confident on key terms, concentrate on the regulations.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 5 5

6. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Invigilator Training
Invigilation training must be mandatory
Tools, knowledge and experience to successfully invigilate
Introduces invigilation practices to perform to the highest level
Addresses trainees concerns
Provides tips and insights
Updates and improves invigilation competence
Ensures that the correct procedures are followed
Shares good practice and experience
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 6

7. Invigilator Training aids
Power point slides/DVD
Links to resources
Discussion activities
Training notes
Information sheets
Quizzes
Videos
as part of your invigilator training presentation (downloadable PowerPoint) to check understanding
to refresh your invigilators' knowledge and stimulate discussion at team meetings
to support a work shadowing exercise as part of invigilator training, where a new invigilator works with an experienced invigilator.
The quiz questions are suitable for inexperienced and experienced invigilators.
Test fraud scenarios
Invigilator Handbook
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 7

8. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Prior to testing
Appropriate signage
Acoustic conditions, heating, lighting, ventilation
A quiet testing environment
Clock placed in prominent position
Candidate warning posters
A room seating plan
A Test Checklist of testees
Stationery equipment distributed
Right question papers and candidate materials
Know to operate the equipment
A tested method of calling for assistance
We are now going to look at your roles and responsibilities as an invigilator. Does anyone have any questions before I begin? What do you think your role and responsibilities are as an invigilator?
give all your candidates the same exam experience
The centre needs to ensure that all reasonable precautions are taken to ensure that only genuine candidates sit the exam. They should have the appropriate checks in place to make sure this happens.
All written exams require an invigilator the ratio for these is : 1 invigilator for 30 candidates, 2 invigilators for 31 to 70 candidates and 3 invigilators for 71 or more candidates.
The room set up should ensure all candidates are facing the same direction with at least 1.25 metres in all direction space between candidates.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 8

9. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Prior to testing
Test is in progress as soon as candidates enter the room.
Check the identification of each candidate.
Ensure all mobile phones and electronic devices are handed in
Remove any unauthorised materials from candidates and place
Make sure candidates are seated according to your seating plan.
Check that they have the items they need for the exam.
Distribute test materials
Give clear instructions about the test completion, time limits
State clearly the policy on cheating and theft.
Know procedure for late arrivals
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

10. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
During the testing
Observe candidates all the time
Move around the room
Respond when a candidate asks for help
No advice about the content of the test paper
Look out for any malpractice or candidates looking unwell
Be vigilant for use of hidden electronic communication
If a candidate is disruptive remove them from the room while you resolve the issue.
Maintain the required number of invigilators. .
Reinforce the rules – warning notices.
Explain why you need to address a situation.
Summon help as necessary.
Know how much information can be given to candidates
Know how to deal with test-takers
Deal with unforeseen eventualities
Be aware of the kind and amount of help that can be given in response to questions.
Deal with disruptive students.
Prevent possible candidate malpractice – By following the regulations for the conduct of external assessment the Invigilator should ensure that they are alert to any potential candidate malpractice during an external assessment. They need to remain alert and observe the proceedings to ensure any discrepancies are picked up and acted on. By giving the candidates the appropriate instructions before the assessment, setting the room up correctly and being vigilant during the assessment the Invigilator will do the utmost to prevent malpractice by the candidates. It is essential that the Invigilator is observing the candidates at all times during the external assessment and not doing other work during it.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 10

11. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Ending the exam
Inform candidates of the remaining time
Tell candidates to stop writing.
Collection of all materials
Complete relevant documentation
Be committed to test material security
Do not read or allow anyone else to read any of the scripts
Collect any unused stationery or equipment.
Hand the scripts and attendance register
Emergency situations
Leave the scripts and question papers in the exam room
Evacuate the room
TCandidates must stay seated until you have collected, and accounted for, all the question papers, scripts and any additional continuation sheets.
ell the candidates to stop writing.
Collecting in all materials – At the end of a test the invigilator must collect in all the test material from the candidates. Invigilators should ensure that the candidates have completed their details on the front and that no candidates leave with any papers or notes made during the test. No candidate should leave the room until the invigilator is satisfied that all material have been returned. Ensure that scripts are kept secure at all times until they are despatched
Completion of paperwork – The invigilator must double check all papers and that any incidents of note is recorded. All papers that were contained in the bag at the start of the external assessment should be returned and no paper should be kept, read or copied by the centre.
Returning papers – All test papers should be returned after the test. If the papers are not sent within the timeline then they MUST be stored securely until they can be returned.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 11 11 11

12. Traditional malpractice examples
Notes hidden
calculator back
the pencil case
inside Juice box
under the patch
under the sleeves
at a headscarf
in the back of the tie
in the back of the shoe
water bottle label cover
in a tampon
in fingernails
inside the breast
on the thighs
Tables 1 and 2 present taxonomy of the few threat categories that are known today. However, for each category, the number of actual methods that people use to cheat or steal may number in the hundreds.
Imposter
Someone to take a exam for you. In Chinese paid imposter dollars each to sit their exam in their name. Investigations led by the Police District of Western Pennsylvania. This problem was in increase and now it is considered a critical offence, up to 7 years.
Sneaking into hall before exam and attaching notes below desk
Developing “Mors code”, coughing, tapping desk etc.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

13. High tech examples of malpractice
Concealed ear-phones
Glasses (Small cameras installed)
Smart watches that look like digital watches
Using ultra violet pens and invisible ink pens
Hollowing out pens to write secret notes
Mobile phones
Calculators with memory functions used to hide answers
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 13

14. Cheating Test Ads and sites in Internet
There are a multitude of sites with step by step instructions and videos and different techniques for cheating.
There are numerous companies that advertise as sell cheating devices worldwide stating that they can be used in cheating exams.
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017 14

15. Train invigilators or…
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

16. Good invigilators are the solution
Dress smartly, wear the invigilator badge
Be aware of aware of emerging situations
Remain calm, vigilant, professional
Keep a written record of any relevant circumstances – who/what/when
An air of authority, yet relate with test-takers and staff
Maintain confidentiality
Provide a supportive atmosphere
Prepare! - Test Invigilators never make assumptions
Accuracy and attention to detail
Flexible, punctual
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

17. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
Conclusion
1. Tests administration is of much more important than it is afforded in language testing literature 2. Practical matters of test administration are critical to test validity - raise validity questions pertaining to fairness, ethics, law and how different cultures view testing and test takers 3. Training Proctors and Invigilators is indispensable in order to ensure a standardized test administration
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017

18. STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017
This is the end of the presentation. Thank invigilators for attending and remind them to read the Invigilator Handbook before they arrive for any exams. If you as a trainer have any questions or feedback on this presentation, please us at
STANAG 6001 Testing Workshop Skopje 5-7 Sept 2017