1. Pre-Employment Testing Software
Using Emergency Response scores from the Data Entry MT and Data Entry MT (Audio) test modules for hiring purposes
Pre-Employment Testing Software
Click on the screen, use the scroll bar, or press the right arrow key to view the next slide. Press the left arrow key to go back to the previous slide.
Note: No audio accompanies this presentation
Copyright 2007 © Biddle Consulting Group. Inc.

2. Please send comments or suggestions to info@criticall911.com
This presentation is provided to CritiCall users as part of their CritiCall Elite Service Plan Membership
Please send comments or suggestions to
CritiCall® is a registered trademark of
Biddle Consulting Group, Inc.

3. Please keep in mind…
This training session applies ONLY to the Emergency Response scores associated with the Data Entry MT and the Data Entry MT (Audio) test modules
The 60% minimum scores recommended for the Emergency Response portion of these two modules are based on a national survey that collected input from over 150 experienced dispatchers and calltakers from a wide variety of PSAPs from around the country
Even so, to insure fairness job experts from your agency should also endorse this 60% minimum score following the procedures described in this presentation before it is used for selection purposes

4. Will an automated version of this process become available in the future?
We will be including an automated process for using the Emergency Response scores from the Data Entry MT and Data Entry MT (Audio) test modules in a future upgrade to CritiCall
In the meanwhile, we wanted to offer you the option to use these scores manually

5. IMPORTANT
Testing must be performed on CritiCall Version or higher if you wish to use the Emergency Response scores from these two test modules
Use Emergency Response scores from ONLY the Data Entry MT and Data Entry MT (Audio) test modules for making hiring decisions

6. This update will not work with CritiCall versions prior to 4.0
If you have not already installed Version or higher installed, go to the web page to download the update
This update will not work with CritiCall versions prior to 4.0

7. Before you use the 60% minimum score for the Emergency Response portion of these two test modules
Have 7 to 10 of your subject matter experts take the Data Entry MT and/or Data Entry MT (Audio) test module with the Validation Wizard turned “on”
Have your Subject Matter Experts (SMEs) read and understand the text on the following page before they take these test modules with the Validation Wizard turned “on”
The wording on the next slide is also available in printed form from the CritiCall User’s Group Web pages Tips section

8. “After you have taken the Data Entry MT and/or Data Entry MT (Audio) test modules you will be asked to indicate a minimum Keystrokes-per-Hour (KPH) score that a minimally qualified applicant should obtain in order to pass and be considered for hiring, promotion, or transfer into the job you are validating the test for (e.g., dispatcher, calltaker, etc.). When recommending a cutoff score please keep in mind that this agency will also require the test taker to correctly respond to a minimum of six out of ten of the emergency response decision-making items while simultaneously entering the data during these two test modules (and only these two modules).”

9. You should you use the 60% minimum for the Emergency Response portions of the Data Entry MT and Data Entry MT (Audio) only after your job experts have suggested a Keystrokes-per-Hour cutoff score using the process described on the previous two slides
The steps to using the Emergency Response Scores from the Data Entry MT and the Data Entry MT (Audio) test modules are as follows…

10. First, make certain that the cutoff scores for all of the test modules in the AutoTest Code you will be using are appropriately set using the “Set Cutoff Scores” feature in the CritiCall Administrator program
Note: You cannot set a cutoff score for the Emergency Response portion of testing using this feature. However, setting the cutoff scores for other portions of the CritiCall testing will make it much easier for you to more quickly determine who passes or fails the test.
Next, print out a “pass/fail” Candidate Score Report for the test taker to determine whether that test taker passed or failed the non-Emergency Response portions of the AutoTest Code
See the Set Cutoff Scores training presentation in the Training section of the CritiCall User’s Group web pages for details about setting cutoff scores and printing pass/fail reports.

11. Question: Why should I print out and check the “Final Status” section of the Pass/Fail version of the Candidate Score Report before I manually check the Emergency Response scores for these two modules?
Answer: If the “Final Status” section of the Pass/Fail version of the Candidate Score Report indicates that the job candidate has already “failed” the CritiCall test, there is no need for you to manually check the Emergency Response scores for these two modules since passing or failing that portion of the test will not undo the previously-determined failure.

12. However, if the Final Status section of the Pass/Fail version of the Candidate Score Report indicates the test taker has “passed” the CritiCall test thus far, print a version of the test taker’s Candidate Score Report without the Pass/Fail information on it
Here’s how…

13. Click here
To print a Candidate Score Report for test takers without the Pass or Fail status automatically indicated, click on the “Candidate Score Report” link on the CritiCall Administrator’s Screen

14.  
First, select the candidate(s) whose Candidate Score Report you wish to print by clicking on the check box by their names

15.  
Do not check here
Click here
Then, to create a Candidate Score Report that indicates test scores, but does NOT automatically indicate a test taker’s pass/fail status, do NOT check () the “Pass/Fail Report” option on the Candidate Score Report dialogue box before clicking on the “Print Report…” button

16. (This is the report version you need for the next steps!)
A printed Candidate Score Report with the test taker’s scores, but without the Pass/Fail indictors, looks like this
(This is the report version you need for the next steps!)
50 %
80 %

17. To use the 60% minimum for the Emergency Response scores for the Data Entry MT and/or Data Entry MT (Audio) test modules, first find either (or both) of those test modules listed on the Candidate Score Report
50 %
80 %

18. 50%
80%
Next, go down the Emergency Response Score column to view the Emergency Response scores from the Data Entry MT and/or Data Entry MT (Audio) test modules

19. IMPORTANT
Use Emergency Response scores from only these two test modules for hiring purposes
50%
80%
Emergency Response scores from other test modules do not have sufficient test reliability to be used for selection purposes

20. Do not to use Emergency Response scores from any of the other test modules for hiring purposes
50 %
80 %
Emergency Response scores from other test modules do not have sufficient test reliability to be used for selection purposes

21. The next slide demonstrates how to make this comparison
Manually compare the score the test taker received on the Emergency Response portion of these test modules against a 60% minimum
If the test taker scores 60% or higher on the Emergency Response portion of these test modules, they pass the Emergency Response section of the test
f the test taker scores less than 60% on the Emergency Response portion of either of these test modules, they fail the Emergency Response section of the test
The next slide demonstrates how to make this comparison

22. Compare the test taker’s Emergency Response scores for these test modules to a 60% minimum score
The test taker “fails” if he or she scores lower than 60% on the Emergency Response portion of either the Data Entry MT and/or Data Entry MT (Audio) test modules
50% - Failed this section
50%
80%
80% - Passed this section
In this example, since the test taker scored less than 60% on the Emergency Response portion of the Data Entry MT test module, she “failed” that portion of the CritiCall testing process

23. Options for using the 60% minimum score
FAIL
SLOW
PASS
Ignore the Emergency Response scores for these two modules entirely
In other words… don’t pay any attention to the scores on this portion of the test
Use the 60% minimum score only for diagnostic purposes
Compare the scores for information purposes, but do not pass or fail a test taker based on their scores on the Emergency Response portion
Fail everyone who scores lower than the 60% Emergency Response minimum score on both the Data Entry MT and the Data Entry MT (Audio) test modules.
Fail everyone who scores lower than the 60% Emergency Response minimum score on the Data Entry MT, or the Data Entry MT (Audio) test modules, or both.

24. The key to minimizing liability is to treat all test takers who score below 60% on this portion of the test in the same way

25. Reminder…
Do not use Emergency Response scores from test modules other than Data Entry MT and Data Entry MT (Audio)

26. Do not use the Emergency Response scores from these two test modules if testing was conducted with a CritiCall version prior to 4.0.3

27. The following are several questions we are frequently asked about Emergency Response scores

28. How does the Emergency Response portion help employers determine if a job candidate has the ability to multi-task?
While responding to Emergency Response items is relatively easy when not performing other tasks, it becomes much more difficult when it must be performed at the same time when entering data. This is the very nature of multi-tasking.
Scientific research advises us that having job candidates perform two job-related tasks at the same time appears to be the best way to determine a person’s ability to multi-task.
CritiCall is unique in that it measures the ability to perform more than one computerized job-related task at the same time. This is a vital element for testing the ability to multi-task since most of a dispatcher’s work is performed using a computer.

29. What about the Emergency Response scores from other test modules?
The Emergency Response score from the Call Summarization 2 MT test module is combined with that module’s data entry and multiple-choice scores to create a composite primary score
The Emergency Response score from that module is used to help create a combined primary score for that module
This is why the Emergency Response column for the Call Summarization 2 MT module is blank on the non Pass/Fail version of the Candidate Score Report
The Emergency Response scores from the remaining current CritiCall test modules currently do not contain sufficient testing reliability to be used for selection purposes
Most other test modules only have two Emergency Response items associated with them, which means a person could only score 0%, 50%, or 100%. This makes it virtually impossible to reliably differentiate between acceptable and not-acceptable performance.
Emergency Response items during other CritiCall test modules are designed to act as “distractions” during testing to more accurately simulate the multi-tasking complexity of the job
We are working at adding more multi-tasking throughout the test as appropriate

30. Why is it acceptable to use a nationally developed minimum score for the Emergency Response portion of these two test modules, but not for other areas of testing?
The 60% minimum for the Emergency Response portion of the test merely sets a minimum hurdle that a test taker must achieve while performing another task on which they are graded
This is similar having a physical ability test that requires a job candidate to run a specified distance within a certain amount of time. The distance is determined in advance (just as the 60% minimum has been determined in advance in this case). Once the distance is determined the employer’s Subject Matter Experts decide on the time limit in which the specified distance is to be run (similar to the SMEs setting a Keystrokes-per-Hour cutoff score for these two test modules after being informed the test taker must also correctly respond to 6 out of 10 Emergency Response items).
Thus, the 60% Emergency Response minimum in this instance is fundamentally different from the cutoff scores used throughout the CritiCall test

31. Finally…
Due to challenges outside of our control to the CritiCall program that can occur when it is installed on a wide variety of computer systems, it is always a good idea to manually double-check the pass/fail status of job seekers who have taken the CritiCall test against the cutoff scores you have chosen to use for all of the CritiCall test modules.

32. We hope you have found this presentation to be helpful
Pre-Employment Testing Software
Your suggestions and comments welcome at
Copyright 2007 © Biddle Consulting Group. Inc.