1 Chapter 18: Selection and training n Selection and Training: Last lines of defense in creating a safe and efficient system n Selection: Methods for selecting the right person for the job n Training and performance support: Training, tutorials, and job aids to transform the person into the right one for the job n Training program design

2 Selection methods (criteria) n High criterion related validity n Costs of training/retention guide emphasis on selection $1 million to train a fighter pilot n Type of task guides type of selection tool

3 Signal detection theory to define selection cutoff Job Performance Selection Test Score Reject Accept Miss Hit Correct Rejection False alarm Success Failure

4 Selection methods n Measures of cognitive ability (SAT test) n Measures of physical ability and psychomotor skills n Personality assessment n Work samples n Structured interviews (not unstructured interviews although commonly used)

5 Performance support n Emphasis on “just-in-time” training and avoid classroom instruction n Maximize task proximity Task proximity Far Near Product affordances Labels On-line help Help line Reference manual On-line tutorial Classroom training Job aids Training

6 Job aids vsTraining n Time available to look up information n Requires much information n Many difficult to learn steps n Task is performed infrequently n High turnover n Steps must be performed rapidly n Task performance hindered if interacting with aid n Person wishes to perform task unaided n Psychomotor or perceptual task that cannot be aided

7 Iterative cycle for system development Understanding Evaluation Design Error criticality Use patterns Goals, functions, and tasks User characteristics and expertise Operating environment Theoretical models Mental models and metaphors Heuristics and guidelines Dialog styles and conventions Previous designs Usability metrics Number and type of subjects Use scenarios and tasks Environmental and social context

8 Training program design: Instructional System Design (ISD) 1) Front-end analysis (Understanding) Task analysis Trainee analysis 2) Design and development (Design) Design concept Application of training theory 3) System evaluation (Evaluation) Design evaluation Program effectiveness

9 Training program design: Instructional System Design (ISD) 1) Front-end analysis (Understanding) Organization Analysis – looks at broad content of job Task analysis – knowledge, skills, and behaviors for successful performance (structured interviews, verbal protocol analysis, observation Trainee analysis (after task)1-prerequisite knowledge 2-demographics 3-attitudes toward methods Understand and determine how to redesign task and possibly training method with results

10 Training program design: Instructional System Design (ISD) 2) Design and development (Design) Design concept - there is almost always more than one solution – decide most beneficial for users Prototypes Formative evaluation (iterative) –Heuristic evaluation –Usability testing Full scale development

11 Training program design: Instructional System Design (ISD) 3) System evaluation (Evaluation) Finalize evaluation criteria Design evaluation Analyze the data –Has a change occurred in trainee task performance? –Is the change a result of the program? –Would change occur with other besides our sample group?

12 Training program design: Program types Method that provides the best training in the shortest time, leads to the longest knowledge and skill retention, and is the cheapest Text Lecture Audiovisual Computer Assisted Simulations On-the-job

13 Training program design: Concepts and issues Transfer of training (positive and negative) –% transfer =100*(Control time-Transfer time)/Control time Overlearning – practice beyond error free performance –Speed and accuracy –Uncommon but critical event –Decrease rate of forgetting Automaticity –Extended practice with consistently mapped responses –Reduced resource demand and increased ability to timeshare tasks –Low rate of forgetting (learning to ride a bicycle) Above real time training – 5 min into 2 or 3, low better than high fidelity Adaptive training – grows with user Guided training – training wheels, can’t make mistakes

14 Training program design : Concepts and issues n Procedural vs declarative knowledge n Type of knowledge trained depends on task analysis n Part task training Segmentation – non overlapping temporal components –One or more difficult segments –Backward chaining Fractionation – broken into components usually performed simultaneously –Gradually add components Segmentation preferred and successful when components are independent n Utility of part task training depends on task analysis

15 Power law of practice n T(N)=T 0 +b(N+N 0 ) -a n If asymptote is zero and learning starts on trial 0 then T(N)= b(N) -a n b=time for first trial and a=0.4

16 Key considerations for training and selection n Training and selection should be grounded in careful analysis of tasks n Provide feedback in a timely manner, but do not interfere with performance or safety n Training tools should be well designed so they do not distract from training objectives n Design program to enhance intrinsic motivation