PSY 6430 Unit 4 Job Analysis Wednesday 2/13: Task analysis described and lecture Monday, 2/18: Lecture Wednesday, 2/20: Exam

F19 Schedule Announcement The F19 schedule will be available for viewing on Monday 2/18 Graduate students can start enrolling on Monday, 3/11 You should figure out your schedule and be on your computer as soon as registration opens on 3/11!! Please spread the word!!

Another Announcement I will not be in class on 2/20, but your exam will be here, someone else will be here to administer the exam I will be at the BAAM conference I will be away from campus Wednesday, 2/20 – Friday, 2/22 I still hope to be able to return your exams on Monday 2/25 but I can’t promise that

Yet Another Announcement: ME1 ME1 will be on Monday, 3/18 It will cover Units 1-4 It will not cover Unit 5, even though you will have taken E5 Units 5-8 are covered in ME2 I will hand out the ME1 study objectives on Monday, 3/11

Yet Another Announcement: ME1 If you have missed an exam, you need to take ME1 If you have taken all the exams to date (that is, exams 1-4), you can use ME1 to replace your lowest score If your ME1 grade is lower than the grades you got on exams 1-4, I ignore ME1 That is, ME1 cannot hurt your grade; it can only help it If you have taken all of the exams to date and are satisfied with your grades, you get the day off

Now a bit of an apology You will be taking ME1 after the last day to withdraw without academic penalty Monday, March 18 Thus, you will not have your exam grades before you need to decide whether or not to withdraw The only other thing I could have done was to administer ME1 and E5 the same week and I didn’t want to do that Also, in the past, no one has withdrawn, so I thought it was better to schedule ME1 after that date than it was to schedule two exams the same week

Introduction: Task Analysis Project Job analysis is the bedrock of any selection procedure and is required from a legal perspective if the selection procedure results in adverse impact and is challenged in court There are several different types of job analysis procedures I am going to focus on one type, which is the type that the authors also focus on: The task analysis

Introduction: Task Analysis Project (but actually SO3 also) It is appropriate to focus on the task analysis because a task analysis can be used for either content validity or empirical validity And must be used for content validity Further, content validity is the most common type of validity procedure used to determine the job relatedness of selection procedures; thus the task analysis is the most common type of job analysis procedure (a bit of introduction before I talk about the project)

Introduction: Task Analysis Project To review a bit from last unit, there are two main types of validity procedures (validity generalization notwithstanding – I will deal with that a bit later) Content validity Empirical validity, which is also called criterion-related validity

Introduction: Task Analysis Project Content validity You determine the job relatedness of the selection procedure using (only) the judgments of experts Because of the difficulty of doing this correctly – there is a lot of room for error – the procedures for the job analysis are “written in stone” by the Uniform Guidelines and must be followed exactly (if, of course, your selection procedures are legally challenged) Empirical or criterion-related validity You determine the job relatedness of the selection procedure by conducting an empirical study and analyzing the results statistically You have a lot more leeway with respect to the job analysis procedures because of the emphasis is on the statistics (we will be covering the types of empirical studies in U6; next slide project description)

Task Analysis Project Due Monday, April 22 (final exam week) Due Monday, April 8 if you want your grade before ME2 Worth 70 points (Units 9 and 10) You must complete this project or you will receive a zero That is, you cannot use ME2 to replace this assignment

Task Analysis Project Lateness, per the syllabus I will deduct 7 points (10%) if you hand in the project after 5:00 pm on Monday, April 22 but before the end of Tuesday I will deduct 14 points (20%) if you hand in the project on Wednesday, April 24 You will not receive any points if you hand it in after Wednesday Get started now: This is a time consuming project

Task Analysis Project A task analysis is the first step in the development of a task inventory (job analysis questionnaire) I am not going to have you develop the questionnaire or get ratings for the task statements, KSAs, or other WRCs Project consists of just the first step in the process - the identification of tasks, KSAs linked to each of the tasks, and other WRCs (the WRCs are very important for ADA) Four models in the course pack from previous classes WMU career and student services: Julie Slowiak Firefighter: Garrett Warrilow Branch office manager, Investment firm: Megan Ireland Automotive master mechanic: Thor Flosason

Task Analysis Project Resource on the web that will make this project a lot easier and a lot less time consuming for you O*NET Occupational Information Network – Dept. of Labor Web site address is in the instructions To start with, go to O*NET and use “Bartender” as the example position Detailed instructions are in the project description

Task Analysis Requirements You may conduct the analysis for one job incumbent The job may be part-time or full-time That said, part of my grading criteria will be based on the complexity of the job If you select a very “simple” job, it will be harder for you to get full credit You cannot do the task analysis for a job you currently have or have had in the past I want you to have the professional experience of collecting information about a job that you are not familiar with - this is much harder than you think

Task Analysis Requirements Task statements You should have 15-30 task statements depending upon the job One problem you will encounter is how narrow or broad they should be KSAs linked to each task statement Other WRCs, specifically, per O*Net Work activities: Physical activities and requirements (required for ADA) Work Context: Environmental conditions (also good for ADA) Work Styles: Typical working incidents (again good for ADA) This is where many job analysts put things like “persuading individuals, working under stress, getting along with others” that are critical for ADA Education and licensing or certification requirements O*Net has other sections as well: do not include those (don’t forget job attendance -getting to work on time, and regular attendance, if that is required - ADA)

Task Analysis Steps: Before Beginning Task statement format The task statement format must be exact See project instructions for the format provided by GFB and SO6 for details My lecture tonight Worksheets provided in course pack List of potential verbs to use when writing task statements in course pack – again see SO6 Caution! The task statement format differs in O*NET O*NET is the end result of a task inventory, not the starting point (Tasks MUST BE EXACT FORMAT)

Task Analysis Steps: Before Beginning KSAs See project instructions for the format provided by GFB See project instructions for the difference between skills and abilities Work activities, work context, work styles, and licensure/certification requirements See O*Net and task analysis models

Task Analysis Steps: Before Beginning Planning the analysis Plan to meet with the job incumbent at least three times, for about 1.5 – 2 hours each time Preparing for the meetings Before the meeting, obtain any written documentation that exists about the job Job descriptions, organizational chart, etc. Locate job title/position using O*Net (or a similar position)

Task Analysis Project: The Meetings First meeting: Task statements only (1.5-2 hrs) Ask the incumbent to identify/state just the major aspects or areas of his/her job After you have that list, go back to the first major area and ask for the tasks in that area Move onto the next major area, etc. (different interview procedure from GFB;; Also strongly recommend that you record the interview – never going to remember everything, miss things ; love to talk about their jobs)

Task Analysis Project: The Meetings Second meeting (1.5-2 hours) Confirm accuracy of task statements, obtain KSAs, and work activities, work context, work styles and education/licensure/certification (time permitting) Before meeting with the incumbent, type up a list of the task statements Identify any missing components to the tasks that you need to ask about Have the incumbent review the task statements for accuracy and completeness Take each task statement one at a time and ask the incumbent what KSAs are required to perform that particular task Before doing this, reread “how to write KSAs”

Task Analysis Project: The Meetings Third meeting: Review of the entire document Type up the entire document and have the job incumbent review it If revision is required, then you should ask the incumbent to review it a fourth time, after revision Some students have done this step remotely which is fine – but it is not good to do the first two steps remotely

Task Analysis Project: Format See models in the course pack Description of the company and position Include the name of the position in the company and the corresponding O*Net position name, if there is one I should be able to get a very good overall view of the position and its main purpose without having to refer to any other document Description of the methods/steps you used to develop the task analysis. How many times did you meet with the incumbent? What did you do during each meeting? How long was each meeting?

Task Analysis Project: Format Task Statements with KSAs linked to each task statement KSAs should be numbered as in the models and in GFB, page 60, Figure 3.4 Work Activities Work Context Work Styles Education/License/Certification Requirements

Task Analysis Project: Some Grading Criteria Complexity of the job: if you choose a “simple” job that has fewer than 15 tasks, I may deduct some points All sections included You will lose points on the first section if I cannot understand the nature of the position from your description alone Everything in your analysis must be written clearly enough so someone who knows virtually nothing about the position can understand it and “picture” what the incumbent does Task statements and KSAs must be written/formatted per the study objectives Grammar and spelling!! You will lose credibility in a professional setting for grammar and spelling errors Page numbers!! (I reserve the right to deduct points for other things such as formatting errors, unprofessional look/style, etc.

Task Analysis Workshop: Optional Wednesday, March 27, I have scheduled an in-class task analysis workshop; that is, attendance is optional The purpose is for you to ask questions about the task statements and KSAs you have developed as part of your analysis About a week or so before the workshop I will ask you to let me know if plan to attend and develop a schedule so I can work with individual students or small groups of students That way, you won’t have to wait too long while I am working with other students (some students find that they don’t need extra help – educational objectives; just in case..)

Task Analysis Workshop: Optional I will only review the writing/formatting of your task statements and KSAs at the workshop That is, I will not read your entire analysis or edit for spelling, grammar, or page numbers You should come to the workshop with specific questions about general writing/formatting of the tasks and KSAs or questions about specific tasks and KSAs I will not read an entire draft of your analysis before you turn it in Unfortunately, in the past, some students have taken advantage of this and handed in very, very rough drafts – not something they would give an organizational client You are graduate students: spelling, grammar, etc. is your responsibility – if you need help with that, ask a friend or the WMU’s writing center to review it

Questions about the project?

SO1: Legal Guidelines Job analysis is mandatory and must be for the job for which the selection procedures are used Analysis of the job should be in writing Job analysts should describe the job analysis procedures used in detail (text lists 9 standards for job analyses that have been culled from various court decisions; I am asking you to learn the first three; reason for 2&3 – as you have seen, these cases can span years – and the original job analyst may not be at the organization anymore – also courts must be able to review everything the job analyst does)

Relationship of job analysis to type of validity procedure Type of Job Analysis Content validity Empirical validity Validity generalization Task analysis only Any type including a task analysis Any type is OK, should be the same as the type used by the other company (As I indicated earlier, there are some rules regarding job analysis and the type of validity procedure you use; you do not have any flexibility If you use content validity legally, but you do have flexibility if you use empirical validity because of the use of stats; hold off a bit on validity generalization – that is a very complicated legal issue right now; while accepted by selection specialists, still not accepted by the courts)

For the exam: Add this to the SOs! Discrepancy between the professional and the legal requirements regarding job analysis Job analysis is legally required for all types of validity procedures (if your procedures are challenged) It is not, however, required professionally for empirical validity (it is required for content and validity generalization) The proof of job-relatedness rests with a statistically significant correlation between test scores and job performance measures If you administer a selection procedure, correlate the scores with job performance measures and the correlation is significant, your tests are job related – you don’t need a job analysis to prove that (usually, if something is considered professionally sound it is considered legally sound and vice versa; However the use of a job analysis for empirical validity is one of those issues where what is professionally sound is not considered legally sound; the other major one is validity generalization)

Introduction: Job Analysis Procedures The authors first talk about a job analysis interview But that is rarely used by itself Used by itself only when there are only a few job incumbents in the position Usually combined with the task inventory and used only as the first step in the process (like you will be doing for your project) The task analysis (and task analysis questionnaire) is the most popular type of job analysis The authors also talk about the Critical Incidents Technique, but I am not going to focus on that Critical incidents technique focuses on excellent and poor ways of handling important job-related situations/scenarios It’s great to use for developing interview questions and job simulation tests but does not result in a comprehensive job analysis (always started with the exemplary performers; great to use after a comprehensive analysis to develop some of the instruments/methods for selection; main usefulness after a comprehensive analysis – supplement; face validity)

SO6: Task Statements The next few slides deal with how to write task statements SO6 asks you to write two task statements for a job you have had and bring them to the exam to hand in This is an unusual SO for me which is why I am pointing this out I am not sure yet how many points I will assign to them but I am likely to assign 3-5 points I don’t care if you want to work together to verify the correct format, but you each need to come up with your own (original) task statements That is, I don’t want duplicate (the same) task statements from different students

SOs 5&6: Task Statement Format What activity is performed? Use a single present tense action verb - verb ends in an “s” To whom or what? The object of the verb Why? What is produced - what is the expected outcome? How? Using what materials, tools, procedures, guidelines, or equipment (moving to Sos 5&6; four components of a task statement; two tasks for a job you have held; no choice in the format; (main reason for that is that you are trying to eliminate as much error in that inferential leap between job information and Identification of the KSAs and other WRCs – subject to error – standardized, required format)

Task Statement Examples Welfare Eligibility Examiner What activity is performed? Asks questions, listens, and records answers To whom or what? Asks and listens to client, records on eligibility form Why? In order to determine eligibility for food stamps How? Using eligibility form, eligibility criteria in manual, interviewing techniques (from GFB; note more than one action verb that are all related to the same WHY; Why is the most frequently omitted component when students write task statements; clean up the task statement – next slide)

Task Statement Examples Welfare Eligibility Examiner Clean it up Asks the client questions, listens, and records answers on standard eligibility form to gather information from which the client’s eligibility for food stamps can be determined using knowledge of interview techniques and eligibility criteria. (notice, several different verbs, actions that go together for the same “in order to”; often a question students have, next Slide another example)

Task Statement Examples Welfare Eligibility Examiner What activity is performed? Determines eligibility To whom or what? Of applicant Why? In order to complete client’s application for food stamps How? Using regulatory policies as a guide (another example - same job; once they obtain information from the client, then they must decide whether the person is eligible using policies; next slide different topic)

SO8: Problems with Job Interview as only source (this should not surprise you) Lack of standardization across job analysts and not practical for large numbers of respondents Likely to get different information from different analysts Not practical for large numbers of incumbents because it’s one-on-one (2-3 hrs at least) (again, rarely, if ever used alone; I have just asked you to list any three of the ones given; more on next page)

SO8: Problems with Job Interview as only source (this should not surprise you) Once you have more than 5-10 job incumbents, it gets very time consuming and effortful, and hence expensive (not cost efficient) Legally, you must have a representative sample stratified by geographical region (if relevant), and protected classes, thus you may end up interviewing more incumbents than would otherwise be necessary Travel costs as well Distortion of information Interviewees may exaggerate the activities and responsibilities to “look good” or because they believe they may get a pay raise Another reason: job incumbents might do different things in different locations, i.e., perform idiosyncratic tasks that are not actually required for the position (so, bottom line is that you should not use this alone unless only have 5-10 incumbents)

SO9: Task Statement Rating Scales The Uniform Guidelines discusses the technical standards for a job analysis if it is used for content validation All of the following four rating scales must be included for task statements Frequency of task performance Task importance or criticality Task difficulty Whether the task can be learned on the job within a relatively short period of time (within 6 months) Because of ADA, you should also add: Is it an essential job task? (KSA rating scales next)

SOs 10: KSA Rating Scales All of the following rating scales must be included for KSAs (if, again, used for content validation) Is the KSA necessary for successful job performance? Is the KSA required upon entry to the job or can it be learned within a relatively short period of time (within 6 months) How difficult is it to obtain the KSA? What type of education/experience/licensure/certification is necessary? (as you can see, the task inventory/questionnaire becomes quite extensive; 4 scales for tasks at least 3 for KSAs)

SO11: Obtaining names of the job incumbents I strongly disagree with the authors about obtaining names on questionnaires Employees tend to be very nervous and concerned about how the information will be used When I have interviewed individuals, even one-on-one, I have almost always had to repeatedly assure them that they would not be identified - rather the information would be combined with the information of others, with no identifiers Most employees just don’t trust the “corporate office,” HR or their bosses not to use the information against them (evaluation, downsizing, elimination of job) You will probably be interviewing people you don’t know (you have no established relationship with these individuals and if you are from “corporate” you are immediately distrusted)

SO11: Obtaining names, cont. You should handle this just like you handle a participant list for research You should create a master list with names and code numbers that correspond to your stratified sample Geographical location, sex, race, etc. Keep it under lock and key - you are the only one who should have access to it (or if you are not in charge of the analysis, the person that is) Put the code numbers on the questionnaire, so you know whether you have a representative sample when the questionnaires are returned (Why codes, next slide)

SO11: Obtaining names, cont. (this slide NFE) Why codes on questionnaires? There is a legal requirement to have a representative sample for your job analysis The legal requirement pertains to who participated in the job analysis, not simply who you sent questionnaires to If you don’t get the sample you need, contact everyone in the underrepresented categories, with the standard, “We are contacting you because we recently sent you a questionnaire to complete. If you have not completed the questionnaire, we would greatly appreciate it if you would do so. If you have, please accept our thanks, and ignore this message.”

SO12: Three criteria used to determine how critical a task is Mean rating – the higher the mean, the more important the task For example, a minimum mean rating of 4.0 on a 7-point rating scale Standard deviation – the lower the SD, the higher the degree of agreement among workers For example, an SD of 1.00 or lower Percentage of respondents who perform the task –most employees must perform it For example, 75% or more must perform it This is really, really an important one to include along with the mean!! (as per the sos, I am having you learn this because people in our filed, BA, in particular, don’t really have a good sense about how to interpret survey data: rely only on means, very misleading)

SO13: Advantages of the Task Inventory It is an efficient way to collect data from a large number of individuals, particularly if they are geographically dispersed Data can be easily quantified due to the rating scales

SO14: Disadvantages of the Task Inventory Development is time consuming and expensive Motivation to complete the questionnaire can be a problem because it is often very long due to the rating scales that must be included Ambiguities and questions may arise when individuals complete the form that can’t be dealt with, which may result in inaccurate and less reliable information

SO15: Court case related to job analysis, the two implications: intro Kirkland v NY State Department of Correctional Services, 1974 The litigated test was a promotional exam for correction officers The passing rates for whites was 31% while the passing rate for blacks was 8% Adverse impact had been demonstrated thus the burden of proof shifted to the organization Organization defended the exam on the basis of content validity The court ruled in favor of the plaintiffs ruling that The job analysis was not adequate, but more importantly because job analysis had been stressed in other cases The process used to construct the test and the content of the test was not adequate (very important case related to job analysis when used for content validity; not in text)

SO15: Court case, the two implications You must document the process used to construct the test (not just the process used to conduct the job analysis) Did you develop a matrix that linked the KSAs to specific items on the test? Who participated in that process and when did those meetings occur? Remember, court cases often occur years after the development of the test and you are simply not going to remember the dates of the meetings, who attended, or the outcome of those meetings (first one this slide, second next slide)

SO15: Court case, the two implications You must document and be prepared to defend each question on the test and the weight given to various questions/groups of questions We haven’t yet talked about this, but if you use content validity it is much, much preferable and legally defensible to construct/write the test yourself rather than selecting an off-the-shelf test

SO18: Number of Job Analysis Participants Table 3.3 on page 74 is really cool It displays the approximate numbers of participants you need for a job analysis for various numbers of employees in a position for an 80% confidence level and a 90% confidence level The main point is that the smaller the number of incumbents in a position, the larger the percentage of incumbents you need to include So, if you are after a 90% confidence level, for example: If there are 20 incumbents in a position, you need a sample of 17 If there are 100 incumbents in a position, you need a sample of 50 If there are 500 incumbents in a position, you need a sample of 80 One thing the authors don’t address here is the sample size you need for an empirical validity study - (in both organizations I worked for, when doing a job analysis, we actually sent the task analysis to every single incumbent. We felt the expense was worth it. NY court system – very litigious)

SO19: Order of the analysis Start out with having SMEs identify the major areas of the position (which actually isn’t necessary for the task analysis, but usually makes it much easier for participants) Then you have the SMEs identify the tasks that fall into each of those major areas After that, you go back to each task statement and have the SMEs identify the KSAs and WRCs that are required to perform each task (the authors talk about the order of the analysis when using SME workshops, but the same holds true for interviews: and it is the order I have told you to use in your project)

SO19: Order of the analysis You should not ask SMEs to identify one task, then identify the KSAs and WRCs for that task, move onto the next task and have them identify the KSAs and WRCs for that task, etc. The reason for that is only a practical reason: It is much easier for SMEs to focus on one level of analysis at a time rather than trying to switch back and forth between tasks and KSAs/WRCs

SO21: VERY IMPORTANT! You cannot use content validity to determine the job-relatedness of abstract personality traits: The Uniform Guidelines prohibits that The reason for that is the inferential leap you must make from a task statement to the KSAs/WRCs Content validity is too prone to error to “leap to” unobservable, abstract characteristics

SO21: VERY IMPORTANT! If you want to determine the job-relatedness of personality traits, you have two choices: Do an empirical validity study Rewrite the ”abstract” personality traits into observable, concrete behaviors/outcomes I love this section: it is excellent advice because there are many times when you simply cannot do an empirical validity study I will deal with this later in the course, but to do an empirical validity study, you need to end up with a statistically significant relationship between the scores on the selection procedure and some measure of job performance In order to do that, you need (a) large sample sizes of at least 50, but preferably 100+, and (b) a valid, reliable measure of job performance Most organizations don’t have one or the other so need to use content validity

SO22: Rewriting personality traits Rewriting personality traits in terms of more observable, concrete behaviors and outcomes is something we, as behavior analysts, should understand completely – and be able to do This is something our consultants do all the time when dealing with clients, who typically start with personality traits in attempts to measure performance

SO22: Rewriting personality traits What does dependability mean? How do you know it when you see it? Is never late, always shows up for work on time, does not miss work without an excused absence (text) Follows instructions without follow-up, completes assignments as asked, follows company policies and rules without being asked Others??

SO23: Physical abilities as WRCs When writing WRCs that involve physical abilities, it is very important that (a) they are essential to the job, and (b) are very specific Why? ADA Review the examples on page 85 very carefully: I will hold you to these standards in your project Ability to read a voltmeter dial from a distance of 5 feet Ability to drive a forklift, grab pallets holding four 55-gallon drums, and then transport the load, lift, and place on storage benches Ability to hear and record equity purchase and sell orders in a loud (75-100 decibel level) commercial area (last slide)

That’s All Folks Questions on U4?