1. Training & Instructional Design Needs Analysis Lecture b This material (Comp20_Unit2b) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.

2. Needs Analysis Learning Objectives 2 Compare the methods used to collect data in a needs assessment (Lecture b) Select appropriate data collection methods to meet different needs analysis situations (Lecture b) Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b

3. ADDIE 3 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Analysis Design Develop Implement Evaluate

4. Data Collection 4 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b The needs analysis stage of your training is likely to be successful if quality methods are used to collect data Different workplace and training situations require different data collection methods

5. Strategy 5 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Identify the purpose and audience Select your method and develop instrument Pilot test your instrumentRevise and retest if neededImplement your instrument

6. Identify the Purpose & Audience 6 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b For example, organizational analysis Need information about organizational goals, available resources, constraints, and support By assessing the attitudes and perceptions of staff and clinicians, and reviewing company documents Audience to target Senior clinicians who can speak accurately about the organization’s goals All employees? A cross-section of the workforce to get a holistic view

7. Data Collection Methods 7 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b SurveyInterview Focus/Discussio n Group Observation Document Review

8. Surveys 8 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b A way to get lots of information quickly and/or easily from a group of people individually Can be in-person, written, or by telephone Description and Purpose

9. Surveys 9 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Advantages Can reach large numbers of people Questions can focus responses as either very specific or open-ended Can establish relationships among indicators. Can complete anonymously Inexpensive to administer Easy to compare and analyze Many sample questionnaires already exist Disadvantages No in-depth responses on issues Does not allow respondents to exchange ideas Limited to specific questions May not reach people who are hard to reach Wording can bias client's responses Are impersonal May not get full story

10. Survey Writing Tips Use simple languageMake the survey shortAvoid double negativesAvoid open-ended questionsDon't write leading questionsAvoid double-barreled questionsOffer an “out” for questions that don't apply Table 1.1 10 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b

11. Interviews 11 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Formal or informal discussion with an individual to gather in-depth information on specific topics When want to fully understand someone's impressions or experiences or learn more about their answers to questionnaires Description and Purpose

12. Interviews 12 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Advantages Can get in-depth information Questions can be flexible and be tailored to each person Can get spontaneous responses Can get honest opinions, individual perspective Can get to questions that people are uncomfortable answering in a group Can get ideas or opinions that never occurred to you Can be done in a setting familiar to participant Disadvantages Time consuming to administer Interviewers needed and can be costly Interviewer can bias client's responses Not efficient for reaching a large number of people Not useful if trying to get answers to specific questions from people or get consistent answers

13. Interview Guide Types of QuestionsDescription Experience QuestionsDeals with actions or experiences Opinion QuestionsDeals with opinions or beliefs Feeling QuestionsDeals with emotions, feelings, like happiness, fear, anxiety, and confidence Knowledge QuestionsDeals with factual information, not opinion or feeling Sensory QuestionsDeals with what respondents see, smell, hear, touch, or taste Background QuestionsStandard questions that describe the respondent's background Table 1.2 13 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b

14. Focus groups 14 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Small group discussions around a set of questions guided by a moderator To explore a new topic, get in-depth knowledge, ask questions about ““how and why,”” understand a gap in understanding between groups of people, or prepare for a survey Description and Purpose

15. Focus Groups 15 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Advantages Participants can build on each other’s comments Can be quicker than a survey Can be less costly Can reach more people than interviews Can hear a variety of ideas and opinions Participants can take advantage of the shared experience Disadvantages Need a good facilitator for safety and closure Difficult to schedule 6-8 people together A group or an individual can dominate the session Information gathered may not represent the larger community Smaller numbers reached Can be hard to analyze responses

16. Observation 16 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Observer records an expert performing job natural or structured setting To gather accurate information about how a program actually operates, particularly about processes Description and Purpose

17. Observation 17 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Advantages Can view operations of a program as they are actually occurring Can adapt to events as they occur Disadvantages Can be difficult to train observers Can be difficult to interpret seen behaviors Can be complex to categorize observations Can influence behaviors of program participants Can be expensive

18. Document Review 18 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b When want impression of how program operates without interrupting the program Analyze budget documents, quality control documents, goal statements, evaluation reports, scheduling and staffing reports, or other documents for existing problems, such as testimonials from other organizations Description and Purpose

19. Document Review 19 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Advantages Can get comprehensive and historical information Doesn't interrupt program or client's routine in program Information already exists Disadvantages Often takes a lot of time Information may be incomplete Need to be clear about what looking for Not a flexible means to get data Data restricted to what already exists

20. Pilot Test Your Instrument 20 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Identify and eliminate problems that might occur Keep practice exercises realistic Try out survey or interview tool on a few persons Pilot test if possible Make changes prior to actual data collection If changes are drastic, another pilot test again and/or re-train data collectors Pilot test results

21. Implement Your Instrument 21 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Here are some guidelines to keep in mind during implementation Keep survey instruments short, one or two pages if possible Provide directions on how to return the survey. Prepare online surveys is possible Always let people how you will use the data Always find a way to report the information back to the participants Offer incentives, such as like lunch for focus groups and interviews

22. Implement Your Instrument 22 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b Confidentiality should always be foremost in your mind Inform respondents about your confidentiality guidelines when you begin data collection Make surveys anonymous Assign fictitious names to respondents or provide answers in categories rather than by respondent

23. Case Example Meaningful Use Clinical Quality Measure: The percentage of patients 18 years of age and older who were current smokers or tobacco users, who were seen by a practitioner during the measurement year and who received advice to quit smoking or tobacco use or whose practitioner recommended or discussed smoking or tobacco use cessation medications, methods or strategies. 23 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b

24. Needs Analysis Summary Choose the method that is most appropriate for your audience and the type of information to be collected. Ultimately, the decision of data collection method will take into account the validity of the responses collected, balanced with budget and efficiency concerns. 24 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b

25. Needs Analysis References – Lecture b References 1.Clark DR. Bloom’s Taxonomy [monograph on the Internet]. Big Dog & Little Dog’s Performance Juxtaposition; c2004 [cited 2010 Jun 21]. Available from: http://www.nwlink.com/~donclark/hrd/sat.html.http://www.nwlink.com/~donclark/hrd/sat.html 2.Clark DR. Instructional System Design (ISD). Big Dog & Little Dog’s Performance Juxtaposition; c2004 [cited 2010 Jun 21]. Available from: http://nwlink.com/~donclark/hrd/ahold/isd.html.http://nwlink.com/~donclark/hrd/ahold/isd.html 3.Carkhuff RR, Fisher SG. Instructional systems design: volumes I & II. Amherst, MA: Human Resource Development Press; 1984. 4.Carliner S. Training Design. Danvers, MA: American Society for Training and Development; 2003. 5.Gagne RM, Wager WW, & Golas K. Principles of Instructional Design (5th ed.). California: Wadsworth Publishing, 2004. 6.Reigeluth CM. Instructional-design Theories and Models: A new paradigm of instructional theory. New Jersey: Lawrence Erlbaum Associates, Inc, 1999. Charts, Tables & Figures 1.1 Table: Zimmerman, J. (2010). Survey writing tips. Department of Biomedical Informatics, Columbia University Medical Center. New York, NY 1.2 Table: Zimmerman, J. (2010). Interview guidelines. Department of Biomedical Informatics, Columbia University Medical Center. New York, NY 25 Health IT Workforce Curriculum Version 3.0/Spring 2012 Training & Instructional Design Needs Analysis Lecture b