Presentation on theme: "Introduction to Designing Needs Assessment Surveys"— Presentation transcript:
1Introduction to Designing Needs Assessment Surveys An AEA Mini Workshop presented byJames W. Altschuld The Ohio State UniversityYi-Fang Lee National Chi Nan UniversityJeff White University of Louisiana at LafayetteHsin-Ling Hung University of North DakotaAnnual Meeting of the American Evaluation AssociationNovember , 2010San Antonio, Texas
2Objectives/AgendaObjectives Understand Needs Assessment Learn where versions of surveys fit the picture Quick overview of survey parameters in NA Get a hands-on feel for the process Have the opportunity to discuss/comment on the experience Hopefully to have some fun on your brief journey into the world of NA Also to provide you with a glimpse into the Needs Assessment Kit (Altschuld, Eastmond, King, Kumar, Stevahn, and White, 2010)
3AgendaIntroduction (1 minute) Questions (3-5 minutes) Basic terms and concepts (8-10 minutes) Survey overview ( minutes) Types and where they fit the process Samples of questions (scaled or otherwise) General analysis strategies Wording problems Scenario overview, hands-on work, discussion, etc. (rest of time allotted for the session)
4BeginningsTerms/concepts Need -definition with attention to use of words (need & needs) -not mixing solutions with needs -definitional problem immediately for wording survey items -examples of needs (see partial table from book 1) -types of needs (book 1) -levels of needs -Wow, issues at the outset Needs assessment -three phase model (see schematic) -types of surveys and where they might fit -why you still might have to include solutions in survey despite above admonitionQuestions for you How many have done NAs? Have you used surveys in the process? For what purposes? Briefly describe the survey? How did you analyze the data? Overall success of the effort? Did you use other methods as well – what were they? Any other comments about what you did?
5Definitions & IssuesNeed: the measurable discrepancy between “what is” or the present state of affairs in regard to the group and situation of interest and the “what should be” or desired state of affairs (Witkin & Altschuld, 1995).Issues: measurable discrepancy is the key-needs not solutions (premature closure on solutions)-verb vs. noun concept (misuse of the word)-‘desired’, ‘likely to occur’, ‘ought to occur’, etc.-wish and want lists-many types of needs
6More Terms and Concepts NA is a systematic set of procedures undertaken for the purpose of setting needs-based priorities and making decisions about organizational improvement and allocation of resources (Witkin & Altschuld, 1995).Issues-context for the NA-readiness for an assessment-NA is an organizational activity-political aspects to the activity-systems concept and how to think about itLots of subtle aspects of need and NAExamplesTypesWhy do it and how does it relate to evaluation
12Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010, Thousand Oaks, CA: Sage.
13Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010, Thousand Oaks, CA: Sage.
14Survey considerations Content (continued) Degree of achievement Satisfaction with services Importance/value of services Motivation to resolve problems Frequency of use Observations Feasibility of resolving problems Literature based content Impediments to solutions List goes on and on Formatting Scales (what type) Open-ended Bars Examples of Surveys but firstSampling Level 1, 2, and/or 3 Wording issues Where in the process For NAC? Phase 1, 2, 3 What type of need Short term vs long term Maintenance Collaborative Other Double, triple scales or more Content Behaviors Attitudes
26Example of Follow-Up Survey Lee, Altschuld, & White (2007)
27Now the Hands-on Exercise Scenarios/orientation Pretend that you are the needs assessor Individually and then collectively What can a NAC tell you? Questions Content Discussion focusing Sketch out areas where they could give you guidance/input So you need a survey for other groups Review item writing rules Review examplesConsiderations for NA surveysWhat contentLevels 1, 2, and/or 3Relation to other methodsUse of multiple scalesImpedimentsOpen-ended questions