June 2003George Mason University1 Needs Assessment Farrokh Alemi, Ph.D.

June 2003George Mason University2 Key Reason for Failure of IT Investment  “I can’t get the report I need?”  “What is this for?”  “This is not what I do.” Why are so many information systems unused?

June 2003George Mason University3 Some Recent Student Postings  “... if I could determine WHY the information is needed, I could better assist in determining WHAT information would be needed …”  “ … we could pinpoint what people want, need, and desire …”  “…why do you need this is how are you going to use it comes into play…”  “…you must be familiar with the changing trends of that industry…”

June 2003George Mason University4 Two Common Complaints  Not the right information  Not at the right time Systems that do not meet our needs

June 2003George Mason University5 Why is it so hard to design systems that meet our needs?  A well-defined set of requirements does not exist  The organizational leaders are changing  The analysts are unable to elicit requirements  Future is not now –The business process is changing –The IT industry is changing

June 2003George Mason University6 What you need versus what you want  Systems are designed for organizations not individuals  Systems should help individuals see beyond their own limitations  Systems should anticipate the future

June 2003George Mason University7 How should we assess information needs?

June 2003George Mason University8 Previous Approaches  Analyze organizational tasks and see how information is used in these tasks  Ask the decision-maker about their needs  Derive requirements from the existing systems  Look at strategic goals and concerns  Do input-process-output analysis

June 2003George Mason University9 Problems with Existing Approaches  Focus on past but not future needs  Focus on a single issue, task, or decision  Focus on information not decisions  Focus on the decision-makers’ personal goals, which may be inconsistent with the organizational goals  Focus on users’ behavior without seeking decision maker’s insight

June 2003George Mason University10 Our Proposed Strategy

June 2003George Mason University11 Key Advantages  Focus on future reduces possibility of identifying information no longer needed  “Decision driven" methodology minimizes the potential for collecting data that may not be used  By relying on a group rather than an individual, we minimize the design of the system around idiosyncratic characteristics of a few people  By adding external experts to the membership of the group, the methodology emphasizes what organization ought to need

June 2003George Mason University12 Example: 3 out of 6 decisions identified 1. How to reimburse the growing number of paraprofessionals who treat mental health clients? 2. What educational programs or policies should the state adopt to upgrade knowledge of mental illness among primary care physicians who deliver many services to rural, mentally ill clients? 3. Many chronically mentally ill patients live in nursing homes that cannot provide adequate care. What policies should the state adopt to improve care for these patients?

June 2003George Mason University13 Step two: 2 out of 69 items identified as needed

June 2003George Mason University14 Step 3: Rate the information in context of specific decisions  For the issue "xxx," assign a score between 1 and 3 to each information item reflecting the need for the information. A score of 1 means the information has low priority, a score of 2 means that the information is nice to have but not essential and a score of 3 means that the item is essential for addressing the issue.  Each commissioner rated the relevance of all 69-information items for their decisions

June 2003George Mason University15 Set Data Collection Priorities  Essential items are important to all issues  Rapid collection items are important to most issues; need no action but plans to collect the information if the need arises  Periodic collection items include items important on some issues  Low priority items are ignored

June 2003George Mason University16 Data Collection Plans and Analysis of Ratings

June 2003George Mason University17 Examples of Essential & Rapid Data collection items

June 2003George Mason University18 Examples of periodic and ignored information sets

June 2003George Mason University19 Take Home Lesson  Needs assessment is important  It matters how needs are assessed  It is important to distinguish needs from wish lists –It is important to look to the future –It important to assess in context of specific decisions –It is important to make data collection plans

June 2003George Mason University20 Needs Assessment Exercise 1. List 3 decisions you will make next week 2. List information needed in each decision 3. Make master list 4. Rate the relevance of each item for each decision 5. Analyze ratings across decisions  Range of ratings  Average of ratings

June 2003George Mason University21 Minute Evaluations  Always complete the minute evaluation at end of each lecture