Applied Research Consultants: Applied Research Consultants: Assessment Services and External Evaluation

Who we are:

Applied Research Consultants (ARC) is: –an organization composed of graduate students –housed within the Psychology Department at Southern Illinois University Carbondale –currently directed by Rhonda Kowalchuk, Ph.D. ▪Founded in 1982, ARC has been using quantitative and qualitative methods to provide research and consulting services for a variety of local, regional, and national organizations.

ARC services ▪Applied Research Consultants uses the latest technology and research methods to provide a wide range of reasonably priced consulting services including: Research Design Survey Development Web-based Statistical Consulting Program Evaluation Focus Groups In-Depth Interviews Needs Assessments Job Analyses Performance Appraisals Personnel Training

University Impact ▪Agricultural Economics Department ▪Alumni Association ▪Athletics Department ▪SIU Board of Trustees ▪College of Liberal Arts ▪McNair Scholars Program ▪Office of the Chancellor ▪School of Law ▪IT department ▪Student Legal Services ▪Political Science Department ▪Center for Rural Health and Social Service Development ▪Core Institute ▪University Communications ▪Center for English as a Second Language ▪Women’s Center ▪Marketing Department

Community Impact ▪Illinois Board of Higher Education ▪Illinois Education Research Council ▪Carbondale Elementary School District 95 ▪Jackson County Health Department ▪Southern Illinois Healthcare ▪The City of Carbondale ▪The Southern Illinoisan ▪Southern 7 Health Department ▪Neighborhood Co-op ▪Ameren ▪University of Illinois Chicago ▪Indiana University-Purdue University Indianapolis (IUPUI)

Current Assessment Example ▪McNair Scholars Program Evaluation –Evaluating the effectiveness of the McNair Scholars Program at SIU in preparing their students for graduate school and entrance to the workforce in comparison to other McNair-qualified students who did not participate in the program. ▪Assessment survey development ▪Sampling procedures (i.e., recruitment of participants) ▪Online administration of the survey instrument ▪Data management and analysis ▪Data interpretation and technical report write-up

Best Practices for Project Evaluation Make both formative and summative evaluation an integral part of project planning Use established, attainable evaluation methods Plan multiple types of data and anticipate threats to validity Perform evaluation throughout the life of the project Ask different questions at different points in time Identify recommendations for project improvement Consult with or retain external evaluators THE 2002 USER-FRIENDLY HANDBOOK FOR PROJECT EVALUATION. NSF, 2002.

Importance of an External Evaluator –Objectivity: No inherent biases towards assessment targets –Extensive background in methodology and statistical analysis ▪Develop a project model with expected outcomes and key evaluation points ▪Develop evaluation questions and define measurable objectives ▪Develop the evaluation design –Helps to meet grant funding requirements ▪NIH, NSF, Dept. of Education –Timeline management ▪Consistent contact ensures assessment stays on track

ARC as an External Evaluator ▪Budget Friendly! –On average one-third the cost of external consulting firms due to university affiliation –Initial consultation meeting at no-cost to client ▪Secure third-party data server –No outside access to collected data without authorized permission. ▪Familiarity with SIUC policies and procedures –Human Subjects Committee ▪Accessibility –Convenient on-campus location –Timely and convenient meeting availability

Thank you for your time! Are there any questions?