An Evidence-Based Practice Approach to School Counselor Education John Carey, Carey Dimmitt, Natalie Kosine National Center for School Counseling Outcome Research University of Massachusetts, Amherst Tim Poynton Suffolk University

Evaluation and Research Competencies for School Counselors Knowledge of Evaluation and Research Methods are critical knowledge for school counselors to have in order to participate in school reform efforts, demonstrate accountability, and effectively define evidence-based practices for local use Data Conceptual and Practical methods for using data School Counselors

Prior Research – School Counselors and Accountability Early research from the 1970’s focused on “time analysis” procedures Answers the question How does a counselor spend his or her time? Still important today, but not in isolation

Prior Research – School Counselors and Accountability Literature from the 1970’s describes additional data sources for accountability, but relatively few school counselors actually engaged in “formal evaluation activities” that used quantitative data with relatively large numbers of students (Vacc, 1981). WHY? Relevance

Accountability Today The ASCA National Model is the “gold standard” for school counseling programs, and is driven by the NCLB act Asserts that counselors must demonstrate “how students are different as a result of the school counseling program in measurable terms” (ASCA, 2003)

School Counselor Competence in Research and Evaluation Early research arguing that school counselors “tend to confuse accountability with research” (Atkinson, Furlong, and Janoff, 1979) is no longer true Several authors assert that lack of training in graduate school (Wilson, 1985; Fairchild, 1993; Vacc, 1981; Myrick, 1984, 1997) and little knowledge of assessment and evaluation procedures (Myrick, 1984, 2003) significantly hinder counselors’ accountability efforts

School Counselor Competence in Research and Evaluation Articles are appearing that describe various research methods to engage in accountability efforts, such as single- subject designs (Foster et al, 2002) and using school–wide data (Dimmitt, 2003; Hayes et al, 2002)

Why School Counselors Underutilize the Research Literature School Counselor Education Programs School Counselor Education Programs –School Counselor Education Programs neglect training in research design, statistics and measurement –School counselors have few opportunities to engage in research during graduate training –School Counselor Education programs underemphasize research approaches that are the most appropriate for school counselors

Why School Counselors Underutilize the Research Literature School Counselor Education Programs School Counselor Education Programs –School Counselor Education program do not infuse research throughout the curriculum –School Counselor Education programs fail to teach that research is an indispensable foundation for practice

Why School Counselors Underutilize the Research Literature Public Schools Present barriers to Research Public Schools Present barriers to Research –Parents are reluctant to allow student to participate in research –Classroom organization precludes strong research designs –School Counselor Role does not include research –Reward structures do not encourage research

Why School Counselors Underutilize the Research Literature School Counselor Attitudes towards Research School Counselor Attitudes towards Research –School Counselors do not perceive research as valuable and relevant to their work –School Counselors commonly fear statistics –Gender stereotypes may inhibit research interest in female school counselors

Why School Counselors Underemphasize Evaluation School Counselor Education Programs School Counselor Education Programs –Neglect evaluation training in favor of research training State Departments of Education State Departments of Education –Do not require evaluation competence for certification School Counselors School Counselors –Fear the results of evaluation –Lack evaluation skills Public Schools Public Schools –Workload precludes evaluation

The Uses of Data Accountability Using Data Decision Making

Evidence-Based Practice All of the prior conceptual work on school counselor accountability, evaluation and and research has missed the mark All of the prior conceptual work on school counselor accountability, evaluation and and research has missed the mark Accountability has historically been viewed as the reason school counselors should engage in data collection activities Accountability has historically been viewed as the reason school counselors should engage in data collection activities In EBP, accountability is the end product of a process counselors can engage to identify areas in need of services, select appropriate interventions to address those needs, and evaluate the program’s effectiveness at meeting those needs In EBP, accountability is the end product of a process counselors can engage to identify areas in need of services, select appropriate interventions to address those needs, and evaluate the program’s effectiveness at meeting those needs

A Model of Evidence-Based Practice Client Values and Expectations Practitioner’s Individual Expertise Best Evidence EBP Adapted from: Shlonsky and Gibbs (2004), “Will the Real Evidence-Based Practice Please Stand Up? Teaching the Process of Evidence-Based Practice to the Helping Professions. In Brief Treatment and Crisis Intervention, 4(2),

A Model of Evidence-Based Practice in School Counselor Education Intervention and Program Evaluation Competence Outcome Research Competence Assessment of Intervention Targets and Goals EBP

A Model of Evidence-Based Practice in School Counselor Education Assessment of Intervention Targets and Goals Outcome Research Competence EBP Intervention and Program Evaluation Competence “Knowing how students changed”

A Model of Evidence-Based Practice in School Counselor Education Assessment of Intervention Targets and Goals EBP Intervention and Program Evaluation Competence Outcome Research Competence “Knowing what generally works”

A Model of Evidence-Based Practice in School Counselor Education EBP Intervention and Program Evaluation Competence Outcome Research Competence Assessment of Intervention Targets and Goals “Knowing what is needed”

A Model of Evidence-Based Practice in School Counselor Education Intervention and Program Evaluation Competence Outcome Research Competence Assessment of Intervention Targets and Goals EBP

An Evidence-Based Practice Competencies Our recommendations will be framed across topical areas which are domains of knowledge school counselors need to be effective practitioners in the 21 st century Our recommendations will be framed across topical areas which are domains of knowledge school counselors need to be effective practitioners in the 21 st century Topical Areas o Assessment of Intervention Target and Goals Competence o Outcome Research Competence o Intervention and Program Evaluation Competence

An Evidence-Based Practice Competencies Developing the knowledge and skills to identify areas for program improvement Requires developing competencies in: 1. Analysis of Institutional data 2. Processes for obtaining planning information from stakeholders 3. Quantitative needs assessment techniques Topical Areas o Assessment of Intervention Target and Goals Competence o Outcome Research Competence o Intervention and Program Evaluation Competence

An Evidence-Based Practice Competencies Developing the knowledge and skills to identify and evaluate published research literature Requires developing competencies in: 1. Documentation and replicability of interventions 2. Strength of the research design 3. Psychometric properties of instruments and measures Topical Areas o Assessment of Intervention Target and Goals o Outcome Research Competence o Intervention and Program Evaluation Competence

An Evidence-Based Practice Competencies Competencies (continued) 4. Evaluate analytic and meta-analytic reviews of outcome studies 5. Summarize the results of major school-counseling related outcome studies 6. Conceptual knowledge of statistical significance and effect size 7. Conduct literature searches and consult panels and other information sources for evidence- based practices Topical Areas o Assessment of Intervention Target and Goals Competence o Outcome Research Competence o Intervention and Program Evaluation Competence

An Evidence-Based Practice Competencies Requires developing the knowledge and skills to generate local knowledge of school counseling intervention and program effectiveness Requires developing competencies in: 1. Strong evaluation designs appropriate for educational settings 2. Selection and use of sound instruments and measures Topical Areas o Assessment of Intervention Target and Goals Competence o Outcome Research Competence o Intervention and Program Evaluation Competence

An Evidence-Based Practice Competencies Competencies (continued) 3. Data analysis techniques, including the use of technology tools to assist in data analysis 4. Interpretation of the results of data analysis 5. Accurate dissemination and reporting of results Topical Areas o Assessment of Intervention Target and Goals Competence o Outcome Research Competence o Intervention and Program Evaluation Competence

A Model of Evidence-Based Practice in School Counselor Education Intervention and Program Evaluation Competence Outcome Research Competence Assessment of Intervention Targets and Goals EBP

Curriculum Implications A course that helps counselors develop skills in all three EPB domains is very helpful A course that helps counselors develop skills in all three EPB domains is very helpful Its important at counselors are also taught how to enact an integrated EBP approach that can be implemented in a school (Data Based Decision- Making, Action Research, Logic Modeling) Its important at counselors are also taught how to enact an integrated EBP approach that can be implemented in a school (Data Based Decision- Making, Action Research, Logic Modeling) EBP skills should be integrated into all professional courses and practica (reading meta- analytic reviews and original research, evaluating research, writing briefs, designing evaluation approaches) EBP skills should be integrated into all professional courses and practica (reading meta- analytic reviews and original research, evaluating research, writing briefs, designing evaluation approaches)

Questions and Discussion

Thank You! For additional information, please visit the Center for School Counseling Outcome Research website: