Presentation on theme: "SHIFTS IN BELIEFS USING DATA COMPARISON FINDINGS OF SCHOOL COUNSELOR BELIEFS ABOUT ASCA NATIONAL MODEL SCHOOL COUNSELING PROGRAM COMPONENTS USING THE SCPCS."— Presentation transcript:
SHIFTS IN BELIEFS USING DATA COMPARISON FINDINGS OF SCHOOL COUNSELOR BELIEFS ABOUT ASCA NATIONAL MODEL SCHOOL COUNSELING PROGRAM COMPONENTS USING THE SCPCS Trish Hatch, Ph.D. San Diego State University Timothy Poynton, Ed.D. Suffolk University
OVERVIEW OF SESSION Activity: What do you think? Have school counselors’ beliefs shifted since the introduction of the ASCA Model? How have beliefs shifted for use of data, accountability and reporting results? Which areas are positive? Has this affected school counselor behavior? What work is still needed? Sharing ideas for supporting school counselors who may not yet believe in the value of data and accountability
SCPSC Hatch and Chen-Hayes (2008) published article about school counselors’ beliefs regarding the ASCA National Model School Counseling Program Components using the School Counselor Program Component Scale (SCPCS). 2002 Survey administered to over 1,200 ASCA members Established psychometric properties of the SCPCS Collected national baseline data on school counselor beliefs about certain program components in the ASCA National Model prior to its release in 2003.
ACTIVITY #1 WHAT DO YOU PREDICT? To what extent do you think school counselor beliefs shifted on the SCPCS items since the introduction of the ASCA National Model? Note rankings from the 2002 survey are activities that align with the ASCA National Model. They are listed from most important to least important What shifts do you predict are significant?
2009 SCPSC INSTRUMENT The SCPCS was modified slightly for use in the 2009 study, based on feedback provided by Professional School Counseling reviewers The original 5 point scale, which ranged from 1 to 5 with named anchors at 1 (very important), 3 (moderately important) and 5 (not important) was modified to a 3 point scale using only the named anchors of the original scale.
USEABLE SURVEYS The initial email invitation yielded 285 usable surveys (46%) Remaining 332 participants completed the survey after the reminder email was sent. The online survey was a multi-page survey which contained a page with informed consent
SURVEY PARTICIPANTS 617 participants 27% (n = 165) elementary school counselors 21% (n = 130) middle/junior high school counselors 32% (n = 196) were high school counselors, 12% (n = 72) worked at multiple levels 4% (n = 27) school counseling program supervisors Twenty-seven respondents (4%) did not answer this question.
DEMOGRAPHICS 83% Female (n = 512) 15% Male (n = 95) 2% (n = 10) did not answer 86% percent of the respondents White (n = 512) 4% identified as Black (n = 23) 4% identified as Hispanic (n = 24) 2% identified as Asian or Pacific Islander (n = 12) 2% identified as multiracial/multiethnic (n = 13) 1% identified as some other race/ethnicity (n = 6) 10 respondents (2%) did not answer this question
MORE DEMOGRAPHICS Forty-six percent of the respondents worked in suburban settings (n = 286), 28% in rural settings (n = 171) 24% in urban settings (n = 147) 13 respondents (2%) did not answer this question Average amount of experience working as school counselor reported by respondents was 8.1 years (SD = 7.1).
Q #1 FACTOR ANALYSIS To assess the possible impact of changing the scale on the factor structure, reliability, and validity of the SCPCS, several analyses were performed in a manner identical to those reported by Hatch and Chen-Hayes (2008). Specifically, internal consistency estimates and a principal components analysis (PCA) were calculated in a manner identical to the initial SCPCS study.
FACTOR ANALYSIS Compared Factor Structures 2002-2009 18 of 19 SCPSC items loaded on same factors Previous item not retained in FA in 2002 (#19) loaded on administrator support. Eigenvalues of three of the four factors in the 2009 data were greater than 1.00 The amount of variance each factor explained differed from the 2002 data
SUMMARY RESULTS Despite the differences between the two studies (scale change and participation due to on-line administration), analyses revealed four distinct and internally consistent factors evidencing construct validity. The factor structure of the SCPCS remained intact in the second study. Internal consistency estimates exceeded acceptable requirements for the scale and each of the four subscales. Thus, the four subscales were found to be consistent and reliable.
EIGENVALUES 9.94 (Use of Data for Accountability, 52.60%) 1.35 (Use of Data for Program Planning, 7.10%) 1.14 (Administrator Support, 5.99%).89 (Mission, Goals, and Competencies, 4.71%) Majority of the variance in the 2002 data was explained by the Use of Data for Program Planning factor (43.50%) Majority of the variance in the 2009 data was explained by the Use of Data for Accountability factor (52.60%).
INTERNAL CONSISTENCY (CRONBACH’S ALPHA) Use of Data for Program Planning =.87 (.82); Use of Data for Accountability =.91 (.80); Administrator Support =.85 (.78); Mission, Goals, and Competencies =.84 (.86) All subscales evidenced acceptable reliability characteristics using the commonly accepted criterion provided by Nunnally (1978).
ANOVA (SUBSCALE MEANS) Statistically significant, F (3, 606) = 27.78, p <.001, η 2 =.12, a small effect size. Post-hoc tests with a Bonferroni correction revealed differences among all possible pairwise comparisons: Mission, Goals, and Competencies (M = 1.32, SD =.41) Use of Data for Program Planning (M =1.45, SD =.46 ) Use of Data for Accountability (M = 1.37, SD =.46) Subscales all significant at the p <.001 level. Observed differences between the Administrator Support (M = 1.41, SD =.44) and Use of Data for Program Planning subscales was significant (p =.033) Difference between the Administrator Support and Use of Data for Accountability did not reach statistical significance (p =.074).
Q #2 RANK ORDER To facilitate analysis of changes in perceived importance of the items over time, individual items were rank-ordered based on observed mean scores. The use of rank-ordered lists of the 2002 and 2009 data facilitates comparison between the two administrations in spite of the change in the scale Allows conclusions to be drawn about the perceived importance of each item relative to all other SCPCS items.
RANK ORDER (PERCEIVED IMPORTANCE) To assess the perceived importance of each individual item in a comprehensive school counseling program, descriptive statistics (mean and standard deviation) were calculated. The SCPCS item means and standard deviations are presented in rank order in Table 2, along with the rank of the item based on the 2002 data.
GREATEST RANK ORDER SHIFTS (TOWARD VERY IMPORTANT) 13 th 6 th use of data to measure the outcome results of the school counseling program 17 th 10 th using various data student data to identify gaps 10 th 5 th use of data to demonstrate the impact of the school counseling program on student success in school
DISCUSSION POSSIBLE EXPLANATIONS Are you SURPRISED by the findings? What do YOU think contributed to these findings? If not what expected, what factors contributed to your differing perspective?
DISCUSSION Notable shift in Use of Data for Program Planning factor accounting for the majority of the variance in 2002 to the Use of Data for Accountability factor accounting for the majority in 2009. Consistent with the 2002 findings, the Use of Data for Program Planning item “included five items related to using data to target interventions and identify program foci” (Hatch & Chen-Hayes, 2008, p. 38).
TOP RANKED: DEVELOPING GOALS FOR PROGRAM Setting goals within the school counseling program is a required component of the Recognized ASCA Model Program (RAMP) (ASCA, 2012a) SMART goals are included in ASCA’s Making Data Work (Young & Kaffenberger, 2009). Setting program goals has now been added as a new topic in the foundation section of the ASCA National Model (2012b).
SIX TOP RANKED ITEMS (2009) CONSISTENT WITH ASCA MODEL 1.Develop goals for the counseling program 2.Write a mission statement or philosophy 3.Utilize school-wide and student data to design new counseling activities 4.Identify specific student competencies to which the school counseling program, curriculum or activities contribute or align 5.Use data to demonstrate the impact of the school counseling program on student success in school 6.Use data to measure the outcome results of the school counseling program
GREATEST POSITIVE RANK ORDER SHIFTS WHAT DOES IT MEAN? Have school counselors begun to prioritize the value, importance, and necessity of using data in their school counseling programs? Provides positive feedback for ASCA and national leaders in school counseling who have lamented the paucity of outcome research and the lack of school counselor engagement in data driven activities and accountability (it’s a start)
GREATEST POSITIVE RANK ORDER SHIFTS WHAT DOES IT MEAN? Two items moved up 7 RANKINGS! o Data to identify gaps o Data to measure impact Shift aligns with consistent requests for school counselors to do more than use data to design and prepare program activities. School counselors have been encouraged to improve efficiency and effectiveness and to promote the schools counselors’ value as worth its cost or resource
RANK ORDER FINDINGS ALIGN WITH NATIONAL REFORM EFFORTS Aligns with work of Education Trust Transforming School Counseling Initiative (TSCI) Central tenets of ASCA National Model (Closing the Gap Action Plans) The College Board’s National Office for School Counselor Advocacy (NOSCA), TSCI National Association of College Admissions Counselors (NACAC)
ASCA 3 RD EDITION (161 PAGES) Data mentioned 124 times Gaps is mentioned 60 times. Results (124 mentions) Accountability (48 mentions) ASCA Model incorporates language from ethical guidelines (ASCA, 2010) stressing the responsibility of school counselors to use equity-based data to identify, address, and resolve attainment, achievement, and opportunity gaps. pretty irrefutable
FUTURE RESEARCH School counselors who responded ASCA members who have received firm directive from the professional association to answer the question: “How are students are different as a result of what school counselors do?” Future research with a more diverse sample is needed to understand more about the beliefs of non-ASCA member school counselors regarding data for planning and accountability.
GREATEST NEGATIVE RANK ORDER SHIFTS 8 th 14 th Delineating non-school counselors related activities 12 th 18 th Adhering to non-school counseling activities Is there a relationship between school counselor beliefs and behaviors regarding accountability and non-school counselor activities Are school counselors who produce student and program results less likely to be asked to perform non-school counselor activities?
CONSULTING “MORE” WITH ADMIN. SHIFTED -4 FROM 3 RD TO 7 TH Are SC’s more competent and confident with regard to improving their school counseling programs and thus, believe there is less need to consult with administrators? Have they already begun to consult “more” regularly with administrators and thus, do not necessarily need to consult even “more”?
PRESENTING RESULTS STILL LOW Consistently low ranking on both surveys in relationship to other items (16 th & 15 th ) Despite strong and consistent messages have been sent for school counselors to share their results with stakeholders (Hatch, 2008; Dimmitt, et al., 2007). Decisions on the employment of counselors can be based on the results they produce (Duarte & Hatch, in review; Fladager, 2012). Further research is needed to understand factors contributing to lack of perceived importance in relationship to other items.
LIMITATIONS Not longitudinal Email to on-line not paper pencil Problems with email: o Did they ever get it? Sent to Junk? o Or not respond?
RESEARCH NEEDED ON… 83% of the respondents in both surveys identified as female. 2009 slightly more diverse (86% white compared to 92% in 2002). Largest increase Hispanic/Latino: doubled 2% in 2002 to 4% in 2009. African American respondents increased minimally from 3% to 4%. Compared to College Board Study: 78% female, 10% African American, and 15% percent Hispanic or Latino (Bruce & Bridgeland, 2012). Future research is encouraged to compare national school counselors’ demographics to membership in professional associations.
AND FINALLY… 3 rd edition ASCA Model reflects current practice School counselors and Common Core Future research needed to access shifts 2002 2009 2016 What do YOU think the NEXT a shifts will be?
SHARING IDEAS Share ideas for supporting school counselors who may not yet believe in the value of data and accountability Ideas to support their future growth in this area…
PRIMARY REFERENCE (EXTENSIVE LIST AVAILABLE UPON REQUEST) Hatch, T., & Chen-Hayes, S. F. (2008). School counselor beliefs about ASCA National Model school counseling program components using the SCPCS. Professional School Counseling, 12(1), 34-42. Hatch, T. & Poynton, T. & Perusse, R. (in review). Shifts in beliefs using data Comparison: Findings of School Counselor Beliefs about ASCA National Model School Counseling Program Components Using the SCPCS
THANK YOU Tim Poynton email@example.com@suffolk.edu Trish Hatch firstname.lastname@example.org (until July 1, 2013)email@example.com Contact Tim if you have questions (Thanks Tim!)