Catherine Griffith, Karen Harrington The Impact of a Construct-Based Approach in Assessing Student Progress at the Elementary Level Katie Gray Unique Potential Consulting Catherine Griffith, Karen Harrington Fredrickson Center for School Counseling Outcome Research and Evaluation

This presentation will outline how and why a research driven, construct based approach to collecting social emotional data via a “Protective Factor Index” [PFI] was developed as part of an ESSC Grant funded project

We will address why a construct based approach to collecting data on non- cognitive factors is important to facilitate meaningful data based decision making for school counselors as well as to enhance the process for other educators in the school.

The goal for such an approach is to allow data teams to determine not just WHAT a student has not learned but WHY he/she is not doing well academically. Once the why is established, a team can expand the menu of interventions for these children to include school counselor led programs and services designed to support the acquisition of social-emotional competencies that research says matter.

The Randolph Project 3 year ESSC Grant awarded in 2012 Grant goals: Implement the ASCA Model and link the work of school counselors to academic achievement Institute a meaningful data-based decision-making model at the district’s 4 elementary schools Create a Protective Factor Index {PFI} reporting system, using student level data to drive interventions and classroom guidance curriculum

The Randolph Team Unique Potential Consulting Fredrickson Center for School Counseling Outcome Research and Evaluation Sebastian Management 4 newly hired school counselors District superintendent and principals

School Counselors and Academic Achievement Ed Trust, The ASCA Model and Accountability School counselors must use data to demonstrate the impact of their work and link it to academic achievement School counselors should be leaders in data based decision making in their schools to support closing achievement gaps and other school improvement efforts

School Counselors and Data Theory Vs Practice Counselors often not adequately trained to use data effectively Most often the data that is available is academic data Counselors operating in a social-emotional “data vacuum” [except for attendance and discipline data] Little information available that reveals information about underlying issues that need to be addressed in order for academic achievement to improve Not clear what socio-emotional data to collect Mechanisms to collect social-emotional data don’t exist

Meeting the Challenges Developing and Piloting A Construct based Approach to Socio-emotional data

What socio-emotional data should be collected? Challenge #1 What socio-emotional data should be collected?

the Research Squier, Nailor and Carey[2014] Comprehensive review of educational and developmental psychology literature to determine what abilities, capabilities and skills are strongly related to academic achievement and later success in lifew Found 4 overarching constructs that explicitly link to student success: Motivation Self- knowledge Self-direction Relationships

The Research [cont.] Bass, Lee and Carey [2015] These 4 constructs associate strongly with students’ academic achievement and well being These 4 constructs are malleable and receptive to intervention These 4 constructs are within the range of expertise of school counselors

Getting Started The Grant Team created a “Protective Factor Index” [PFI] that consists of a 13-item rating scale based on the Construct-Based Approach (CBA) to school counseling (Squire, Nailor & Carey, 2014), as a research-based mechanism for collecting these social emotional data.

The PFI Construct K-5 Standards Engages in class activities   Motivation Engages in class activities Demonstrates an eagerness to learn Demonstrates perseverance in completing tasks Self-knowledge Identifies personal feelings Identifies things he/she is interested in learning Identifies things he/she values Self-direction Demonstrates the ability to self-regulate feelings and actions Demonstrates resilience after setbacks Uses learning strategies effectively Maintains an organized work area; Produces neat and careful work Relationships Works collaboratively in groups of various sizes Respects and accepts authority Forms respectful, equitable relationships with peers

Getting district buy-in Challenge #2 Getting district buy-in

The Grant Team proposal Use the district’s existing Student Information System Include the PFI on all report cards K-5 Create a scoring rubric to accompany the PFI Train classroom teachers to use the rubric to determine whether a student was demonstrating expected age appropriate skills in each domain Have classroom teachers complete the PFI each time they enter grades

Challenge #3 Creating a user-friendly system for collecting the socio-emotional data

A sample report card

The Rubric

Kindergarten and First Grade MOTIVATION   Meets Standard (4) Progressing (3) Emerging (2) Not Meeting Standard (1) Engages in class activities Listens and contributes during class activities and discussions; Follows directions; Takes turns; Joins an activity when asked Consistently, independently is engaged 90-100% of the time With occasional teacher support is engaged 75-89% of the time With frequent teacher support is engaged 60-74% of the time Needs ongoing teacher support and is often disengaged Demonstrates an eagerness to learn Volunteers to take on new challenges (e.g., trying a new activity); Asks questions to further understanding of a topic; Maintain a positive attitude when faced with challenges; Expresses enthusiasm when learning something new Consistently, independently is eager to learn is eager to learn 75-89% of the time is eager to learn 60-74% of the time Needs ongoing teacher support and is often not eager to learn Demonstrates perseverance in completing tasks Uses strategies to work through challenges and complete work; Maintains a positive attitude when faced with a difficult or unfamiliar task; Recovers from setbacks Consistently, independently demonstrates perseverance demonstrates perseverance 75-89% of the time demonstrates perseverance 60-74% of the time Needs ongoing teacher support and often does not demonstrates perseverance Exhibits positive and optimistic behavior Maintains a positive attitude when engaged in class activities; Expresses enthusiasm; Believes in own capability as a learner; Shares excitement about learning with peers; Identifies aspects of school that are interesting and exciting Consistently, independently is positive and optimistic is positive and optimistic 75-89% of the time is positive and optimistic 60-74% of the time Needs ongoing teacher support and is often is not negative and pessimistic

2nd and 3rd Grade SELF-KNOWLEDGE Meets Standard (4) Progressing (3)   Meets Standard (4) Progressing (3) Emerging (2) Not Meeting Standard (1) Identifies academic strengths and abilities When asked, can name multiple strengths and abilities (e.g., “I am a good writer,” or “I have neat handwriting” Consistently, independently names academic strengths and abilities With occasional teacher support names academic strengths and abilities With frequent teacher support Needs ongoing teacher support to name academic strengths and abilities Identifies things he/she is interested in learning When asked, can name multiple learning interests (e.g., “I like learning to read,” or “I want to learn more about butterflies”) Consistently, independently identifies learning interests identifies learning interests Needs ongoing teacher support to identify learning interests

2nd and 3rd Grade SELF-DIRECTION   Meets Standard (4) Progressing (3) Emerging (2) Not Meeting Standard (1) Demonstrates the ability to self-regulate actions and emotions Sustains attention during activities (such as centers) and routines (such as circle time); Resolves or avoids conflicts with others; Exhibits safe behavior across school settings, throughout the day; Refrains from emotional outbursts (e.g., temper tantrums or major upsets) Consistently, independently self-regulates 90-100% of the time With occasional teacher support self-regulates 75-89% of the time With frequent teacher support self-regulates 60-74% of the time Needs ongoing teacher support and is often unable to self-regulate Demonstrates resilience after setbacks Recovers from (academic and social) disappointments and challenges without a lengthy disruption in engagement (e.g., being told that s/he completed an assignment incorrectly) Consistently, independently demonstrates resilience demonstrates resilience 75-89% of the time demonstrates resilience 60-74% of the time Needs ongoing teacher support and is often does not demonstrate resilience Makes productive use of classroom time Participates in activities as assigned; Stays focused on the task at hand; Completes assignments and activities on time Consistently, independently is productive is productive 75-89% of the time is productive 60-74% of the time Needs ongoing teacher support and often not productive

4th and 5th grade RELATIONSHIPS Respects and accepts authority   Meets Standard (4) Progressing (3) Emerging (2) Not Meeting Standard (1) Works collaboratively in groups of various sizes Listens to peers; Participates actively (e.g., sharing verbally and/or listening); Works with classmates to complete a task in the time given; Demonstrates flexibility taking on different roles during group work (e.g., can follow and lead) Consistently, independently works collaboratively in groups With occasional teacher support works collaboratively in groups With frequent teacher support Needs ongoing teacher support and often does not work collaboratively in groups Seeks assistance when necessary Uses classroom resources and protocols (e.g., asking a peer, using a reference sheet) before asking an adult for help; Actively seeks the appropriate adult/s who can provide the needed support Consistently, independently seeks assistance when needed 90-100% of the time seeks assistance when needed 75-89% of the time seeks assistance when needed 60-74% of the time Needs ongoing teacher support and is often unable to seek needed assistance Respects and accepts authority Follows directions provided by all adults at school; Communicates politely with adults; Responds to limit-setting without disruptive outbursts; Knows and follows school rules Consistently, independently respects authority 90-100% of the time respects authority perseverance 75-89% of the time respects authority perseverance 60-74% of the time Needs ongoing teacher support and often disrespects authority figures Forms respectful, equitable relationships with peers Demonstrates care for classmates and has formed friendships with peers; Identifies peers by name; Cooperates with peers and can work with others towards a common goal (e.g., completing a group science project or playing in a soccer game at recess); Avoids or resolves interpersonal conflicts (during instructional and non-instruction times) Consistently, independently forms respectful relationships with peers forms respectful relationships with peers Needs ongoing teacher support and often behaves disrespectfully towards peers

Counselors leading with data vs. collecting data The beauty of this system for data collection is that students are rated by the teachers who are with them all day and have a picture of whether the student understands a construct AND whether he/she puts it into practice. The counselor is the analyzer/interpreter of data trends, rather than the actual data collector. The teacher is the beneficiary of additional data, interpreted by the school counselor, to help understand why a child may be struggling. Together, teachers and counselors can have data based discussions about how to intervene.

What professional Development is necessary to support this approach? Challenge #4 What professional Development is necessary to support this approach?

Teacher PD Teachers needed to know WHY they were being asked to complete the PFI They also needed to know HOW to complete it accurately

Counselor PD Counselors needed to learn HOW TO EXTRACT the PFI data from the SIS Counselors also needed to learn how to understand/analyze and use the data they were seeing Counselors needed training in EXCEL Counselors needed training in leading data based conversations with teachers and families Counselors needed help identifying evidence based interventions that are available to support needs identified by the data at the universal [tier1] as well as Tier 2 level

PD formats Large group Embedded How/Why for teachers, families Counselors have been charged with delivering on- going embedded professional development by presenting PFI data after each marking period and leading conversations about school wide trends at faculty meetings, grade level trends at Common Planning Time meetings and classroom and individual trends in consultation meetings with teachers.

Promising practices

#1: It is possible to collect socio-emotional Data via existing SIS systems Teachers reported they do not feel overburdened by adding PFI information when they complete report cards With training in exporting to EXCEL, as well as the filter, sort and chart making functions of EXCEL, counselors can use PFI data to examine district, school, grade level, classroom and individual trends relative to each of the factors on the PFI. Counselors report it takes them 2-3 hours after each marking period to manipulate the data, analyze it and prepare charts

#2: The Rubric is a valuable tool Each factor is described with concrete, observable behaviors and ratings are quantified Analysis of the data …..Karen add here about reliability Important to share with parents Parental feedback has been very positive; parents agree socio-emotional factors are important and they value getting this information on a regular basis

#3 PFI data used to drive tiered interventions Analysis of baseline data indicated that self- regulation was problematic at every grade at every school

Self regulation DATA from 4 elementary schools

Data driven interventions Counselors revised their Tier 1 services to include targeted instruction on self- regulation to all classes in every grade via the developmental guidance curriculum Teachers’ summer reading was “The Zones of Regulation” Counselors led a faculty in-service on “The Zones” Additionally, data is used to identify students needing additional support [Tier 2] so they can be placed in small groups focusing specifically on self-regulation

#4 Counselors are seen as leaders and data experts Counselors are taking on a leadership role in helping their colleagues understand the connections between non-cognitive factors and academic achievement Teachers and Administrators are having “ah- ha” moments where they see these connections and then are looking for strategies to help promote the acquisition of these factors Counselors are collecting teacher led strategies to promote self-regulation and are publishing best practices for all.

A counselor developed PPT for 4th grade teachers’ common planning time

Protective Factor Index 2013-2014 Term 3 Data

What are Protective Factors? The protective factor index highlights a different set of skills than those traditionally focused on on a report card. These 13 skills have been statistically linked to success in children. Therefore, this data is very important in understanding areas where our students are thriving and areas they need to improve in. By looking at this data, it can help us to be more focused on the type of interventions we put into place.

What are we using this data for? Tier 1: Classroom lessons that target areas that a high number of students are struggling in Tier 2: Social/Emotional groups are formed based on this data Tier 3: Individual’s needs can be determined using this data and can lead appropriate interventions

Who produces this data? YOU! This data comes from the 2013-2014 report cards (specifically focusing on Term 3 for this power point). The 2014-2015 report cards will have a 4 point scale, instead of just “struggling” or “on target” to more specifically indicate student growth.

#1) Demonstrates an eagerness to learn

#2) Demonstrates perseverance in completing tasks

#3) Demonstrates resilience after setbacks

#4) Demonstrates the ability to self-regulate actions and emotions

#5) Engages in class activities

#6) Exhibits positive and optimistic behavior

#7) Forms respectful, equitable relationships with peers

#8) Identifies academic strengths and abilities

#9) Identifies things he/she is interested in learning

#10) Makes productive use of classroom time

#11) Respects and accepts authority

#12) Seeks assistance when necessary

#13) Works collaboratively in groups of various sizes

Questions/Comments What did you notice? Where are we struggling? Where are we doing well? What interventions do we already have in place? What interventions might be helpful for us to put in place to increase our number of on target students?

#5: District is exploring incorporating PFI at MS and HS Superintendent is prioritizing budget funds to support building more data collection infrastructure Training MS and HS counselors to be leaders in data-driven conversations about the connections between non-cognitive factors and academic achievement

The crucial next steps Over time it will become clear whether intervening to support the development of non-cognitive factors results in improved academic achievement.