1. Catherine Griffith, Karen Harrington
The Impact of a Construct-Based Approach in Assessing Student Progress at the Elementary Level
Katie Gray
Unique Potential Consulting
Catherine Griffith, Karen Harrington
Fredrickson Center for School Counseling Outcome Research and Evaluation

3. This presentation will outline how and why a research driven, construct based approach to collecting social emotional data via a “Protective Factor Index” [PFI] was developed as part of an ESSC Grant funded project

4. We will address why a construct based approach to collecting data on non- cognitive factors is important to facilitate meaningful data based decision making for school counselors as well as to enhance the process for other educators in the school.

5. The goal for such an approach is to allow data teams to determine not just WHAT a student has not learned but WHY he/she is not doing well academically. Once the why is established, a team can expand the menu of interventions for these children to include school counselor led programs and services designed to support the acquisition of social-emotional competencies that research says matter.

6. The Randolph Project 3 year ESSC Grant awarded in 2012 Grant goals:
Implement the ASCA Model and link the work of school counselors to academic achievement
Institute a meaningful data-based decision-making model at the district’s 4 elementary schools
Create a Protective Factor Index {PFI} reporting system, using student level data to drive interventions and classroom guidance curriculum

7. The Randolph Team Unique Potential Consulting
Fredrickson Center for School Counseling Outcome Research and Evaluation
Sebastian Management
4 newly hired school counselors
District superintendent and principals

8. School Counselors and Academic Achievement
Ed Trust, The ASCA Model and Accountability
School counselors must use data to demonstrate the impact of their work and link it to academic achievement
School counselors should be leaders in data based decision making in their schools to support closing achievement gaps and other school improvement efforts

9. School Counselors and Data
Theory Vs Practice
Counselors often not adequately trained to use data effectively
Most often the data that is available is academic data
Counselors operating in a social-emotional “data vacuum” [except for attendance and discipline data]
Little information available that reveals information about underlying issues that need to be addressed in order for academic achievement to improve
Not clear what socio-emotional data to collect
Mechanisms to collect social-emotional data don’t exist

10. Meeting the Challenges
Developing and Piloting A Construct based Approach to Socio-emotional data

11. What socio-emotional data should be collected?
Challenge #1
What socio-emotional data should be collected?

12. the Research Squier, Nailor and Carey[2014]
Comprehensive review of educational and developmental psychology literature to determine what abilities, capabilities and skills are strongly related to academic achievement and later success in lifew
Found 4 overarching constructs that explicitly link to student success:
Motivation
Self- knowledge
Self-direction
Relationships

13. The Research [cont.] Bass, Lee and Carey [2015]
These 4 constructs associate strongly with students’ academic achievement and well being
These 4 constructs are malleable and receptive to intervention
These 4 constructs are within the range of expertise of school counselors

14. Getting Started
The Grant Team created a “Protective Factor Index” [PFI] that consists of a 13-item rating scale based on the Construct-Based Approach (CBA) to school counseling (Squire, Nailor & Carey, 2014), as a research-based mechanism for collecting these social emotional data.

15. The PFI Construct K-5 Standards Engages in class activities
Motivation
Engages in class activities
Demonstrates an eagerness to learn
Demonstrates perseverance in completing tasks
Self-knowledge
Identifies personal feelings
Identifies things he/she is interested in learning
Identifies things he/she values
Self-direction
Demonstrates the ability to self-regulate feelings and actions
Demonstrates resilience after setbacks
Uses learning strategies effectively
Maintains an organized work area; Produces neat and careful work
Relationships
Works collaboratively in groups of various sizes
Respects and accepts authority
Forms respectful, equitable relationships with peers

16. Getting district buy-in
Challenge #2
Getting district buy-in

17. The Grant Team proposal
Use the district’s existing Student Information System
Include the PFI on all report cards K-5
Create a scoring rubric to accompany the PFI
Train classroom teachers to use the rubric to determine whether a student was demonstrating expected age appropriate skills in each domain
Have classroom teachers complete the PFI each time they enter grades

18. Challenge #3
Creating a user-friendly system for collecting the socio-emotional data

19. A sample report card

20. The Rubric

21. Kindergarten and First Grade
MOTIVATION
Meets Standard (4)
Progressing
(3)
Emerging
(2)
Not Meeting Standard (1)
Engages in class activities
Listens and contributes during class activities and discussions; Follows directions; Takes turns; Joins an activity when asked
Consistently, independently is engaged
90-100% of the time
With occasional teacher support
is engaged 75-89% of the time
With frequent teacher support
is engaged 60-74% of the time
Needs ongoing teacher support and is often disengaged
Demonstrates an eagerness to learn
Volunteers to take on new challenges (e.g., trying a new activity); Asks questions to further understanding of a topic; Maintain a positive attitude when faced with challenges; Expresses enthusiasm when learning something new
Consistently, independently is eager to learn
is eager to learn 75-89% of the time
is eager to learn 60-74% of the time
Needs ongoing teacher support and is often not eager to learn
Demonstrates perseverance in completing tasks
Uses strategies to work through challenges and complete work; Maintains a positive attitude when faced with a difficult or unfamiliar task; Recovers from setbacks
Consistently, independently demonstrates perseverance
demonstrates perseverance 75-89% of the time
demonstrates perseverance 60-74% of the time
Needs ongoing teacher support and often does not demonstrates perseverance
Exhibits positive and optimistic behavior
Maintains a positive attitude when engaged in class activities; Expresses enthusiasm; Believes in own capability as a learner; Shares excitement about learning with peers; Identifies aspects of school that are interesting and exciting
Consistently, independently is positive and optimistic
is positive and optimistic 75-89% of the time
is positive and optimistic 60-74% of the time
Needs ongoing teacher support and is often is not negative and pessimistic

22. 2nd and 3rd Grade SELF-KNOWLEDGE Meets Standard (4) Progressing (3)
Meets Standard (4)
Progressing
(3)
Emerging
(2)
Not Meeting Standard (1)
Identifies academic strengths and abilities
When asked, can name multiple strengths and abilities (e.g., “I am a good writer,” or “I have neat handwriting”
Consistently, independently names academic strengths and abilities
With occasional teacher support
names academic strengths and abilities
With frequent teacher support
Needs ongoing teacher support to name academic strengths and abilities
Identifies things he/she is interested in learning
When asked, can name multiple learning interests (e.g., “I like learning to read,” or “I want to learn more about butterflies”)
Consistently, independently identifies learning interests
identifies learning interests
Needs ongoing teacher support to identify learning interests

23. 2nd and 3rd Grade SELF-DIRECTION
Meets Standard (4)
Progressing
(3)
Emerging
(2)
Not Meeting Standard (1)
Demonstrates the ability to self-regulate actions and emotions
Sustains attention during activities (such as centers) and routines (such as circle time); Resolves or avoids conflicts with others; Exhibits safe behavior across school settings, throughout the day; Refrains from emotional outbursts (e.g., temper tantrums or major upsets)
Consistently, independently self-regulates
90-100% of the time
With occasional teacher support
self-regulates 75-89% of the time
With frequent teacher support
self-regulates 60-74% of the time
Needs ongoing teacher support and is often unable to self-regulate
Demonstrates resilience after setbacks
Recovers from (academic and social) disappointments and challenges without a lengthy disruption in engagement (e.g., being told that s/he completed an assignment incorrectly)
Consistently, independently demonstrates resilience
demonstrates resilience 75-89% of the time
demonstrates resilience 60-74% of the time
Needs ongoing teacher support and is often does not demonstrate resilience
Makes productive use of classroom time
Participates in activities as assigned; Stays focused on the task at hand; Completes assignments and activities on time
Consistently, independently is productive
is productive 75-89% of the time
is productive 60-74% of the time
Needs ongoing teacher support and often not productive

24. 4th and 5th grade RELATIONSHIPS Respects and accepts authority
Meets Standard (4)
Progressing
(3)
Emerging
(2)
Not Meeting Standard (1)
Works collaboratively in groups of various sizes
Listens to peers; Participates actively (e.g., sharing verbally and/or listening); Works with classmates to complete a task in the time given; Demonstrates flexibility taking on different roles during group work (e.g., can follow and lead)
Consistently, independently works collaboratively in groups
With occasional teacher support
works collaboratively in groups
With frequent teacher support
Needs ongoing teacher support and often does not work collaboratively in groups
Seeks assistance when necessary
Uses classroom resources and protocols (e.g., asking a peer, using a reference sheet) before asking an adult for help; Actively seeks the appropriate adult/s who can provide the needed support
Consistently, independently seeks assistance when needed % of the time
seeks assistance when needed 75-89% of the time
seeks assistance when needed 60-74% of the time
Needs ongoing teacher support and is often unable to seek needed assistance
Respects and accepts authority
Follows directions provided by all adults at school; Communicates politely with adults; Responds to limit-setting without disruptive outbursts; Knows and follows school rules
Consistently, independently respects authority
90-100% of the time
respects authority perseverance 75-89% of the time
respects authority perseverance 60-74% of the time
Needs ongoing teacher support and often disrespects authority figures
Forms respectful, equitable relationships with peers
Demonstrates care for classmates and has formed friendships with peers; Identifies peers by name; Cooperates with peers and can work with others towards a common goal (e.g., completing a group science project or playing in a soccer game at recess); Avoids or resolves interpersonal conflicts (during instructional and non-instruction times)
Consistently, independently forms respectful relationships with peers
forms respectful relationships with peers
Needs ongoing teacher support and often behaves disrespectfully towards peers

25. Counselors leading with data vs. collecting data
The beauty of this system for data collection is that students are rated by the teachers who are with them all day and have a picture of whether the student understands a construct AND whether he/she puts it into practice.
The counselor is the analyzer/interpreter of data trends, rather than the actual data collector.
The teacher is the beneficiary of additional data, interpreted by the school counselor, to help understand why a child may be struggling.
Together, teachers and counselors can have data based discussions about how to intervene.

26. What professional Development is necessary to support this approach?
Challenge #4
What professional Development is necessary to support this approach?

27. Teacher PD
Teachers needed to know WHY they were being asked to complete the PFI
They also needed to know HOW to complete it accurately

28. Counselor PD
Counselors needed to learn HOW TO EXTRACT the PFI data from the SIS
Counselors also needed to learn how to understand/analyze and use the data they were seeing
Counselors needed training in EXCEL
Counselors needed training in leading data based conversations with teachers and families
Counselors needed help identifying evidence based interventions that are available to support needs identified by the data at the universal [tier1] as well as Tier 2 level

29. PD formats Large group Embedded How/Why for teachers, families
Counselors have been charged with delivering on- going embedded professional development by presenting PFI data after each marking period and leading conversations about school wide trends at faculty meetings, grade level trends at Common Planning Time meetings and classroom and individual trends in consultation meetings with teachers.

30. Promising practices

31. #1: It is possible to collect socio-emotional Data via existing SIS systems
Teachers reported they do not feel overburdened by adding PFI information when they complete report cards
With training in exporting to EXCEL, as well as the filter, sort and chart making functions of EXCEL, counselors can use PFI data to examine district, school, grade level, classroom and individual trends relative to each of the factors on the PFI.
Counselors report it takes them 2-3 hours after each marking period to manipulate the data, analyze it and prepare charts

33. #2: The Rubric is a valuable tool
Each factor is described with concrete, observable behaviors and ratings are quantified
Analysis of the data …..Karen add here about reliability
Important to share with parents
Parental feedback has been very positive; parents agree socio-emotional factors are important and they value getting this information on a regular basis

34. #3 PFI data used to drive tiered interventions
Analysis of baseline data indicated that self- regulation was problematic at every grade at every school

35. Self regulation DATA from 4 elementary schools

36. Data driven interventions
Counselors revised their Tier 1 services to include targeted instruction on self- regulation to all classes in every grade via the developmental guidance curriculum
Teachers’ summer reading was “The Zones of Regulation”
Counselors led a faculty in-service on “The Zones”
Additionally, data is used to identify students needing additional support [Tier 2] so they can be placed in small groups focusing specifically on self-regulation

37. #4 Counselors are seen as leaders and data experts
Counselors are taking on a leadership role in helping their colleagues understand the connections between non-cognitive factors and academic achievement
Teachers and Administrators are having “ah- ha” moments where they see these connections and then are looking for strategies to help promote the acquisition of these factors
Counselors are collecting teacher led strategies to promote self-regulation and are publishing best practices for all.

38. A counselor developed PPT for 4th grade teachers’ common planning time

39. Protective Factor Index
Term 3 Data

40. What are Protective Factors?
The protective factor index highlights a different set of skills than those traditionally focused on on a report card. These 13 skills have been statistically linked to success in children. Therefore, this data is very important in understanding areas where our students are thriving and areas they need to improve in. By looking at this data, it can help us to be more focused on the type of interventions we put into place.

41. What are we using this data for?
Tier 1: Classroom lessons that target areas that a high number of students are struggling in
Tier 2: Social/Emotional groups are formed based on this data
Tier 3: Individual’s needs can be determined using this data and can lead appropriate interventions

42. Who produces this data?
YOU! This data comes from the report cards (specifically focusing on Term 3 for this power point). The report cards will have a 4 point scale, instead of just “struggling” or “on target” to more specifically indicate student growth.

43. #1) Demonstrates an eagerness to learn

44. #2) Demonstrates perseverance in completing tasks

45. #3) Demonstrates resilience after setbacks

46. #4) Demonstrates the ability to self-regulate actions and emotions

47. #5) Engages in class activities

48. #6) Exhibits positive and optimistic behavior

49. #7) Forms respectful, equitable relationships with peers

50. #8) Identifies academic strengths and abilities

51. #9) Identifies things he/she is interested in learning

52. #10) Makes productive use of classroom time

53. #11) Respects and accepts authority

54. #12) Seeks assistance when necessary

55. #13) Works collaboratively in groups of various sizes

56. Questions/Comments What did you notice? Where are we struggling?
Where are we doing well?
What interventions do we already have in place?
What interventions might be helpful for us to put in place to increase our number of on target students?

57. #5: District is exploring incorporating PFI at MS and HS
Superintendent is prioritizing budget funds to
support building more data collection infrastructure
Training MS and HS counselors to be leaders in data-driven conversations about the connections between non-cognitive factors and academic achievement

58. The crucial next steps
Over time it will become clear whether intervening to support the development of non-cognitive factors results in improved academic achievement.