1. Sel in ymca afterschool project results
November 28, 2018
© 2017 YMCA of the USA

2. Table of contents Project goals & context Project learnings
Lessons learned & next steps
SEL in YMCA Afterschool Project Results

3. Project goals & context

4. SEL in YMCA Afterschool Project goal
This project sought to positively impact adult practices by providing Y Afterschool Programs with tools, resources, training, and technical assistance to assess, plan, and improve their program's quality and the social-emotional competencies of the young people they serve. The goal was that these efforts would improve the social-emotional development of participants, and ultimately their educational, career, and life success.
2 min
In a larger effort at Y-USA for a renewed focus on character development
WHAT IS YOUR DEFINITION OF PRACTICES?

5. SEL in YMCA ASP sites

6. ASSESS PLAN IMPROVE SEL in YMCA As project implementation process
Youth Social-Emotional Competencies
Staff Social-Emotional Competencies
Program Quality
PLAN
IMPROVE
TRAINING & DATA COLLECTION PHASE (August-December)
PLANNING WITH DATA PHASE (December-February)
IMPROVEMENT PHASE
(February- June)
DATA & CONTINUOUS QUALITY IMPROVEMENT-
Importance of linking SEL data and PQA data….how does one feed of other?
Planning with Data session- happening early next year, will be the opportunity to do this. All of the SEL practices and the PQA practices that we’re using here are linked to one another.

7. SEL in YMCA As project implementation process
ASSESS
PLAN
IMPROVE
TRAINING & DATA COLLECTION PHASE (August-December)
PLANNING WITH DATA PHASE (December-February)
IMPROVEMENT PHASE
(February- June)
Program Level
Assess the quality of programs and identify staff training needs
Analyze data reports, identify and discuss emerging needs using the data, and select methods targeted for program improvement
Execute the improvement plan to improve instructional program quality and meet staff needs
Data Collection Tools/
Resources
School-Age Program Quality Assessment (PQA)
PQA Report
Youth Work Methods Workshops
Participant Level
Observe social-emotional skills of participants
Analyze reports, identify and discuss emerging needs of participants using the data, and select strategies targeted for program improvement
Execute the improvement plan to improve participant’s social-emotional skills
What does this mean to you?
Can someone walk me through this?
Data Collection Tools/
Resources
DESSA-mini
*Follow-up with full DESSA for participants who demonstrate need for instruction
DESSA Report(s)
SEL Strategies

8. Staff social-emotional competencies
assessments
YOUTH Social-emotional competencies
Staff social-emotional competencies
The SEL in YMCA Afterschool Project sites utilized the Devereux Student Strengths Assessment (DESSA)-mini version (8 items), with an option to complete a full DESSA (72 items) for participants who demonstrated need for instruction to further develop their social-emotional competencies, at the beginning and end of the program year. ​
The DESSA is a standardized, strength-based measure of the social and emotional competencies of children in kindergarten through 8th grade. The DESSA focuses on 8 competencies of social-emotional learning: self-awareness, self-management, goal-directed behavior, social awareness, relationship skills, personal responsibility, decision-making, and optimistic thinking.
Following an assess-plan-improve sequence, out-of-school time staff assessed the social-emotional competencies of youth, utilized individual, group, or site-based data reports to create an improvement plan, and had access to SEL intervention strategies that help improve specific social-emotional competencies to carry out their improvement plans.
Studies show that high-quality youth programs can help children develop social-emotional competencies, which is a key predictor of a successful adult life¹. The goal of this project was to provide afterschool program staff with tools that they can use to build and improve their own social and emotional competencies as well as effective strategies to promote the development of social and emotional competencies in youth.
Developed by the American Institutes for Research, the Staff SEL Reflection tool focuses on five social and emotional competencies, including self-awareness, self-management/emotion regulation, social awareness, relationship/social skills, and responsible decision-making.
The tool is designed to help afterschool program staff reflect upon their own social and emotional competencies and their ability to support young people's SEL through program practices.
SOCIAL AWARENESS
SELF-MANAGEMENT/
EMOTION REGULATION
RESPONSIBLE
DECISION-MAKING
RELATIONSHIP/
SOCIAL SKILLS
SOCIAL-EMOTIONAL COMPETENCIES
SELF-AWARENESS
RESPONSIBILITY AND CHOICE
PROGRAM STAFF
LANGUAGE
WARMTH AND SUPPORT
SOCIAL PRACTICES
YOUTH-CENTERED PROBLEM-SOLVING
SELF-REFLECTION AND SELF-ASSESSMENT
GROUP DISCUSSIONS
BALANCED PROGRAM PRACTICES
PROGRAM IMPLEMENTATION PRACTICES
COOPERATIVE/GROUP LEARNING
ENCOURAGING GRIT AND PERSISTENCE
COMPETENCE BUILDING
SEL in YMCA Afterschool Project Results
¹ American Institutes for Research, 2015.

9. assessments Program quality
Research has shown that afterschool programs that are of high quality make larger impacts than those without¹. In order to provide programs of high quality to our youth participants, it is important to us to measure program quality at each of our programs to define what effective practice looks like, evaluate a site against a set of quality standards, and allow programs to self-assess, reflect, and improve upon their programs.
The School-Age Program Quality Assessment (S-PQA) is a validated tool that uses a continuous program quality improvement model and is designed to evaluate the quality of programs. It is based on observations of the program and contains items that focus on specific elements of best practice when working with children and youth. The pyramid on the left represents the four domains assessed by the S-PQA: Safe Environment, Supportive Environment, Interaction, Engagement.
The S-PQA is part of a continuous improvement process, utilizing an assess-plan-improve approach. The process began with assessment in order to build on out-of-school time staff’s existing strengths and identify areas to address. These areas then went into an improvement plan as specific, measurable, attainable, results-focused, and time-bound goals and benchmarks for success. Finally, there were strong supports for youth leaders to manage improvement, and the high quality Youth Work Methods series of workshops for staff were made available to all sites. As a whole, the assess-plan-improve sequence establishes a supportive system for continuous improvement.
Program quality
¹ Eccles, J.S. and J.A. Gootman (2002). Community programs to promote youth development. Washington, DC, National Academies Press.
SEL in YMCA Afterschool Project Results

10. Ymca of the usa site Support
SEL in YMCA Afterschool Site
Association Leads
Afterschool Online Community
Reminders and Resources
(NC) State Alliance
Learning Community
Calls
Evaluation & Technical Training and Support
SEL in YMCA Afterschool Project Results

11. Project learnings
SEL in YMCA Afterschool Project Results

12. demographics
The SEL in YMCA Afterschool Project was implemented in 30 afterschool sites across three states; California (5 associations in 5 sites), Minnesota (3 associations in 5 sites), and North Carolina (7 associations in 20 sites). Altogether, they completed DESSAs for a total of 1,200 participants in both Fall and Spring. Full DESSAs were completed for 140 participants.
About 47% of participants were boys and 82% of participants were in grades kindergarten through 4th grade.
State
Mini-DESSA (matched)
Full DESSA (matched)
California
300 18
Minnesota
241 2
North Carolina
659
120
TOTAL
1,200
140
This slides shows matched data for all participants. Overall, there were 2,050 participants overall for which there is data (pre OR post)
SEL in YMCA Afterschool Project Results

13. changes in social-emotional competencies
121 youth moved from need to typical or strength
Something to note for pre-data
116/1200 (~10%) had “perfect” scores
217/1200 (~18%) had scores within 5 points of perfect
NOT a normal distribution (see next slide)
AG Last year: Need 15% to 9%
Typical 66% both
Strength 19% to 25%
SEL in YMCA Afterschool Project Results

14. changes in social-emotional competencies by state
California
minnesota
North carolina

15. Changes in social-emotional competencies california
In the state of California, DESSA-minis were completed for 300 participants at the beginning and the end of the program year.
There was a decrease in the number of participants in need for instruction*, from 21% to 14%, as well as an increase of participants in a typical range of social-emotional competencies, from 71% to 77%. Of the 62 participants who began the year in need for instruction, 37 (60%) moved to typical or strength ranges by the end of the program year.
*T-scores on the DESSA are categorized as follows: T-Scores of 60 and above indicate that the child has social and emotional strengths, T-scores of inclusive indicate that the child is showing a typical amount of positive behaviors related to social and emotional competence, and T-scores of 40 and below indicate that the child has a need for instruction to further develop social and emotional skills as compared to the national standardization sample.
SEL in YMCA Afterschool Project Results

16. Changes in social-emotional competencies minnesota
In the state of Minnesota, DESSA-minis were completed for 241 participants at the beginning and the end of the program year.
There was a decrease in the number of participants in need for instruction*, from 8% to 6%, as well as an increase of participants in a typical range of social-emotional competencies, from 68% to 74%. Of the 18 participants who began the year in need for instruction, 16 (88%) moved to typical or strength ranges by the end of the program year.
*T-scores on the DESSA are categorized as follows: T-Scores of 60 and above indicate that the child has social and emotional strengths, T-scores of inclusive indicate that the child is showing a typical amount of positive behaviors related to social and emotional competence, and T-scores of 40 and below indicate that the child has a need for instruction to further develop social and emotional skills as compared to the national standardization sample.
SEL in YMCA Afterschool Project Results

17. Changes in social-emotional competencies north carolina
In the state of North Carolina, DESSA-minis were completed for 659 participants at the beginning and the end of the program year.
There was a decrease in the number of participants in need for instruction*, from 13% to 9%, as well as an increase of participants in a typical range of social-emotional competencies, from 53% to 78%. Of the 86 participants who began the year in need for instruction, 70 (81%) moved to typical or strength ranges by the end of the program year.
It should be noted that of the 34% of the participants that began in the strength range, 97 participants (43%) were given a perfect social-emotional competency score at the beginning of the program year, possibly accounting for the decrease in the strength range percentage from the beginning to the end of the program year.
SEL in YMCA Afterschool Project Results
*T-scores on the DESSA are categorized as follows: T-Scores of 60 and above indicate that the child has social and emotional strengths, T-scores of inclusive indicate that the child is showing a typical amount of positive behaviors related to social and emotional competence, and T-scores of 40 and below indicate that the child has a need for instruction to further develop social and emotional skills as compared to the national standardization sample.

18. changes in social-emotional competencies
Typically, if full DESSAs were completed, you should see all or most of the youth in the need category at the beginning of the program year, but there was one site that completed full DESSAs on all of their participants (didn’t do a mini at all).
In the SEL in YMCA Afterschool Project, full DESSAs were completed for 140 participants at the beginning and the end of the program year.
There was a decrease in the number of participants in need for instruction*, from 24% to 21%, as well as an increase of participants in the strength range of social-emotional competencies, from 11% to 33%. Of the 33 participants who began the year in need for instruction, 11 (33%) moved to typical or strength ranges by the end of the program year.
SEL in YMCA Afterschool Project Results

19. Staff social-emotional competencies
State
Self-Reflection (Fall & Spring)
California 2
Minnesota 8
North Carolina 10
TOTAL 20
Across the three states, 20 staff completed the Staff SEL Reflection tool at the beginning and end of the program year.
The charts demonstrate the percentage of staff who felt competent in the areas of social-emotional competencies, staff practices, and program implementation practices at the beginning and end of the program year.
SEL in YMCA Afterschool Project Results

20. Staff social-emotional competencies
SEL in YMCA Afterschool Project Results

21. Program quality PQA Scores Improvement Youth Work Methods Plans
Responsibility (2)
Reflection (5)
School-Age Planning (5)
School-Age Choice (1)
School-Age Leadership (5)
Manage Feelings (1)
Active Engagement (5)
Skill Building (3)
Encouragement (1)
Emotional Safety (1)
ASSESS
PLAN
IMPROVE
PQA Scores
(n=27)
3.54 /5
3.98 /5
3.89 /5
4.57 /5
One of the goals of the SEL in YMCA Afterschool Project was to implement the program quality improvement process during the program year. Of the 30 afterschool sites that participated, 27 completed the S-PQA, with higher scores on the lower levels of the pyramid and the lowest score in the Engagement area.
Of the 27 sites that completed the S-PQA, 29 improvement plan goals across the four areas of the pyramid.
The improvement plan goals focused heavily in the areas of Supportive Environment and Engagement.
SEL in YMCA Afterschool Project Results

22. Lessons learned & next steps
SEL in YMCA Afterschool Project Results

23. Lessons learned Bright Spots
Learning community provided space for sites to share successes, challenges, and be accountable
Growth in staff competencies may have been supported by A-P-I cycle
State alliance model was key in continuing work in NC
Positive experiences of using the data to inform practice
Evaluation support was key for direct access to resources, tools, and learnings
Challenges
Staff turnover was a challenge at many sites
Sites in Associations with similar work and those that took advantage of site “buddy” pairs felt more supported
Time to complete assessments and process was a challenge at many sites
SEL in YMCA Afterschool Project Results

24. Next steps
All sites have access to assessment systems for the program year
Sites in North Carolina will continue to check-in via the Learning Community calls and possibly seek additional funding
Two Associations in North Carolina expanding work to additional sites
SEL in YMCA Afterschool Project Results

25. Maria D. Guzmán-Rocha Research & Evaluation maria.guzman@ymca.net
SEL in YMCA Afterschool Project Results