1. Use of Portfolios in Specialist Training Programs to Demonstrate Positive Impact on Student Learning
Laurie McGarry Klose, Ph.D., LSSP
Associate Professor and Director, School Psychology
Trinity University
San Antonio, TX

2. Learning Objectives
Participants will learn how portfolio products can be utilized to demonstrate measurable positive impact on students.
Participants will observe the utility of electronic portfolio platforms for managing program approval data.

3. Agenda
Review course requirements preparing future interns to evaluate impact of services.
Examine internship requirements related to demonstrating positive impact.
Look at information from Trinity's last NASP approval process.
Discuss generation of impact data charts.
Questions and comments.

4. Measurable Positive Impact Data

5. Coursework Preparation
Counseling Class
Behavior Analysis and Intervention Class
Advanced Statistics Class
Research Evaluation Class

6. Internship Requirements
Completion of two consultation cases and reports- one that addresses academic concerns and one that addresses behavioral problems. Your cases should use single-case designs and document positive impacts on student learning and behavior, respectively. Reports should include goals, charts, graphs and specific data.
Completion of two counseling cases- individual and/or group. Document with summary of goals and outcomes of both. One case must be summarized in a report using Goal Attainment Scaling and will be assessed using the attached rubric.

7. Documentation
Assessment 6

8. This assessment is based on data collected by students during Internship and presented in their Portfolios. As described in Assessment 5, Interns are required to complete several specific types of cases during this year. A minimum of two consultations are expected and the process of determining outcomes is specifically taught prior to this final course. Experience and practice with this is begun with the first year spring class ED 5384 Behavior Management and Special Education with the behavior change project. The fall second year course ED 6381 Research Methods II has a case study project as the major assignment as does the second year spring class ED 6380 Consultation Theory and Methods. One of the Internship consultations must address a behavior problem and the other an academic problem. A sequence and approach with required statistics is familiar to the students and the rubric has also been utilized previously in Practicum; however, at this point, a positive outcome is expected which must be demonstrated through both Goal Attainment Scaling and an Effect Size calculation. In addition, a Social Validity scale and a Behavior/Academic Intervention Rating Scale must be completed by the consultee.

9. Included are the data from two cohorts
Included are the data from two cohorts. In the first class, there were problems with one of the outcome scores for several cases; however at least one outcome score was accurately presented for each case. Faculty learned from the procedural weaknesses apparent in these case presentations and the second group’s data met our expectations completely. Most importantly, all the cases demonstrated a positive impact on students’ academic and behavioral performance in school settings.

10. 2012-2013 Academic Consultation Cases
Intern
Student’s
Grade
Problem/Goal
BIRS
Goal Attainment
Scale
Effect Size DB
3rd
Turn in assignments
5.71 +2
5.23 MC
5th
Use math strategies
4.14 +1
12.6 TC
1st
Word recognition
5.29
6.31 LM
9th
Work completion
4.7
2.5 AM
11th
4.29
2.68 EM K
Handwriting improvement 4 DR
Utilize ELL strategies
4.71
6.43 LS
Subtraction using manipulatives 5 *
7.67 TY
2nd
Math facts 6
8.16
Means
4.87
+1.86
6.25

11. 2012-2013 Behavioral Consultation Cases
Intern
Student’s Grade
Problem/Goal
BIRS
Goal Attainment Scale
Effect Size DB
1st
Increase on-task behavior 5 +2 6 MC
5th
Attempt work
5.3 * TC
3rd
Remain seated
5.29 +1 11 JG
Decrease off-task behavior
3.14 AM
8th
Compliance with adult requests
5.14
1.97 EM
2nd
Reduce off-task behaviors
4.28 DR
Increase on-task behaviors
5.33
4.7 GT
4th
Focus on work 4 TY
Decrease task refusal
3.6
Means
4.79
+1.56
5.53

12. 2013-2014 Academic Consultation Cases
Intern
Student’s Grade
Problem/Goal
BIRS
Goal Attainment Scale
Effect Size CB
2nd
Increase reading fluency 5 +1
3.9 ED
3rd
Increase math skills 4
2.84 EF
Increase addition skills 6 +2
6.9 LF
12th
Completing work
7.78 RF
1st
Letter identification
3.8
2.9 CL
Reading fluency
3.86
2.29 DM
Answering math problems
3.88
4.29 LP
5th
Using tutoring services
3.57
12.5 AW
Increase sight words
5.9
Means
4.56
+1.33
5.08

13. 2013-2014 Behavioral Consultation Cases
Intern
Student’s Grade
Problem/Goal
BIRS
Goal Attainment Scale
Effect Size CB K
On-task behavior
4.7 +1 3 ED
3rd
Time in class
2.9 +2
8.87 EF
4th au
Remaining calm
5.42 .9 LF
3rd au
Eat lunch 5
2.89 RF
6th
On-task behaviors 4
1.4 CL
5th
Time on task
4.2
2.01 DM
4th
Increase time on task
4.43
3.15 LP
Increase coping skills
4.29
2.83 AW
2nd
Follow school rules
5.9
3.8
Means
4.54
+1.22
3.2

14. NASP Program Approval Board Response
Element 9.6 is rated as Acceptable.
The school psychology program employs a systematic and valid process by which program faculty ensures that interns demonstrate competencies to begin effective practice as school psychologists, evidenced by direct, measurable, positive impact on children, families, schools, and other consumers.
The program provides comprehensive data that candidates demonstrate positive effects on P- 12 student learning. Candidates analyze data from case studies and are required to select and compute effect size and goal attainment scaling.

18. Comments, Questions, Discussion