1. The University of Texas System Assessment of
Teacher Preparation Programs
Funded by the Houston Endowment
September, 2004
9/18/2018

2. Overview of Presentation
National Center for Educational Accountability
Introduction to the study
Research questions
Indicators
Data sources
Data security and confidentiality
Teacher Survey
Data collection (Mary Lummus-Robinson)
School district partnership (Holly Williams)
Questions & discussion
9/18/2018

3. National Center for Educational Accountability
Created in 2001
Sponsoring organizations
Education Commission of the United States
Just for the Kids
The University of Texas at Austin
9/18/2018

4. Introduction to the study
Purpose:
To identify relatively high performing teacher certification programs in the UT System and to determine what makes these programs so successful and what staff from other programs can learn from them.
9/18/2018

5. Introduction to the study
UT System requested development of a system for evaluating teacher preparation programs
4-year study of graduates from 9 UT System components
4 cohorts of graduates beginning in
teaching in Texas public schools
grades 4-8
grades 9-11 on an exploratory basis
9/18/2018

6. Introduction to the study
Unique contributions of the study
Value-added assessment of student growth
Colleges of Education evaluated on the basis of the performance of the graduates’ students
Translation of evaluation results into improvement of teacher preparation programs
9/18/2018

7. Research Questions
Is teacher preparation program related to students’ growth on the Texas Assessment of Knowledge and Skills (TAKS) when controlling for teachers’ prior academic achievement and for school characteristics?
9/18/2018

8. Research Questions
Is teacher preparation program related to graduates’ performance on teacher certification exams when controlling for graduates’ prior academic achievement?
9/18/2018

9. Research Questions
Is teacher preparation program related to graduates’ self-reported teaching ability when controlling for graduates’ prior academic achievement and for school characteristics?
9/18/2018

10. Research Questions
What are the characteristics and content of the most highly effective teacher preparation programs?
9/18/2018

11. Indicators
Academic growth of graduates’ students as measured by the Texas Assessment of Knowledge and Skills (TAKS)
Graduates’ teacher certification exam scores (TExES/ExCET)
Graduates’ self-reports of their teaching ability as indicated on the Teacher Survey
Qualitative data on the characteristics and content of each program
9/18/2018

12. Data Sources School districts
State Board for Educator Certification (SBEC)
Graduates
UT System components
Texas Education Agency (TEA)
9/18/2018

13. Data Security and Confidentiality
Protect confidentiality of the data
Merge datasets
Encryption algorithm developed at TEA
Shared with UT System and SBEC
School district data to TEA for encryption
Files merged at NCEA after encryption
9/18/2018

14. Teacher Survey
Interstate New Teacher Assessment and Support Consortium Core Standards
How well do you do the following?
How well did your teacher preparation program prepare you to do the following?
How well have your in-school training experiences prepared you to do the following?
9/18/2018

15. Teacher Survey
Interstate New Teacher Assessment and Support Consortium Core Standards (item examples)
Adjust plans to meet students’ needs
Foster relationships with colleagues
Communicate student progress to parents
9/18/2018

16. Teacher Survey Rating of overall teaching ability
Attributions of teaching ability
Teacher preparation program
In-school training experiences
Natural ability
Other
9/18/2018

17. Data Collection Overview
Data sources
5, 297 Teacher Preparation Program Graduates
1,675 Texas High School Principals
9 UT System academic components
Texas Education Agency (TEA)
State Board for Educator Certification (SBEC)
102 school districts
9/18/2018

18. Teacher Survey Administered online March-August 2004
5,297 Graduates sampled for online survey
15% Response Rate
6 attempts made to reach target graduates
Introductory letter
Follow up letter
Two reminders
Two mailings including a hard copy of the survey
9/18/2018

19. Data Collection UT Components Graduates’ prior academic achievement
Transcript data
Aggregates for college
Specific data collected at each institution varies
State Board of Educator Certification
Certification type and level
Exam scores
Years of experience
Employment information
9/18/2018

20. Data Collection Texas Education Agency
Teacher and student demographic data
Holding all district data submitted
Teacher Preparation Program Staff Survey
Specify unique attributes of each program
100% response rate
9/18/2018

21. Data Collection School Districts
Districts procedures for submitting research request vary
102 districts contacted
8 had detailed procedures
Followed set procedures
94 had no procedures
NCEA developed generic request
9/18/2018

22. Data Collection School Districts
Between 4-12 contacts made with each district
Introduction
Specific Data Request
Follow up via Mail, Phone Calls, and
Follow up Data Request
Repeat
9/18/2018

23. Data Collection School Districts
102 School Districts contacted
51 Districts participated in study
51 Districts declined
35 Districts had no response
6 Districts had problems extracting data
5 Districts declined citing strain on resources
4 Districts’ superintendents declined participation
1 District refuses all research unless state mandated
9/18/2018

24. District Participation
Austin Independent School District
Participation in the NCEA Project will help lead to
Ongoing improvement in UT System teacher preparation programs
A more Highly Qualified applicant pool for our District
Academic growth for AISD students
9/18/2018

25. New Teachers New teachers in AISD, 2004-05 800 new teachers hired
300 of these hold an alternative certification
Special interest in findings regarding student outcomes for teachers who are alternatively certified
Results and recommendations in this area will likely impact our future hiring practices
9/18/2018

26. Research in AISD
Due to an increase in requests by external groups to conduct research in AISD, in 1996, our Dept. formalized a Research Proposal Application and Review Process (similar to an IRB process)
Researcher submits application describing methodology of the study and the benefit to district ( N=70)
Committee of staff with expertise in the appropriate program area(s) and staff with expertise in research methodology review application
Project is approved, approved with modifications, or declined
9/18/2018

27. NCEA Project NCEA submitted application
Project was approved because of the clear benefit to AISD
Many of our teachers come through the UT system;
The content area of Teacher Preparation is of interest, particularly in light of NCLB requirements;
The design of the study and the evaluation system was well thought out and clearly articulated; and
The project did not infringe upon student or campus staff time.
9/18/2018

28. District Expectations
We look forward to reaping the benefits of this project
Expectations include
Best Practice findings will be used to make modifications, where recommended, to teacher pre-certification training in the UT system
Outcomes regarding alternatively certified teachers will help district staff make more knowledgeable hiring decisions
The evaluation system will be used for continuous improvement in UT system teacher preparation programs
9/18/2018

29. Wrap-up Questions Available information Business cards
Higher Achievement for All Students (NCEA)
9 Essential Elements of Statewide Data Collection Systems
9/18/2018