1. Participatory Adult Learning Strategy II: Overview
Carol M. Trivette, Ph.D.
Asheville and Morganton, North Carolina
Concord, NH,
July 16, 2015

3. Purposes of the Presentation
Describe the key characteristics of an evidence-based approach to the implementation of professional development.
Share results from two studies where we used the PALS II approach in trainings projects.

4. A Lesson Learned from More Than 40 Years of Experience
No intervention practice, no matter its evidence base, is likely to be adopted and used if the implementation methods used to teach or train practitioners to use the practice are themselves ineffective. Therefore, concern for the characteristics of implementation practices that are associated with optimal learner and practitioner outcomes should be of paramount importance as part of implementation research.

5. Child and Family Benefits

6. Definition of Terms
Implementation practices include methods and procedures used by implementation agents (trainers, coaches, instructors, supervisors, etc.) to promote interventionists’ use of evidence-based intervention practices.
Intervention practices include methods and strategies used by intervention agents (teachers, therapists, clinicians, parents, etc.) to effect changes or produce desired outcomes in a targeted population or group of recipients (e.g., students with disabilities).

7. Implementation Practices Intervention Practices
Relationship Between the Two Types of Practices
Implementation Practices
Intervention Practices
Participatory adult learning
Coaching
Mentoring
Just-in-time training
Guided design
Accelerated learning
Interest-based student learning
Natural environment practices
Instructional classroom practices
Communication and language learning
Family systems intervention practices

8. Principles
Evidence-based intervention practices are a necessary—but not sufficient—condition to ensure optimal outcomes for students and families.
Practitioners must be trained on intervention strategies using evidence-based adult learning implementation practices.

9. Two Types of Evidence-Based Practices
Evidence-Based Intervention Practices
Early childhood intervention practices
Evidence-Based Implementation Practices
Adult learning methods

11. Development of the Participatory Adult Learning Strategy (PALS)a
Center for Early Literacy Learning
Research synthesis of studies of adult learning strategies
a Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1),

12. Research Synthesis of Adult Learning Studiesa
Research synthesis of studies of accelerated learning, coaching, guided design, and just-in-time training
58 randomized control design studies
2,095 experimental group participants and 2,213 control or comparison group participants
Combination of studies in university and non-university settings
Learner outcomes included learner knowledge, skills, attitudes, and self- efficacy beliefs
The influence of the adult learning methods on the learner outcomes was estimated by weighted Cohen’s d effect sizes for the differences on the post test scores for the intervention vs. nonintervention group participants
a Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1),

13. Efficacy of the Adult Learning Methods
Effect size for the intervention vs. nonintervention group comparisons is d = .42 (95% Confidence Interval = .36 to .47).

14. Efficiency of the Four Adult Learning Methods

15. Translational Findings
Process for unpacking and unbundling the key characteristics of the adult learning methods

16. Six Characteristics Identified in How People Learna Were Used To Code and Evaluate the Adult Learning Methods
Planning
Introduce
Engage the learner in a preview of the material, knowledge or practice that
is the focus of instruction or training
Illustrate
Demonstrate or illustrate the use or applicability of the material, knowledge
or practice for the learner
Application
Practice
Engage the learner in the use of the material, knowledge or practice
Evaluate
Engage the learner in a process of evaluating the consequence or outcome
of the application of the material, knowledge or practice
Deep Understanding
Reflection
Engage the learner in self-assessment of his or her acquisition of knowledge
and skills as a basis for identifying “next steps” in the learning process
Mastery
Engage the learner in a process of assessing his or her experience in the
context of some conceptual or practical model or framework, or some
external set of standards or criteria
a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press.

17. Effect Sizes for the Six Adult Learning Characteristics
Planning Application Understanding
Average effect sizes and 95% confidence intervals for the relationships between the six adult learning method characteristics and the study outcomes.

18. Most Effective Adult Learning Method Practices
Characteristic
Practice
Mean Effect Size
Introduction
Out-of-class learner activities/self-instruction
0.64
Classroom/workshop presentations
0.63
Pre-class learner exercises
0.54
Illustration
Trainer role playing/simulations
0.55
Learner-informed input
0.53
Practicing
Real-life learner application
0.94
Real-life learner application/role playing
0.86
Evaluation
Self assessment of strengths/weaknesses
Reflection
Identify performance improvement goals
1.27
Journaling/behavior suggestions
0.82
Mastery
Standards-based assessment

19. Cumulative Effects of Different Combinations of
the Most Effective Adult Learning Method Practices
Average Cohen’s d effect sizes and 95% confidence intervals for the relationship between different combinations (clusters) of adult learning methods characteristics and the study outcomes.

20. PALS II (Participatory Adult Learning Strategy)
FOUNDATIONS
Introduce and Illustrate
MASTERY
Active
Learner Involvement
APPLICATION
Deep Understanding and Next Steps
Practice and Assess
EVALUATION
Review and Reflect

21. Trainer and Trainee Roles in the Different Phases of PALS II
PALS Phases
Trainer Roles
Trainee Roles
Foundations
Preview learning topic
Complete pre-training preview
Describe key elements
Pre-class/workshop exercises
Provide examples
Provide input on the learning topic
Include trainee input
In-class/workshop warm-up exercises
Illustrate application
Demonstrate application
Application
Facilitate application
Provide examples of application
Observe trainee application
Trainee role playing, games, etc.
Provide in vivo feedback/guidance
Implement/practice use of the subject matter
Facilitate learner assessment of options
Evaluate use of the knowledge or practice
Evaluation
Establish learning standards
Standards-based evaluation
Engage learners in self-assessment
Conduct self-assessment
Provide guidance to learners
Trainer-guided learner reflection
Provide behavioral suggestions
Journaling
Group discussions of understanding
Mastery
Joint planning
Trainer guidance
Identify needed information/experiences
Trainer/trainee mentoring

22. PALS II
Overview of PALS II
training models
Video

23. Prompting Family-Systems Intervention
Study I
Prompting Family-Systems Intervention

24. Family-Systems Intervention Practices
Promoting Adoption of
Family-Systems Intervention Practices
473 IDEA Part C early intervention practitioners
5 types of training (presentations, day and multi-day workshops, field-based and enhanced field-based)
Participants were randomly assigned to complete the study outcome measure at one or six months after training
The outcome measure included items on the usefulness of the training and the extent to which the training improved their abilities to work with families

25. Six Characteristics Identified in How People Learna Were Used To Code and Evaluate the Adult Learning Methods
Planning
Introduce
Engage the learner in a preview of the material, knowledge or practice that
is the focus of instruction or training
Illustrate
Demonstrate or illustrate the use or applicability of the material, knowledge
or practice for the learner
Application
Practice
Engage the learner in the use of the material, knowledge or practice
Evaluate
Engage the learner in a process of evaluating the consequence or outcome
of the application of the material, knowledge or practice
Deep Understanding
Reflection
Engage the learner in self-assessment of his or her acquisition of knowledge
and skills as a basis for identifying “next steps” in the learning process
Mastery
Engage the learner in a process of assessing his or her experience in the
context of some conceptual or practical model or framework, or some
external set of standards or criteria
a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press.

26. Characteristics of the Different Types of In-Service Training
Practice Characteristics
Type of In-Service Training
Conference Presentations
Half-Day/
Full-Day Workshops
Multi-Day Workshops
Field-Based Training
Enhanced Field-Based Training
Trainer introduction of the practice + ++
Trainer illustration of use of the practice
Trainee application/use of the practice -
Trainee evaluation of his/her use of the practice
Trainee reflection on his/her learning
Trainee assessment of learner mastery
Multiple learning sessions
NOTE. - = No activity or opportunity, ± = limited opportunity, + = multiple opportunities, and ++ = multiple and varied opportunities.

27. Participants’ Judgments of the Benefits of the Five Types of Training

28. Interaction Between Hours and Type of Training for
Improved Family Systems Intervention Abilities

29. Principles
Training formats must be designed to ensure that evidence-based adult learning strategies can be used.
More time spent using the most effective strategies increases the learning rate.

30. Participant Chat
After looking at these data, what training formats/strategies come to mind that you can use that might support changes in practitioner’s practices?

31. Effectiveness Project
Study II
Head Start Teacher
Effectiveness Project

32. Head Start Teacher Effectiveness and Child Outcomes
36 Classrooms in Early Head Start Programs:
18 children per classroom
2 full-time adults per classroom
49 teachers and teacher assistants:
Average teacher age was 40 years
Average years in current position were 5.1 (SD = 4.7)
Collected the following data:
Background Demographics
Teacher Characteristics
Implementation Fidelity
Intervention Fidelity
Social Validity
Child Outcomes

33. Training Content
Training focused on two components of classroom practices known to promote positive child outcomes:
Child Learning Practices
Instructional Practices

34. Key Features
The key features of the professional development (PALS II) model include:
Introduce and Illustrate
Practice and Assess
Review and Reflect
Mastery
Active participation of teachers in their own learning
Ability to respond to and individualize training

35. Practice Reflection Checklist
Specify the practice indicators for each project component
Child Learning Practices: 18 indicators
Instructional Practices: 16 indicators
Help teachers understand key characteristics of the practices
Serve as the standards against which teachers examine and improve their practices

36. Training Tools Practice Scenarios Classroom Interest Assessment Tool
Practice Checklists
Child Observation Guides
Planning Tools (Child Learning Practices/ Instructional Practices)

37. Relationship Among the Two Types of Fidelity and Child/Parent Outcomes
Implementation Fidelity
Intervention Fidelity
Child/Parent Outcomes

38. “Coaching” Characteristics
Weekly sessions for 16 weeks
Session length: M = 2.5 hours, SD = 1.5, Range =
In classroom (93%)
Trainer introduces new information (72%)
Trainer provides illustration (94%)
Observation of classroom (95%)
Trainee practices specific skill (64%)
Trainee assess on practice (81%)
Trainer provides verbal feedback (98%)
Goal planning (83%)

39. Implementation Fidelity
Project Measures
Implementation Fidelity
Measure
Construct
HSTDI Training Record
Frequency, length, and mode of training
Teacher Engagement in Adult Learning Process
Introduction, illustration, evaluation, mastery, teacher receptiveness—Coach’s assessment
Teacher Training Feedback Form
Exposure, adherence, responsiveness regarding training process—Teacher’s assessment
Teacher Practice Feedback Form
Exposure, adherence, responsiveness regarding training on project practices or content—Teacher’s assessment

40. Outcome Measures Child Learning Practices Instructional Practices
Project Measures
Outcome Measures
Measure
Construct
Child Learning Practices
Interest-based child learning, Child learning activities, Increased learning opportunities
Instructional Practices
Child engagement, Teacher responsiveness, Response elaboration
CLASS
Emotional support, Classroom Organization, Instructional Support

41. Implementation Fidelity
Relationship Between Differences in Implementation Fidelity and Teachers’ Classroom Practices
Classroom Practices
Implementation Fidelity
Cohen’s d
Low
High
Mean SD
Child Learning Practices
Interest-Based Child Learning
9.22
2.59
10.80
1.81
.71
Child Learning Activities
29.00
6.34
32.90
2.23
.82
Increased Learning Opportunities
22.00
4.39
23.90
2.92
.51
Instructional Practices
Child Engagement
9.44
2.24
10.60
1.35
.63
Teacher Responsiveness
20.44
5.53
23.50
3.75
.64
Response Elaboration
18.78
4.66
21.80
3.68
.72
CLASS
Emotional Support
4.88
1.19
5.58
0.87
.67
Classroom Organization
4.75
0.68
5.01
.32
Instructional Support
1.70
0.73
2.09
0.99
.45
a Number of subscale items. b One-tailed test.

42. Implementation Fidelity
Relationship Between Differences in Training Implementation Fidelity and Follow-Up Observation of Teachers’ Classroom Practices
Classroom Practices
Implementation Fidelity
Cohen’s d
Low
High
Mean SD
Child Learning Practices
Interest-Based Child Learning
10.22
1.72
10.00
1.24
-.15
Child Learning Activities
31.89
2.93
32.50
2.91
.21
Increased Learning Opportunities
22.11
3.66
22.30
3.02
.06
Instructional Practices
Child Engagement
9.67
2.12
10.10
1.52
.23
Teacher Responsiveness
23.22
4.71
24.80
3.26
.39
Response Elaboration
20.33
3.60
21.50
.35
CLASS
Emotional Support
5.46
0.97
5.82
0.72
.42
Classroom Organization
5.16
0.86
5.13
0.96
-.03
Instructional Support
1.96
1.87
0.73
-.13

43. Study III
Center for Early Literacy Learning:
Train the Trainer Model

44. CELL Early Literacy Learning Model
Literacy-Rich Environments
Everyday Literacy Activities
Responsive Teaching
Early Literacy Learning Outcomes
CELL Early Literacy Learning Model
This is the CELL early literacy learning model. As you can see, the model includes five inter-related components including literacy-rich environments, child interests, everyday literacy activities, responsive teaching, and early literacy outcome indicators. These important components are the foundation for all of the early literacy practices found in the CELL practice guides that are located on our website at
The components of the early literacy learning model are displayed in an overlapping fashion because of the difficulty of separating them from each other in theory or in practice. We want you to understand that each component is important, but that early literacy learning results from an interrelated mix of these components in the context of your interactions with children. None of the components works independently of the others.
Child Interests

45. Principles
Active participation of the learner in all phases of the training is essential.
Individualize training to support learner mastery of the practice.
Encourage practitioners’ self-reflection at every phase of the process (Specific checklists can be helpful) to increase the rate of change in practitioners’ practices.
The more training strategies (introduce, illustrate, practice, etc.) used during training, the greater the change in practitioner practices.
Checklists are effective tools for engaging practitioners in self-reflection.

46. Methods of Assessing Implementation Fidelity
Implementation fidelity can be gathered through observation to determine if the trainer uses the correct training strategies.
Implementation fidelity data can be collected from trainees about whether the trainer provides the appropriate training content and strategies.
Implementation data ask the trainee to indicate the extent to which they feel competent with the practice content and training strategies they are supposed to use.

47. CELL Trainer Feedback Form
Content Items
1. The importance of literacy-rich environments was well explained by the trainer.
The key characteristics of responsive teaching were described and illustrated in ways that made the instructional practice easy to understand.
Training-Method Items
The importance of active participation of trainees as a method was made explicitly clear.
The training method was described in enough detail to understand its key elements.
Practice Items
A sufficient amount of time was devoted to each component of the CELL literacy model.
The activities for engaging trainees in learning the CELL literacy practices were especially helpful.

49. Questions or Reactions

50. Thank you