The Campbell Collaborationwww.campbellcollaboration.org Education Panel Session Comments and Points for Discussion Sandra Jo Wilson Vanderbilt University Editor, Campbell Collaboration Education Coordinating Group

The Campbell Collaborationwww.campbellcollaboration.org Truancy Interventions Focused on students already exhibiting attendance problems. Broad inclusion criteria in terms of study designs. Statistically significant effects on attendance outcomes. – Also, significant variability in those effects. – And, some question about the clinical significance.

The Campbell Collaborationwww.campbellcollaboration.org

What moderates the effects of truancy interventions? A variety of moderators was examined, including publication type and research design, participant characteristics, and intervention characteristics. No moderator was significantly associated with larger or smaller treatment effects. Small sample makes it difficult to detect significant moderators. But, interesting that different modalities had similar effects, and that participant characteristics were not influential.

The Campbell Collaborationwww.campbellcollaboration.org Volunteer Tutoring Update of a previous review Intervention scope is similar to the attendance intervention review Participant scope is broader Broader array of outcomes Inclusion regarding research designs more stringent Statistically significant affects on most of the achievement outcomes – Comprehension and mathematics not significant

The Campbell Collaborationwww.campbellcollaboration.org Updating a Previous Review Including newly available literature and broadening the geographic scope This should give us greater opportunities for examining variability New innovations in meta-analysis methodology make updating reviews attractive from a methodological standpoint, in addition to the obvious policy relevance.

The Campbell Collaborationwww.campbellcollaboration.org Discussion Points What challenges did you face in narrowing down your question? What challenges did you face in gathering necessary data from source studies? What areas of support would make it easier for you to do a review like this in the future? What were the difficult choices you had to make during the study collection process or the review process?

The Campbell Collaborationwww.campbellcollaboration.org Scoping a Review The research question – Are you interested in whether a particular program is effective vs. are you interested in what best improves some problem? – What research designs would be the best for answering such a question? What designs are actually used.

The Campbell Collaborationwww.campbellcollaboration.org Other Scoping Considerations Your constituents – Your funding agency – The users Budget

The Campbell Collaborationwww.campbellcollaboration.org Challenges in the Conduct of a Review Definitional and language challenges – Has implications for your search, for the scope, for how you combine (or not) outcomes in a meta- analysis, and for moderator analysis What is truancy, how is it different from school refusal or attendance problems? Programs with different labels but the same content? Missing or vaguely reported data in primary studies

The Campbell Collaborationwww.campbellcollaboration.org Support and Training for Conducting a Review Creating the ideal review team Content and methods experts Making contact with other groups who may have access to different resources – Can help with the search and with definitional issues

The Campbell Collaborationwww.campbellcollaboration.org Other Discussion Points? Reporting relevant and useful reviews – How do we create useful systematic reviews? Questions for our review authors? Other topics?

The Campbell Collaborationwww.campbellcollaboration.org Thank you to Brandy and Sarah And our audience! Sandra Jo Wilson