1. Evidence Based Practice 3
Summarising papers and assessing their merits

2. Session Plan 2 tasks: Develop a structured summary for each paper
Tea Break
Assess the methodological merits of each paper

3. Why do a structured summary?
Ensure you record the important information
Give you an easily comparable summary of the aim, method, measures, findings and conclusions of each paper.
To facilitate this it is easiest to use a template such as the one included in your pack.

4. Template 1 Full Reference Details of the paper:
All authors
Full title of the paper
Publication Date
Journal name
Volume number and page numbers (OR doi)
So that when you lose your copy you can find it again easily

5. Template 2 Summarise the aim of the study Summarise the study method
Into one sentence if possible
Summarise the study method
Qualitative, case study series
Quantitative, semi-structured interview
Mixed methods, list
Pilot study?

6. Template 3 Describe the sample selection procedures: Random /purposive
Specific characteristics
Gender
Age limits
Specific diagnosis or problem
Number sought and reason for this
How many participants?
How was this number decided?

7. Template 4 Summarise what participants were asked to do
This may be different if you have more than one group in the study
What measures were made?
How were these measures undertaken?
Were any researchers involved in measurements?

8. Template 5 If the study involved randomisation how was this done?
Who was aware of the grouping?
If researchers were involved in measurements, were they blinded to the intervention?
Were participants blinded to the intervention they received?

9. Template 6 How was data analysed? Summarise the main results
Were the appropriate procedures used?
In a qualitative study what procedures were used to ensure rigour in the data analysis?
What happened if participants withdrew?
Summarise the main results
Concentrate on results that are important for answering your question

10. Template 7 What conclusions have the authors drawn?
Are these reasonable given the study methods and procedures?
Are these consistent with the results?
Leave a space for inserting a ‘quality’ score
This may be in sections for a qualitative paper or it may be numerical for quantitative papers.

11. Fill in one template for each of the papers you have located and read.

12. Assessing the merit of each paper
We are not going to assess the quality of the writing or the writing style (this is usually heavily influenced by editorial staff of the journal).
We are interested in noting the parts of each paper that contribute to improving the scientific merit of the study and therefore the weighting we would give to the study.

13. Selecting a framework to use
Comparisons between papers are clearest when all papers have been assessed on the same points
Value of the assessment is increased when all of the important methodological points are included in the assessment
Different types of studies have different methods for reducing bias so the framework needs to be appropriate for the type of study

14. Possible frameworks
PEDro GRADE PRISMA Downs and Black COSMIN AGREE QADAS Daly et al

15. PEDro 1. Eligibility criteria were specified 2. Random allocation
3. Concealed allocation
4. Baseline similarity between groups
5. Subject blinding
6. Therapist blinding
7. Assessor blinding
8. Follow-up > 85%
9. Intention-to-treat analysis
10. Between group statistical comparisons
11. Point measures and measures of variability reported.
Item scoring: 1 = present, 0 = absent. Criteria 1 is not included in the total score

16. Identify the framework
What framework is most likely to be applicable to the two papers you have summarised?
Get a copy
Read what you are asked to score to confirm that it is applicable

17. Using the framework
Assess each of your 2 papers using the selected framework
Record the outcome for each assessment point for each paper
What does this indicate about each paper?
How could each paper have been improved?
Are there any points you think should have been included in the assessment that are missing?

18. Using the framework When you use any assessment framework consider:
For each assessment item does the paper include the important information?
In a randomised controlled trial are you given the method of randomisation?
Is it a true randomisation?
Are groups matched?
Does the matching cover the important attributes?

19. Next time Pulling the results together Making tables and graphs
Working out how to reconcile conflicting results
Reporting your findings

20. Preparation Read any other papers you have located
Make a structured summary for each paper
Score each papers on the framework you have used
Bring the structured summaries of all of your papers (including the ones you have done today)