1. EVALUATING STATISTICAL REPORTS
HOW TO PASS AS 91266
EVALUATING STATISTICAL REPORTS

2. EVALUATE WITH STATISTICAL INSIGHT
TICK SHEET
Achievement
2A1+3A2+3A3
Merit
1M1+3M2
Excellence
1E1+2E2
DESCRIBE
EVALUATE
JUSTIFY
EVALUATION
EVALUATE WITH STATISTICAL INSIGHT
Title
Source
Purpose A1 A3 M1 E1
Population measures A2 M2 E2
Variables
Sampling methods
Sample size
Survey methods
Sampling errors
Non- sampling error
Findings of the survey
Summary

3. VARIABLES DESCRIBE LIST AT LEAST 3 OF THE ACTUAL VARIABLES MEASURED
CLASSIFY AS:
CATEGORICAL
ORDINAL
QUALITATIVE
NUMERICAL
AND
EVALUATE WRT PURPOSE

4. POPULATION MEASURES DESCRIBE EVALUATE
MEAN MEDIAN MODE QUARTILES STANDARD DEVIATION
ADVANTAGES DISADVANTAGES SUGGESTIONS OF WHICH IS THE BEST TO HAVE USED FOR THIS DATA.

5. SAMPLING METHODS DESCRIBE EVALUATE ADVANTAGES DISASVANTAGES
PROBABILITY SAMPLES Systematic Cluster Stratified Simple Random NON-PROBABILITY SAMPLES Person in the Street Self-Selected Quota sampling
ADVANTAGES
DISASVANTAGES
SUITABILITY WRT PURPOSE

6. SAMPLE SIZE DESCRIBE EVALUATE WRT ORIGINAL POPULATION SIZE
ALLOWABLE SAMPLING ERROR
PURPOSE OF REPORT

7. SURVEY METHODS DESCRIBE EVALUATE
FACE TO FACE INTERVIEW
TELEPHONE SURVEY
COMPUTER-ASSISTED SURVEY
SELF-ADMINISTERED SURVEY
CLUSTER GROUPS
PAPER QUESTIONNAIRE
WEB-BASED QUESTIONNAIRE
ADVANTAGES AND DISADVANTAGES WRT PURPOSE OF REPORT

8. SAMPLING ERRORS DESCRIBE EVALUATE SAMPLING ERROR IS AFFECTED BY:
SAMPLE SIZE
SAMPLE DESIGN
VARIABILITY WITHIN THE POPULATION
INCREASING SAMPLE SIZE TENDS TO REDUCE SAMPLING ERROR
RECOMMEND A DIFFERENT SAMPLING METHOD
RECOMMEND HOW TO CHOOSE A MORE REPRESENTATIVE SAMPLE

9. NON-SAMPLING ERRORS DESCRIBE EVALUATE EXPLAIN HOW THEY OCCURRED
COVERAGE ERROR
RESPONSE ERRORS
NON-RESPONSE ERRORS
PROCESSING ERRORS
ANALYSIS ERRORS.
EXPLAIN HOW THEY OCCURRED
DISCUSS WAYS TO REDUCE SAMPLING ERRORS

10. NON-SAMPLING ERRORS describe evaluate
The wording of questions, the order in which they are asked and the number and type of options offered can influence survey results.
Answers given by respondents do not always reflect their true beliefs because they may feel under social pressure not to give an unpopular or socially undesirable answer.
Answers given by respondents may be influenced by the desire to impress an interviewer.
If people who refuse to answer are different, with respect to survey issues, from those who respond .This can also happen with people who are never contacted and people who have yet to make up their mind.
If the excluded or under-represented group is different, with respect to survey issues, then bias will occur.
This creates bias because individuals with strong opinions about the survey issues or those with substantial knowledge will tend to be over- represented.
Bias can occur because respondents may tend consistently to have views that are more extreme than those of the population in general.
Bias will occur.

11. FINDINGS OF THE SURVEY
DESCRIBE
EVALUATE

12. SUMMARY

13. REMEMBER! Effectiveness Must relate this to the purpose Justification
Must include supporting evidence
Must always make clear links to the context, including reference to the background information

14. How to get a/ m / e Source Summary Purpose
Achievement
Achievement with Merit
Achievement with Excellence F
Identifies & comments on at least 3 different features
Same as Achieved but adds statistical evidence from the report and discusses the processes used
Same as Merit but:
examines the quality of the survey process (loaded questions, questions changing from year to year, framing of questions to get a positive response, adequate sample size)
examines contextual information (small footnotes with important information eg small response for a question, further clarification necessary, discussion on some data omitted)
When discussing the purpose and results there are further questions such as: are there supporting studies?, is there funding tied to this report which may promote bias?, is there other data available?
Discussion of how the report could be improved such as: change data representations and displays, not change questions between years, continue the survey over a longer time.
Source
Summary
Purpose
Discusses results with respect to the purpose
Features
Population measure and variables
Sampling methods
Survey methods
Sampling and possible non-sampling errors
Sample size
Effectiveness
Must relate this to the purpose
Justification
Must include supporting evidence
Must always make clear links to the context, including reference to the background information