Presentation on theme: "Quantitative Methods Topic 1 Research Design. 2 Subject Aims Data analysis methods appropriate for investigating issues across a range of topics in education."— Presentation transcript:
Quantitative Methods Topic 1 Research Design
2 Subject Aims Data analysis methods appropriate for investigating issues across a range of topics in education. Conceptual understanding of statistics rather than formal mathematical derivations Univariate and bivariate statistics. Skills in questionnaire design
3 Assessments There will be 10 exercises (30%), and 1 project (4000 words) (70%)
4 SPSS is required SPSS (Statistical Package for the Social Sciences)
6 Reading materials for Topic 1 Module1.pdf: Educational research: some basic concepts and terminology T. Neville Postlethwaite
7 Topic 1 outline Types of educational research Planning educational research Research questions
8 Issues/headlines in Education Australian students lag in maths, science Funding splurge fails to improve student results Best assessment practice Boys lag behind girls Accountability policies Use technology in the classrooms
9 Types of Research Questions addressed by quantitative methods Descriptive Association/Correlational Causal/Explanatory
10 Descriptive Problems - 1 Which subject fields are more frequently chosen by Australian Year 11 and 12 students? Do girls choose different subject fields compared with boys? If so, what are the subjects preferred by girls? Are there differences in subject choice according to differences in social status and ethnic family backgrounds? What are the areas that Professional Development Program for teachers should cover? Are all primary school teachers qualified to teach? What are the status of school building in a country?
11 Descriptive Problems - 2 May not be as straightforward as the computation of frequencies and averages. Break-in On the radio, an advertisement for an insurance company ran as follows: Every 10 minutes, a car is stolen in Zedland. Every 21 minutes, a house is broken into. Take up an insurance policy today. Using only the information given in the advertisement, can you conclude anything about the chance a car will be stolen in Zedland? that it is more likely to have a car theft than a house break- in? Give reasons to support your answer.
12 Association - 1 Are there any relationships between students reading achievement and their mathematics achievement? Is there an association between reading achievement and the number of books in the home ? Association does not always lead to causal.
13 Association - 2 In a town in Europe, the number of storks is positively correlated with the number of babies born. Crime rate is positively correlated with ice cream sales.
14 Causal relationships - 1 Does smaller class size increase student performance? Do students performance improve if more homework is assigned? Does parental attitude have an impact on student achievement?
15 Causal relationships - 2 Causal relationship is very difficult to identify. Number of books at home is positively correlated with student achievement. Can student achievement be improved by placing more books in a home? Mediating variables High parental expectations of achievement is correlated with both number of books at home and student performance.
16 Causal relationships - 3 Study design to establish casual relationships needs to be confirmatory than exploratory. Confirmatory approach E.g., we hypothesise that increasing homework can increase student performance. Carry out a study where some students are assigned more homework and some are not. Exploratory approach Obtain student performance data and other information including homework, school administration, hours of teaching, etc. Find correlations between student performance and other variables.
17 Research Stages Stage 1: Research aims Stage 2: Literature Stage 3: Research design Stage 4: Instrumentation Stage 5: Piloting Stage 6: Data collection Stage 7: Data cleaning and Data analysis Stage 8: Research report
18 Research aim(s) Example: To identify factors influencing student withdrawal from school and explore the extent to which each factor contributes to student withdrawal from school
19 Literature Review Review the related literature What have been done in the field? What are controversial debates? School of thought? What are the gaps? (knowledge and methods) What are the findings? Develop a theoretical framework
20 Research questions What are the factors that influence students academic achievement? Which method of instruction is most effective in teaching young children to read?
21 Propositions Gender, ethnicity, family economic status, parents education and occupation, parents and students attitudes, were important factors relating to students' academic achievement. School effectiveness has an impact on student achievement Teacher qualification has an impact on student achievement
22 Research design At this stage the following should be identified: Source of information Who is appropriate to provide the necessary information Characteristics of the target population Data collection methods
23 Data collection methods Cross-sectional vs longitudinal If longitudinal, how many times? Sample vs cohort If sample, how many? Questionnaire, test, interview, published statistics How many questions?
24 Establishing the link between information needed, source of information and methods of data collection Information needed/variables to be measured Source of information Methods of data collection
25 Link between the information needed, sources and methods Information neededSourceData collection Student genderStudentsQuestionnaire Family characteristicsparentsQuestionnaire Parent attitudesParentsQuestionnaire Student attitudesStudentsQuestionnaire Student achievementStudentstests School characteristicsSchool/PrincipalQuestionnaire/interview
26 Questionnaires Needed Student questionnaire Parent questionnaire School questionnaire
27 Instrumentation Develop/validate and pilot instruments (test or questionnaires)
28 Variables included in the Parent Questionnaire Family social status/wealth Mother Education Father Education Mother Occupation Father Occupation Family size Parent attitudes
29 Variables included in the Student Questionnaire Student gender Ethnicity Student attitudes Student academic achievement Student behaviour
30 Data collection and data management Field work supervision Entering the data into data file Cleaning the data
31 Data Analysis Descriptive Correlational Causal Note that statistics can only provide correlational information. Any causal interpretation is made by people.
32 Writing up the reports and discussions Technical report Policy report General public
33 A few examples to contemplate Margins of error in test scores For Year 5 numeracy, each child is tested on just 40 questions each year in a national test. If David obtained 25 out of 40 on the 2009 test, how much would we expect Davids scores to vary if tests similar to the 2009 test are administered? For a 40-question test, Davids scores might vary by as much as 5 score points. In percentage terms, if a students score is 70% on a test, we expect the range of this students scores on similar tests to be between 58% and 82%.
34 Simpsons paradox MenWomen Arts 3 out of 4 (75%)1 out of 1 (100%) Sci 0 out of 1 (0%)1 out of 4 (25%) total 5 men and 5 women apply for university places (http://en.wikipedia.org/wiki/Simpson%27s_paradox)
35 False positives Prevalence rate for a disease in a population is 1%. Test for this disease -ve/-ve 95% +ve/-ve 5% (5% of those who do not have the disease show a +ve result) +ve/+ve 95% -ve/+ve 5% (5% of those who have the disease show a -ve result) Is this a good test? Suppose 10,000 people were tested. 600 had +ve result. What is the proportion of the people with +ve result who actually have the disease?