Presentation on theme: "Carlo Magno, PhD Lasallian Institute for Development and Educational Research De La Salle University, Manila."— Presentation transcript:
Carlo Magno, PhD Lasallian Institute for Development and Educational Research De La Salle University, Manila
Work in a group List down in the manila paper four ways on how you use test/assessment data in your school. Discussion during presentation
EVIDENCES Improvement of the teaching and learning process Improvement of counseling processes
Learning and innovation skills Creativity and Innovation Critical Thinking and Problem Solving Communication and Collaboration Information, media, and technology skills Information Literacy Media Literacy ICT (Information, Communications and Technology) Literacy Life and career skills Flexibility and Adaptability Initiative and Self-Direction Social and Cross-Cultural Skills Productivity and Accountability Leadership and Responsibility
21 st century skills Assessment International Level Assessment National Level Assessment Regional/District Level Assessment Classroom Level
Help generate assessment information about learners. ASCA Standards Counseling standards in the RP
Informs development of academic programs and special programs Informs further improvement of the curriculum Identifies sections/students that needs further help Reflection on how to teach or deliver the curriculum better Decisions on the allocation of resources and priorities Informs what is happening in the schools (academic standards)
Achievement gains per class/section Achievement gains per subject area Achievement gains per level Trends: Comparison across school years Trends: Comparison with other schools/countries/states/region
Standardized tests Standards-based test Results from inventories Teacher-made tests Interview data Data from teachers Data from parents
Involves two variables where one increases with the other Examples: Grades and motivation: Does student motivation increase with students’ grades? Attitude in Math and Math performance: Does students’ attitude in math increase with their performance in math achievement test? Math anxiety and test in math: Does anxiety decrease math test scores? The choice between the variables should be guided by a theory (theoretical or conceptual framework). Both variables should be quantitatively measured.
Linear Regression There is a straight line relationship between variables X and Y When X increases, Y also increases-positive relationship When X increases, Y decreases or vice versa – negative relationship
Problem: Is there a significant relationship between achievement and aptitude? Hypothesis: There is a significant relationship between achievement and aptitude
Analysis 2 variables that are interval or ratio: Pearson r 2 variables are ordinal: Spearman rho 2 variables and each is a dichotomy: phi coefficient High Satisfaction in teaching Low satisfaction in teaching High teaching performance 5021 Low teaching performance 1248 A significant relationship occurs if scores are extreme enough to surpass the probability of error. If p value is < obtained value: reject the null hypothesis If the obtained value > critical value : reject the null hypothesis
Direction or direction Strength Significance Variance
Involves group formed in categories (2 or more) and these categories are compared on an characteristic. The groups are called as the independent variable The characteristics of where the groups are compared on are called as the dependent variable. Examples: Is there a significant difference between males and females on their math performance? Is there a significant difference between public and private school students in their study habits? Are there a significant differences among the school ability of students from across three years (2010, 2011, 2012)? Are there significant differences among teachers, administrators, and staff on their attitude towards the RH bill?
Take note that the IV... is categorical can have two or more levels can also be more than one.... Example: Can gender and socio-economic status differentiate students general intelligence? A theoretical or conceptual framework is needed to justify the comparison.
Case: Third year high school males and females are tested in their Mathematical Ability
Males: Mean = 20.14 SD=3.48 Females: Mean = 27.33 SD = 5.89
1. H 1 = There is a significant difference between males and females on their math scores 2. =.05 df = N1 + N2 –2 df = 7 + 6 –2 df = 11 t critical value = 2.201
3. Computation t = X 1 - X 2 x 1 2 + x 2 2 1 + 1 N 1 + N 2 – 2 N 1 N 2 t = - 2.73
4. Decision and Interpretation Since the t obtained which is – 2.73 is greater than the t-critical which is 2.201, the null hypothesis is rejected. This means that there is a significant difference between males and females in their math scores. Females (M=27.33) significantly scored higher on math as compared to the males (M=20.14)
4. Decision and Interpretation (another way using p values) Since the p value obtained which is 0.0195 is less than the alpha level which is.05, the null hypothesis is rejected. This means that there is a significant difference between males and females on their math scores. Females (M=27.33) significantly scored higher in math as compared to the males (M=20.14)
Independent Variable B A1A2A3 B1A1 B1A2 B1A3 B1B1 Mean Main Effect for B B2A1 B2A2 B2A3 B2B2 Mean A1 Mean A2 MeanA3 mean Main Effect for A Main effect of A Main Effect of B Interaction effect of A and B (A X B)
Talent Achievement Effect of Achievement and Type of school on Talent Low AchieversHigh Achievers Type of school Public school Private School
H1: Achievement have a significant main effect on talent (there is a significant difference between high and low achievers on talent) Type of school have a significant main effect on talent (there is a significant difference between public and private school students in their talent) There is a significant interaction effect between achievement and type of school (there are significant differences among high achievers in public, high achievers in private, low achievers in public, and low achievers in private in their talent Effect of Achievement and Type of school on Talent
Analysis If two categories are compared on one DV: t-test for two independent samples If three or more categories (one IV) are compared on one DV: One way Analysis of Variance (ANOVA) If two IV are investigated on one DV: two way ANOVA If two or more IV are investigated on two or more DV: Multivariate Analysis of Variance (MANOVA)
Carlo Magno, PhD De La Salle University, Manila
The effect of a treatment is tested on a specific change on a characteristic. The treatment that is given to participants are called as the independent variable. The independent variable should be manipulated. Ex. Groups are randomly assigned to listening and watching stimulus to enhance their memory. Ex. Groups are randomly assigned to reading a text or watching a news to enhance their recall of the information. The characteristic that changes dues to the variation or manipulation of the IV is called as the dependent variable.
How is the IV manipulated? Presence of absence Amount Type
The effect of think-aloud reading on the reading comprehension of grade 8 students. 1 st group: think-aloud while reading 2 nd group: silent reading
The effect of cognitive load of concept on the recall of college students. 1 st group: 200 words to study 2 nd group: 500 words to study 3 rd group: 800 words to study 4 th group: 1,000 words to study 5 th group: 1,200 words to study
The effect of labeling on the teachers conduct assessment of students Results Trouble makerslow conduct AverageAverage conduct Ideal studentsHigh conduct
In an experiment done by dela Cruz, Cagandahan and Arciaga (2004), the effect of nonbehavioral intervention techniques was investigated on the computational abilities of fourth year high school students. The non-behavioral intervention techniques has three levels, bibliotherapy, small group interaction and games. These techniques were used as a teaching strategy in a lesson in a math class for three sections. Each of the strategy was used for each section. One section did not receive any strategy which served as the control group. After undergoing the strategy, the students were tested where they answered a series of computation items.
BibliotherapySmall group interaction GamesControl Group X1X2X3X4 X1X2X3X4 X1X2X3X4 X1X2X3X4
1. H 1 : The non-behavioral intervention techniques have a significant effect on computational ability H 1 : There are significant differences among the groups receiving bibliotherapy, small group interaction, games and control in their computational ability. 2. 2 =.05 df between = groups – 1 = (4-1=3) df within = (N – 1) – df between ((209-1)- 3)=205 df total = df between + df within (3 + 205) F ratio critical value = 2.65
3. Computation F ratio computed = 4.62 4. Decision and Interpretation Since the F ratio obtained which is 4.62 is greater than the F ratio critical which is 2.65, the null hypothesis is rejected. The non-behavioral intervention techniques have a significant effect on computational ability.
The group who received the small group interaction significantly scored the highest among other intervention techniques.
The hypothesis needs to be backed up by a theory. The effect of IV on the DV should be strictly controlled, no other factors should affect the DV other than the IV. Extraeneous variables (show examples) Techniques of constancy
Research Design – Refers to the outline, plan or strategy specifying the procedure to be used in seeking an answer to the research question True Research Designs - Answers the research questions or adequately tests hypothesis. Extraneous variables are controlled Inclusion of a control group External validity - Generalizability
1. After-Only Design Dependent variable is measured only once and this measurement occurs after the experimental conditions have been administered to the experimental group. TreatmentResponse Measure Experimental ConditionXY Control ConditionY Between Subjects Design – If different subjects are used in each experimental treatment condition. Within Subjects Design – If the same subjects are used in each experimental condition.
1.1 Between-Subjects After Only Design subjects are randomly assigned to the experimental and control group.
Simple Randomized Subjects Design Includes more than one level of the independent variable
Factorial Design Two or more independent variables are simultaneously studied to determine their independent and interactive effects on the dependent variables. Main effect – influence of one independent variable Interaction effect – Influence that one independent has on another
Within Subject After-Only Design Same subjects are repeatedly assessed on the dependent variable after participating in all experimental treatment conditions
Combined Between- and Within-Subjects Designs Factorial Design Based on a mixed Model Two independent variables have to be varied in two different ways. One independent variable requires a different group of subjects for each level of variation. The other independent variable is constructed in such a way that all subjects have to take each level of variation.
2. Before-After Design The treatment effect is assessed by comparing the difference between the experimental and control groups’ pre- and posttest scores.
The Solomon Four-Group Design - Designed to deal with a potential testing threat. - Testing threat occurs when the act of taking a test affects how people score on a retest or posttest. - The design has four groups - Two of the groups receive the treatment and two does not. - Two of the groups receive a pretest and two does not. - By explicitly including testing as a factor in the design, we are able to assess experimentally whether a testing threat is operating.
Switching Replications Design - There is a need to deny the program to some participants through random assignment. - A two group design with three waves of measurement. - The implementation of the treatment is repeated or replicated. - In the repetition of the treatment, the two groups switch roles: - The original control group becomes the treatment group in phase 2 while the original treatment acts as the control. By the end of the study all participants have received the treatment.
Randomized Block Design Constructed to reduce noise or variance in the data Requires that the researcher to divide the sample into relatively homogeneous subgroups or blocks. Then, the experimental design desired is implemented within each block or homogeneous subgroup. The key idea is that the variability within each block is less than the variability of the entire sample. Thus each estimate of the treatment effect within a block is more efficient than estimates across the entire sample
What are the three approaches in conducting a study?
Construct a plan for your classroom research Research Question Hypothesis What conceptual/theoretical framework will be used? (be ready to explain) Method ▪ Experimental Design ▪ Participants (who and how many) ▪ Instruments used (how will you measure the DV?) ▪ Procedure