Coding Procedures (Data Extraction) Purpose 1.Identify characteristics that may provide an objective explanation of intervention effects. 2.Provide the.

Slides:

Advertisements

Similar presentations

Defining Characteristics

Advertisements

Types of Experimental Designs Non-experimental –Post-test only –Pre-post test Experimental –Pre & post-test with control group –Pre & multiple post-test.

Experimental Research Designs

Quasi-Experimental Design

Experimental Designs.

Introduction to Research Design Threats to Internal Validity Two or More Groups Social Threats.

QUASI-EXPERIMENTS w Compare subjects in different conditions on a DV w Lacks one or more criteria for an experiment (cause, comparison, control) w Interpreted.

Experimentation. What experiments are not: Two commonsense uses of “experiment”: – any study – to “try something out” – …both of these understandings.

Non-Experimental designs: Developmental designs & Small-N designs

QUASI-EXPERIMENTS What is a Quasi-Experiment?

Questions What is the best way to avoid order effects while doing within subjects design? We talked about people becoming more depressed during a treatment.

Summary of Propensity Score Matching in Education

Controlling Variance Variance can cause misinterpretation of the results Variance is found in the population under study 4 ways to control variance: Randomization.

Chapter 11 Quasi-Experimental Designs ♣ ♣ Introduction   Nonequivalent Comparison Group Design   Time-Series Design   Regression Discontinuity Design.

1 Exploring Quasi-experiments Lab Assignment #5 Due May 16, 2008 Locate an empirical journal article that employs a quasi-experimental design. Closely.

Chapter 9 Experimental Research Gay, Mills, and Airasian

S-005 Types of research in education. Types of research A wide variety of approaches: –Theoretical studies –Summaries of studies Reviews of the literature.

The World Bank Human Development Network Spanish Impact Evaluation Fund.

Experimental Design The Gold Standard?.

Writing an APA-Style Research Report

Research Design Methodology Part 1. Objectives  Qualitative  Quantitative  Experimental designs  Experimental  Quasi-experimental  Non-experimental.

The fundamentals of caring for you, your family and your child with mental illness Paul Deal, Ph.D. Missouri State University.

Overview of MSP Evaluation Rubric Gary Silverstein, Westat MSP Regional Conference San Francisco, February 13-15, 2008.

Selecting a Research Design. Research Design Refers to the outline, plan, or strategy specifying the procedure to be used in answering research questions.

Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,

Dr. Bonnie J. Faddis & Dr. Margaret Beam RMC Research Fidelity of Implementation and Program Impact.

Developing Evidence-Based Products Using the Systematic Review Process Session 3/Unit 9: Inclusion and Exclusion Criteria Screening and Coding Studies.

Methodology Describe Context & setting Design Participants Sampling Power Analysi s Interventions Outcome (study variables) Data Collection Procedures.

Induction to assessing student learning Mr. Howard Sou Session 2 August 2014 Federation for Self-financing Tertiary Education 1.

Chapter 11 Experimental Designs

Planning an Applied Research Project Chapter 7 – Forms of Quantitative Research © 2014 John Wiley & Sons, Inc. All rights reserved.

S-005 Intervention research: True experiments and quasi- experiments.

1 Session 3: Framework for a Coding Guide October 27, :00-3:30 PM (EDT) Instructor: Chad Nye.

Chapter 8 Causal-Comparative Research Gay, Mills, and Airasian

Assisting GPRA Report for MSP Xiaodong Zhang, Westat MSP Regional Conference Miami, January 7-9, 2008.

Quantitative and Qualitative Approaches

Classifying Designs of MSP Evaluations Lessons Learned and Recommendations Barbara E. Lovitts June 11, 2008.

Evaluating Impacts of MSP Grants Hilary Rhodes, PhD Ellen Bobronnikov February 22, 2010 Common Issues and Recommendations.

Chapter 3 should describe what will be done to answer the research question(s), describe how it will be done and justify the research design, and explain.

Introduction section of article

Chapter 12 Quasi-Experimental and Single-Case Designs.

Experimental Research

Chapter 10 Common Experimental Research Designs

7.0 Evaluating Reviews of Research. Systematic Review (Meta-Analysis)

Evaluating Impacts of MSP Grants Ellen Bobronnikov Hilary Rhodes January 11, 2010 Common Issues and Recommendations.

Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition

Research Design ED 592A Fall Research Concepts 1. Quantitative vs. Qualitative & Mixed Methods 2. Sampling 3. Instrumentation 4. Validity and Reliability.

Chapter 10 Finding Relationships Among Variables: Non-Experimental Research.

Designs of Quasi-Experiments Studies for Assessing the Transport Enhancements and Physical Activity.

Experimental & Quasi-Experimental Designs Dr. Guerette.

Evaluation Designs Adrienne DiTommaso, MPA, CNCS Office of Research and Evaluation.

An Action Research Project Chevonne Byas EDUC 7021 T Fall 2010 Overeating and Underachieving: The Effects of Physical Fitness on the Academic Achievement.

The Power of Comparison in Learning & Instruction Learning Outcomes Supported by Different Types of Comparisons Dr. Jon R. Star, Harvard University Dr.

Types of Experimental Designs (Educational research) True Experimental Quasi-Experimental.

IREL 561: Research Methods Fall 2013 Week 10 Based largely on Neuman’s Basics of Social Research, Chapter 8 Prepared by Craig Webster, Ph.D.

Chapter 14 Research Synthesis (Meta-Analysis). Chapter Outline Using meta-analysis to synthesize research Tutorial example of meta-analysis.

CONDUCTING EDUCATIONAL RESEARCH Guide to Completing a Major Project Daniel J. Boudah Chapter 5 Designing and Conducting Experimental Research Lucy B. Houston,

S-005 Intervention research: True experiments and quasi- experiments.

School of Public Administration & Policy Dr. Kaifeng Yang 研究设计 : 实验研究的基本问题.

Chapter 11 Experimental Designs PowerPoint presentation developed by: Sarah E. Bledsoe & E. Roberto Orellana.

Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.

Chapter 9 Scrutinizing Quantitative Research Design.

Issues in Evaluating Educational Research

Ron Sterr Kim Sims Heather Cruz aka “The Carpool”

Designing an Experiment

Evidence Based Practice 3

Quantitative Research

Impact Evaluation Methods

Quasi-Experimental Design

Explanation of slide: Logos, to show while the audience arrive.

Presentation transcript:

Coding Procedures (Data Extraction) Purpose 1.Identify characteristics that may provide an objective explanation of intervention effects. 2.Provide the reader with a transparent standard of the extracted data 6.0

A. Publication Source 1.Journal2. Dissertation 3.Thesis 4. Unpublished Paper 5. Book 6. Other: ________ B. Subject Characteristics (page(s): ____________) Group Pretest Posttest Attrition Age Grade(s) (n) (n) (n) (n)(n) (n) Tx: ______ _____ ______ ______ ______ _____ Cp: ______ _____ ______ ______ ______ _____ Co: ______ _____ ______ ______ ______ _____

C. Socioeconomic Status (page(s):___________ 1.Low2. Middle 3. Upper 4. Low-Middle5. Middle Upper6. Labeled Mixed 7.Unclear 8. ___________9. NR Comments:__________________________________

D. Sample Source (page(s)_________) 1.Public School 2. Secular Private School 3. Religious Private School 4. Other:_________________ 5. NR Comments

E. Study Setting (page(s) ______) 1.Urban2. Suburban 3. Rural Geographic Setting: ______________________ Comments:

F. Author’s Labeling of Participants’ Achievement Levels (page _____) 1.Average Achieving 2. High Achieving 3. Underachieving 4. At-Risk 5. Unclear 6. Other:___________________7. NR G. Race/Ethnicity (page ______) 1.African American 2. American Indian3. Asian 1.4. Hispanic/Latino 5. White 6. Other: ____________________7. NR

H. Intervention Characteristics 1.Type of intervention (page __________________) 2.Total length of intervention program: __________ or NR (page _____) 3.Length of time PER intervention session:________ or NR (page _____) 4.Number of sessions: ________ per ________ or NR (page _____) 5.Describe the intervention(s) intended for delivery to the study groups including how and when it was administered. ______________________________________________

6.Fidelity of implementation data reported? (page(s)______) 0=No 1=Yes If yes, describe the intervention(s) intended for delivery to the study groups including how and when it was administered: ________________________________________________ ________________________________________________ 7. Intervention implemented as described? (page(s) _____) 1=No 2=Yes 3= Somewhat 9=NR

I. Outcome Measure(s) Type of Measure Instrument Variable(s)Who Administered the Measurement Instrument? 1. Norm Referenced Test (page____) _________ _________ _________ a.Parent b. Teacher c. Researcher d. Other e. NR 2. Criterion Referenced Test (page____) _________ _________ ________ a. Parent b. Teacher c. Researcher d. Other e. NR 4. Rating/Survey (page____) _________ a. Parent b. Teacher c. Researcher d. Other e. NR 5. Class Grade (page____) _________ _________ _________ a. Parent b. Teacher c. Researcher d. Other e. NR

J. Design Characteristics Recruitment Pool (page_____) 1.Referral 2. Criterion3. Norm Reference Test 4. Existing Group5. Cohort 6. Volunteer 7.Other ___________________ 9. NR Research Design Type (pg____) Randomized Design 1.Group (Cluster, Place-based) Randomized Design 2.Quasi-Experiment: Interrupted Time Series Design* 3.Quasi-Experiment: Regression Discontinuity* 4.Quasi-Experiment: Nonequivalent Comparison Group Design* 5.Pre-Experimental: Single Group Pre/Posttest Design* 6.Single Subject Design* 7.Other Designs: _____________________________________

K. Assignment Procedures Subject Assignment (page(s) _____) 1.Individual Random 2.Group Random 3.Individual Match Random 4.Quasi-random

Method of Random Allocation (page(s) _____) Method of Random AllocationWho Implemented the Random Allocation? 1. Random Numbers (page____)a.Parent b. Teacher c. Researcher d. Other e. NR 2. Coin Flip (page____)a. Parent b. Teacher c. Researcher d. Other e. NR 3. Other:_______(page____)a. Parent b. Teacher c. Researcher d. Other e. NR

Was concealment of allocation used as part of the randomization procedures? 1=no 2= yes 3=NR a. If yes, were the allocation procedures concealed from any of the following people? 1. Researcher (pg _____) 2. Participant (pg _____) 3. Intervener (pg _____) 4. Assessor (pg _____)

L. Quasi-Experimental Assignment Procedures (If applicable, otherwise skip to section M) 1. Was any of the following research design techniques used to equate groups prior to the intervention? a. Matching on pretest proxy (pg _____) b. Matching on a pretest (pg _____) c. Matching on other variables (pg _____) d. No Matching (pg _____) 2. Was statistical equating used prior to the intervention? 1=no 2=yes3=NR Comments:________________________________________

M. Effect Size Characteristics Comparisons: Group 1: _________vs. Group 2:_________ Outcomes __________ __________ ___________ Groups __1__ _2_ __1__ __2__ __1_ __2__ Mean __________ _____ _____ ____ _____ SD _____ _____ _____ _____ ____ _____ N at Posttest _____ _____ _____ _________ _____ F value _____ __________ __________ ____ Chi-square _____ __________ __________ ____ p value ______________________________ t value ______________________________

Method of Analysis (pg _____) a. Intention to Treat: b. Treated Participants Only

7.0: Excerpt from Master Coding Guide for Study Coding Code the Fantuzzo Study (7.1)