1 Introduction to Policy Processes Dan Laitsch

2 Overview (Class meeting 4) Sign in Agenda –Cohort break outs –Review last class –Mid term assessment –PBL Groups –Significance [dismiss] –Policy and unifying content –T-tests –PBL groups –Action research –PBL and dismiss

3 Class : Review Stats –Hypothesis testing –Z scores PBL –Topic determined Policy –Role Play

4 Cohort Break Out Courses and dates (summer session) –EDUC 813: organizational Theory (Drescher) April24/25, May 8/9, May 22/23, June 6/7, June 19/20, and June 26/27 Summer Institute –EDUC 822: Evaluation of Educational Programs July 2, 3, 6, 7, 8, 9, 10, 13, 14, 15, 16. (Mornings 8:30 to 1:30 or Evenings 4:30 to 9:30). SI public lecture times included as part of class hours (July 6, Evening; 7,9,14 and 16, 1:00 pm to 3:00 pm). Action Research Time Frame Comprehensive exams

5 Midterm Assessment Data drive decision making –What do the following data “tell” you? –What questions do they leave unanswered? Analysis and response

6

7

8 Response Heavy workload –Addressing past student concerns –Creating balance –Unifying vision Possible solutions –Goals: meet course description (policy processes) –Prepare students for Action Research –Continued tomorrow

9 PBL groups Touch base Status check –Group functioning? Forming, storming, norming, performing? –Topic identified? –Action plan? –Turn in report (handout) Plan for tomorrow –2-3 hours of group time (2 break out 1 hour to 1.5 hours each)

10 Part IV: Significantly Different Using Inferential Statistics Chapter 9 Significantly Significant What it Means for You and Me

11 What you learned in Chapter 9 What significance is and why it is important –Significance vs. Meaningfulness Type I Error Type II Error How inferential statistics works How to determine the right statistical test for your purposes

12 The Concept of Significance Any difference between groups that is due to a systematic influence rather than chance –Must assume that all other factors that might contribute to differences are controlled

13 If Only We Were Perfect… Significance level –The risk associated with not being 100% positive that what occurred in the experiment is a result of what you did or what is being tested The goal is to eliminate competing reasons for differences as much as possible. Statistical Significance –The degree of risk you are willing to take that you will reject a null hypothesis when it is actually true.

14 The World’s Most Important Table

15 Type I Errors (Level of Significance) The probability of rejecting a null hypothesis when it is true Conventional levels are set between.01 and.05 Usually represented in a report as p <.05

16 Type II Errors The probability of rejecting a null hypothesis when it is false As your sample characteristics become closer to the population, the probability that you will accept a false null hypothesis decreases

17 Significance Versus Meaningfulness A study can be statistically significant but not very meaningful Statistical significance can only be interpreted for the context in which it occurred Statistical significance should not be the only goal of scientific research –Significance is influenced by sample size…we’ll talk more about this later.

18 How Inference Works A representative sample of the population is chosen. A test is given, means are computed and compared A conclusion is reached as to whether the scores are statistically significant Based on the results of the sample, an inference is made about the population.

19 Deciding What Test to Use

20 Test of Significance 1. A statement of null hypothesis. 2. Set the level of risk associated with the null hypothesis. 3. Select the appropriate test statistic. 4. Compute the test statistic (obtained) value 5. Determine the value needed to reject the null hypothesis using appropriate table of critical values 6. Compare the obtained value to the critical value 7. If obtained value is more extreme, reject null hypothesis 8. If obtained value is not more extreme, accept null hypothesis

21 The Picture Worth a Thousand Words

22 Glossary Terms to Know Significance level Statistical significance Type I error Type II error Obtained value –Test statistic value Critical value

23 End of Class PBL Work if time allows Clarifying grades Journal, portfolio, stats notebook Homework: –Thinking about research What areas are you thinking about? What questions do you have? Prepare to chat with colleagues tomorrow

24 Agenda –Policy and unifying content –T-tests –PBL groups –Action research –PBL and dismiss

25 Unifying themes Diffusion models –Communication networks Diffusion of innovation Adoption –Internal (policy window) Severity (crisis) Opportunity –External (policy borrowing) National Regional Leader-Laggard Isomorphism (similar states) Vertical

Unifying themes Internal (policy window) –Severity (crisis) –Opportunity Evidence/Data (my insert) –Research –Statistics External (policy borrowing) –Governments (CMEC) –Organizations (CTF, JCSH, CERC-CA) Problem Solution Policy Study

Unifying themes Problems –Identification (what is the problem) –Analysis (what is the cause) Solutions –Research (what has been done) –Context (how does it fit here) Policies –Action (what are the rules and procedures) Evaluation –Analysis (what happened) –Refinement (what might we change) PBL Research Reviews Action Research Policy Analysis

Unifying themes Leadership –Identifying context (observation and data gathering) Data gathering and synthesis (problem identification) Identifying parameters (policy analysis) –Setting direction (goals and outcomes) Research (identify interventions) Policy (identify rules and procedures for action) Analysis (identify consequences) –Achieving Goals (problem solving) Implementation of actions and activities Application of rules and procedures (policy) –Evaluation (refining context)

Part IV: Significantly Different Using Inferential Statistics Chapter 10 t (ea) for Two Tests Between the Means of Different Groups

What you learned in Chapter 10 When to use a t test How to compute the observed t value Interpreting the t value and what it means

t Tests for Independent Samples Determining the correct statistic

Computing the Test Statistic Numerator is the difference between the means Denominator is the amount of variation within and between each of the two groups

Degrees of Freedom Degrees of freedom approximate the sample size Degrees of freedom can vary based on the test statistic selected For this procedure… n 1 – 1 + n 2 – 1

So How Do I Interpret… t (58) = -.14, p >.05 –t represents the test statistic used –58 is the number of degrees of freedom –-.14 is the obtained value (from the formula) –p >.05 indicates the probability (n.s.) p = n.s. –p <.05 indicates the probability (sig.)

Special Effects… Effect size is a measure of how different two groups are from one another Standardized difference between to group means Jacob Cohen

Computing Effect Size Small = Medium = Large =.50 and above

Effect Size Calculator

Glossary Terms to Know Degrees of freedom t Test –Independent t Test –Obtained value –Critical value Effect size

Part IV: Significantly Different Using Inferential Statistics Chapter 11 t (ea) for Two (Again) Tests Between the Means of Related Groups

What you learned in Chapter 11 When to use a t test for dependent means How to compute the observed t value Interpreting the t value and what it means

t Tests for Dependent Samples Determining the correct statistic

Computing the Test Statistic Numerator reflects the sum of the differences between two groups

Degrees of Freedom Degrees of freedom approximate the sample size Degrees of freedom can vary based on the test statistic selected For this procedure… –n – 1 (where n is the number of observations)

So How Do I Interpret… t (24) = 2.45, p >.05 –t represents the test statistic used –24 is the number of degrees of freedom –2.45 is the obtained value (from the formula) –p >.05 indicates the probability (n.s.) p = n.s. –p <.05 indicates the probability (sig.)

45 PBL Groups Break into groups Lunch

46 Action Research Pair share Model and paper process –Observations –Questions –Data –Methods –Analysis Discuss

47 PBL Groups PBL Work if time allows