PSY 250 Hunter College Spring 2018 Detecting Effects of an Experimental Manipulation Data Collection and Analysis PSY 250 Hunter College Spring 2018

Organizing and Summarizing Data Data Sheets: Used in collecting and summarizing data. In a research report data sheet(s) may placed in the Appendix of the paper. A data sheet in the form of an Excel file provides an excellent means of organizing numerical data, creating tables, graphs, doing statistical tests. For an excellent tutorial in on the basics information in use of Excel try https://www.youtube.com/watch?v=YiNHBeu_WJI .

Organizing and Summarizing Data Data Sheets contain may contain: Columns of data showing what was done on each of a sequence of trials – e.g. the response made, the reaction time. Summary of data from groups of participants Counts of behaviors that occur as a function of time. Results section of your paper contains the interpreted data Descriptive statistics frequency distribution tables graphs summary statistics Inferential statistics

Results may be described graphically

Photographs may be used to help your reader understand what was done.

Results may be summarized in tables or graphs Results may be summarized in tables or graphs. These are referred to in the text by number. Select the way of describing your data that is clearest. Table 1. Number and affiliation of males and females participants. Males Females Total Democrats 4 5 9 Republicans 6 1 7 Other 8 Totals 17 24

Scales of Measurement Nominal Scale: Categorizes objects or individuals, no order Ordinal Scale: Categories are ordered (increasing or decreasing) but intervals between categories need not be equal. Interval Scale: same as ordinal scale but intervals between categories are equal Ratio Scale: same as an interval scale but there is a true zero.

Descriptive Statistics - Measures of Central Tendency and Variability Arithmetic Mean - average - only measure of CT that can be manipulated algebraically. Geometric Mean – transform scores in logarithms, sum and get the mean. Median - score that divides the distribution in half Mode - score that occurred most frequently Variability Range - difference between the largest and the smallest scores +1 Semi-Interquartile Range – Half the range of the middle 50% of the scores. Variance – the average squared deviation of scores from the mean(s2) Standard Deviation - square root of the variance (s) Correlation Coefficient – description of the degree of relationship among variables.

Inferential Statistics – What can be inferred from the data. Population - all people, animals, or objects to which the intention is to apply the results. The population from which the sample is drawn may be limited or broad. The broader the population from which the sample is drawn the greater the external validity of the research findings. Example: The population of current psychology students is narrower than the population of all students currently attending Hunter College. Sample - a smaller group than the population intended to represent the population. Most statistical tests assume random sampling from the larger population. If the sample was not randomly selected refer to it as haphazard selected. A convenience sample is a non-random sample, an example is a sample intended to represent Hunter College students by selected by choosing every third Hunter student who enters the college through the Kaye playhouse.

Statistical Inference: Test to determine whether a difference is probably or probably not due to chance. Null Hypothesis: Mean 1 = Mean 2 Decisions Decide whether a difference in only one direction is to be considered : a one-tailed test . For a difference in either direction: a two tailed test is called for. Decide on the probability to be exceeded to reject the null hypothesis. If the null hypothesis is rejected, accept the alternative hypothesis as tenable (probably correct). Chose the appropriate statistical test Based on a “pilot study” decide how many subjects must be sampled to detect a difference if one does exist – the power needed. The smaller the expected effect and variability the greater the number of subjects needed for a meaningful test of significance,. Evaluate the results for statistical significance – accept or reject the null hypothesis.

Signal Detection Theory is an application of Statistical Decision Theory to a situation requiring a generally decision between two alternatives “Noise Alone” and “Signal plus Noise.” In signal detection theory the addition the experimental manipulation (the “signal”) the level of activation, in the diagram below moves the signal distribution to the right. The degree of separation of the means of the two distributions (noise and noise plus signal) in z-scores is referred to as d-prime, d’. A point on the x-axis, the criterion, C, separates two decisions, “no” from “yes” decisions. Here it is shown where the probability of noise alone and signal plus noise are equally likely. Decide “Yes” There is an effect, a signal is present.    Noise Only  Signal Present   YES Decide signal is present   Type I error False Alarm p=α   NO Decide only noise is present Correct Correct Rejection  Type II error Miss p=β Correct Hit Decide “No” There is only noise In null hypothesis testing the criterion of rejecting the null hypothesis is generally set at p< .05. Lowering the criterion to say p <.01 decreases false alarms but simultaneously increases misses. What can be done to increase the probability of detecting a real effect of the experimental manipulaition?

Any decrease in the probability of a Type I Error (false alarm) is accompanied by an increase in a Type II (miss) error To decrease the probability of a miss while keeping the probability of a false alarm at some reasonably low value we must increase the difference between the distribution of events, reduce variability and/or increase the number of observations. In statistics this variability and number of observations (e.g. subjects) determines the effect size.

3. Increase the number of observations What can be done to decrease the probability of a miss without without changing the probability of a false alarm? 1. Increase the difference between conditions – effect size (d’ in this illustration. If you would like to see if a change in environment decreases anxiety then make the change in the environment as large as possible. 2. Decrease variability This can be done by holding variables constant, for example, using a single age group. If you know that age has an effect on the dependent variable use subjects of similar. 3. Increase the number of observations If you have a sufficient number of observations even a very small difference can be detected. The number needed is determined by a power analysis. One should decide how many observations (e.g. number of subjects) are to be tested before the experiment is begun. It is not legitimate to keep adding subjects until the difference is significant. Note that the t-score is the ratio of the difference between the means of two groups divided by the estimated standard error of the difference between the means. For a given difference between the means the smaller the denominator the larger the t-score.

How Effect Size is Found The effect size can usually be estimated from the data typically reported in journal articles: Reported difference Reported variability The degrees of freedom tells the reader how many subjects (observations) were used in the reported statistics. Eta squared ( η2 ) estimates the strength in the population from the strength of the effect in the sample. For the t-test it is t2 divided by t2 + df η2 = .10 = a week effect η2 = .10 to .30 = a moderate effect η2 = or greater than .30 = a relatively strong effect The social science statistics website not only guides you towards selection of the appropriate statistical test it also appears to calculate effect size.

Two Types of Inferential Statistics Parametric statistics - Assumes the data in the population are normally distributed. Parametric statistics require that the data are interval or ratio. Therefore they cannot be used for ordinal (ordered) or nominal (named) data. Parametric statistics are inappropriate for treatment of small numbers of observations even if the data are in interval or ratio form. Non-parametric statistics do not make this assumption. They are distribution free, hence, can be used in the statistical analysis of ordinal and nominal data.

Scales of Measurement Nominal Scale: Categorizes objects or individuals, no order Ordinal Scale: Categories are ordered (increasing or decreasing) but intervals between categories need not be equal. Interval Scale: same as ordinal scale but intervals between categories are equal Ratio Scale: same as an interval scale but there is a true zero.

This flow diagram can be used to chose the correct statistical test This flow diagram can be used to chose the correct statistical test. (1) First decide on the scale of measurement – nominal, ordinal, or interval (2) Then decide whether the hypothesis is one of difference or one of association (3) If the hypothesis is of difference are the samples independent or correlated? (4) Based on sets of measurement select you can now select the appropriate test. A useful website for carrying out some of these tests is: http://www.socscistatistics.com/

How Useful are Your Findings? Internal Validity The extent to which the results of an experiment unambiguously identify the cause and effect relationship. Consider how well variables that may have an effect on the dependent variable were controlled. Look for possible confounds (changes in the dependent variable that may be due to another variable that changed along with it). External Validity The extent to which the findings can be applied to a broad population of interest. All research is limited in its applicability to situations beyond the experiment. The external validity of a finding increases as many experiments under different conditions are done.