A complete presentation. Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE.

A complete presentation

Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE

The goals of science  Science offers a method of distinguishing what is true and real from what is not.  All scientific investigation involves a detailed examination of the subject matter that is being studied and this is done in a systematic and objective way.  Psychological science has four key objectives:  Description  Understanding  Prediction  Control The application of scientific method

Description and understanding  The first step towards understanding and explaining behaviour is to describe it accurately. Detailed and unbiased observation is a key link between the real world and more abstract scientific ideas; it enables psychologists to be clear about the nature of their subject matter and to describe specific phenomena, which can later lead to possible explanations.  The next step is to develop our understanding to explain how and why it occurs. Psychologists usually start by deriving a hypothesis from a theory and then gather evidence to see whether the hypothesis, and thus the underpinning theory, is supported.  As a first step in testing explanations of phenomena, researchers look for patterns and trends in the data they collect. If the analysis of the data suggests a degree of certainty, then they can be more confident about their explanation. If this is not the case, then alternative explanations may need to be provided. The application of scientific method

Prediction and control  After describing and attempting to explain a particular aspect of behaviour, psychologists will usually try to predict when that behaviour is likely to occur.  For example, we know that lacking a sense of control over aspects of our life is associated with stress symptoms such as migraine and gastric ulcers. We might therefore predict that people whose work is monotonous, repetitive and regulated by others would be more likely to have high rates of stress and absenteeism than people whose job allows them more freedom to organise their work. Specific predictions, such as this, can be tested.  Finally, scientists aim to control or modify a specific phenomenon by manipulating the factors that cause it. In certain circumstances, it may be desirable to modify behaviour.  Prediction and control are particularly important for applied psychologists who use psychological knowledge and theories to bring about improvements in many aspects of people’s lives. The application of scientific method

Traditional view of science  The traditional view of science focused on its objectivity achieved through such methods as careful observation and tightly controlled experiments. Personal values and biases are assumed to have no place in scientific thinking.  The term ‘empirical’ is used to describe evidence which can be experienced through the human senses and thus measured, shared and made publicly observable.  Scientists are expected to provide objective evidence to support any scientific claims they make.  The early behaviourists, such as Watson (1913) argued that the subject matter of psychology should be restricted to observable behaviour. This meant that important psychological concepts, such as the mind or self, could not be studied and were not considered to have a place in the scientific study of behaviour. The application of scientific method

Modern views of science  Some consensus is emerging about the main characteristics that distinguish science from other types of activity: 1. Objectivity: this is still regarded as an important feature of science. Scientific data should be gathered in a way that strives to be as objective as possible. 2. Replicability: research findings need to be replicable to avoid basing policy, practice and actions on findings that are either unreliable or based on a ‘fluke’ occurrence. 3. The importance of scientific paradigms: a paradigm is a world- view of general theoretical orientation that is accepted by the majority of scientists working in a given discipline. It determines how researchers approach their work and also what is deemed to be acceptable evidence by the research community. The application of scientific method

Objectivity  Popper (1972) challenged the assumption of total objectivity in any science when he argued that all people, including scientists, have beliefs, preferences, expectations and interests and that these influence observations they make and could introduce bias to their scientific investigations. According to Popper, it is simple not possible to observe something without having some idea of what you are looking for. What we observe may partly depend on what we expect to see and in a research context is driven by relevant hypothesis or theories.  Things which may inhibit objectivity are:  The control of variables – the researcher decides what to control and what not to control.  Interpretation of data – this can be influenced by the researchers experience, expectations and motivation.  Expectations of participants.  Some psychologists argue that data can never be wholly objective, as our interpretation of psychological data is determined by cultural, historical and social influences.  Some would argue that it is impossible to achieve total objectivity in psychology, as it focuses on the study of human behaviour, mental processes and experience, involving people studying other people. The application of scientific method

Objectivity (cont.)  Researchers using qualitative methods do not necessarily view objectivity as an indication of the worth of their research in the same way as quantitative researchers do. Qualitative researchers are far more concerned with understanding meanings; they often use participants’ subjective feelings and reflections as a key source of data.  Rather than focusing on objectivity, qualitative researchers aim to demonstrate the confirmability of their findings by drawing on other sources of data and additional perspectives in an attempt to verify any claims they make. The application of scientific method

Replicability  Replicability serves two important purposes: 1. It guards against scientific fraud. 2. It enables scientists to check whether particular results were a one-off, fluke occurrence because of the particular way the study was carried out, or the people studied, or the place where the study was carried out.  If findings cannot be replicated then we can have no confidence in them and they should not be applied to inform policy.  Replicability is usually higher when the research situation is tightly controlled in a laboratory, but tends to be lower for experiments undertaken outside a laboratory setting.  To enable others to replicate a particular study, psychologists are expected to publish full and precise details of their research. This should include:  Exactly what they did  How the study was carried out  All details about the sample, including how it was collected  Where the study was carried out  How the data were collected and analysed  If a study is replicated and the findings differ from the original, the research community needs to decide the reasons for this. The application of scientific method

The role of paradigms  A scientific paradigm determines how researchers approach their work and also what is deemed to be acceptable evidence by the research community. It provides a general theoretical orientation that is accepted by the majority of scientists working in a given discipline and is used to assess the appropriateness (or otherwise) of specific studies. The application of scientific method

Theory construction; hypothesis testing; empirical methods; laws and principles THE SCIENTIFIC PROCESS The application of scientific method

 The scientific process involves building theories, designing studies to test specific aspects of a theory by testing hypotheses, collecting and analysing data to provide evidence that can be used to support, adjust, or reject the theory. The application of scientific method

Theory construction  Everyday theories are sometimes referred to as ‘implicit theories’ to distinguish them from scientific theory, but they are still theories. Like implicit theories, scientific theories are formulated in an attempt to explain behaviour that has been repeatedly observed. Once a scientific theory has been constructed it must be subjected to rigorous testing to see whether the gathered evidence supports (confirms) or challenges (questions) the theoretical explanation about why something happens. Theories need to be tested in order to ass to our body of scientific knowledge.  To formulate a testable theory, a psychologist would draw on and try to extend existing work by consulting the research literature to find out what research has been carried out by other experts in the field and which of the possible causes would be most fruitful to investigate further. The application of scientific method

Theory construction (cont.)  Scientific progress is made when clear and explicitly formulated theories are developed and systematically tested by research and are either validated, modified or rejected on the basis of research evidence.  If research evidence that has been replicated does not support an existing theory that is was designed to test, then that theory must either be modified to accommodate all the available evidence or rejected and, possibly, replaced with a new theory. Scientific knowledge advances when new evidence becomes available that provides a better explanation of scientific observations. Such evidence must have been demonstrated to be reliable, valid and reproducible. The scientific community constantly scrutinizes the reliability and validity of any new evidence and the robustness of any conclusions drawn on the basis of new evidence.  In summary, a scientific theory is provisional and can be supported, modified or abandoned in the light of new research evidence that is judged to be reliable and valid. The application of scientific method

Hypothesis testing  A hypothesis is a precise, testable statement of the expected outcome of a research study.  Psychological theories and models are tested by generating specific, testable hypotheses from the theory and investigating whether evidence supports them or requires aspects of the theory to be modified to accommodate the findings. Sometimes the accumulating evidence raises such difficulties for a theory that it has to be rejected.  Testing a single hypothesis cannot usually test and entire theory. Instead, specific hypotheses are generated and tested in order to test elements of a theory. A theory, or more likely an aspect of a theory, is challenged if research evidence does not support the predictions articulated within the hypothesis when they are tested.  When experimental data confirm specific predictions derived from a theory or model, researchers become more confident that the theory or model is valid. Confidence in research findings is enhanced by replication. Confidence is also determined to some extent by the probability level of the results. The application of scientific method

Hypothesis testing (cont.) The scientific process indicates how scientists investigate aspects of the physical and social world to generate new knowledge. Stage 1: Induction Step 1: Carry out detailed observations of the topic of interest Step 2: Identify any patterns or trends from the observation data Step 3: Suggest a possible explanation for any patterns that have been noted by generating explanatory theory that will account for the observations and is amenable to testing Stage 2: Deduction Step 4: One or more hypotheses are deduced to test aspects of the theory Step 5: Carry out research to test the hypothesis; check whether the evidence that is systematically gathered and analysed supports the predictions derived from the theory Step 6: Assuming that the research evidence is valid, reliable and has been replicated, the theory is either supported by the evidence, adjusted to accommodate the evidence or rejected because it fails to account for the findings Step 7: Undertake further research to attempt to test further aspects of the theory. If findings are replicated this provides additional support for the theory. If not, the accumulating evidence may suggest that the theory needs to be adjusted or even abandoned The application of scientific method

Hypothesis testing (cont.)  The scientific process involved the two complementary processes of induction and deduction. The inductive method is a form of reasoning that rests on the following assumption: If a researcher makes a sufficient number of observations of events at various times and places, and if the findings are fairly uniform, then it is legitimate to generalise from the observations to a general law or scientific principle.  The inductive method progresses from specific observations (Step 1) to generalised statements/predictions (Step 2) and the generation of theory (Step 3).  Theories generated through induction progress from particular instances (specific observations) to more generalised principles based on systematic investigation using deduction (steps 4-7). These latter four steps are sometimes referred to as the hypthetico-deuctive method. This term captures the process of systematically generating and testing hypotheses, using the process of deduction, which results in research findings that either support or challenge the underpinning theory. The application of scientific method

Empirical methods  A fundamental characteristic of science is its reliance on the empirical methods of observation and measurement.  Empiricism is the name given to the belief that the only source of true knowledge is through our senses, and that careful observation and measurement are needed to generate this form of knowledge. Bacon and other empiricists argued that reality consists of what is available to the senses and, therefore, only phenomena that are publicly observable and can be agreed upon by others can be validated as knowledge.  All scientific knowledge must be based on evidence received via our senses through direct observation, direct experience or measurement, rather than on intuition, personal opinions or beliefs. Thoughts, feelings and subjective experiences can only be studied, therefore, if they can be made observable. However, observation is only one aspect of empiricist thinking; the other part involves the use of the inductive method.  Empiricists argued that all scientific ideas must be subjected to empirical investigation through careful observation or investigation of perceivable events or phenomena, and that this test should be the final test of their truth. Science has emerged ad a trusted approach to knowledge generation because of its reliance on direct, sensory experience.  Scientific claims, based on accumulated empirical evidence, can be judged as true, using criteria that are public and available for all the see and judge, because of the emphasis on the objectivity of empirical evidence. The application of scientific method

Popper’s principle of falsifiability  Popper (1969) argued that one of the hallmarks of science, which distinguishes it from non-science, is the principle of falsifiability. Since there is no logical way in which theories can be proved to be true using induction, scientists should aim to demonstrate they are wrong by ruling out alternative explanations of a specific phenomenon. They should generate theories and hypotheses that can potentially be refuted (disproved) by research.  For example: ‘no amount of observation of swans can ever prove that they are always white, even though out everyday experience might suggest that this is the case. Just a single observation of a black swan, however, would lead to certainty that the theory (all swans are white) is false.  According to Popper, advances in scientific understanding are based on scientific theories that are formulated in such a way that precise predictions can be tested and can, potentially, be disproved by evidence. The application of scientific method

The influence of paradigms  Kuhn (1970) defined a paradigm as ‘the shared set of assumptions about the subject matter of a discipline and the methods appropriate to its study’, and suggested that there are three stages to the development of any scientific discipline: 1. Pre science – this refers to the period when there is a range of views about the most appropriate theoretical approach to adopt, so there is no generally accepted paradigm. 2. Normal science – This is said to occur when there is a generally accepted paradigm, which determines the research that is carried out within the discipline. 3. Revolutionary science – This occurs when there is a paradigm shift and a new paradigm replaced the previous one.  According to Kuhn, major scientific discoveries tend to occur through revolution caused by a paradigm shift, rather than through steady logical development towards an ultimate goal. A ‘crisis’ occurs when more and more things cannot be explained and there is a tension between those scientists who wish to maintain the status quo by retaining the current paradigm and those proposing a new one. The application of scientific method

Validating new knowledge; peer review VALIDATION The application of scientific method

Validating new knowledge  One of the key responsibilities or members of the scientific community is to validate new knowledge. This process of validation starts with the critical scrutiny of the research proposals and ends with the peer review of all new contributions to the body of psychological knowledge, prior to publication and wider dissemination. The application of scientific method

Scrutiny of research proposals  If a study is externally funded, the funding body will usually send the research proposal to a number of experts for their views on all aspects of the planned study in order to maximise the likelihood of the study achieving its aims. The research councils and other funding agencies only support research that is robust, well designed and likely to contribute to the body of knowledge.  A research ethics committee will also scrutinise any research proposal that involves human participants to ensure high ethical standards are met at all stages of the research process. The views and judgement of the research ethics committee will determine what investigations are permitted to go ahead and the research methods that are used. The application of scientific method

Publicly available evidence  Once a study has been completed and the data have been analysed and interpreted, the researcher is expected to share their ideas and findings with their colleagues in the wider research community.  However, before any formal report of a research study is permitted to enter the public domain, members of the scientific community are expected to assess the quality of the research to ensure that it is worthy of wider dissemination.  There are a number of different routes for dissemination such as conferences, scientific meetings and publishing in reputable academic journals.  Researchers are expected to uphold the important principle that their reputation and possible individual advancement is less important than the greater good of the scientific discipline. The application of scientific method

Publicly available evidence (cont.)  Accurate reporting of findings is therefore considered to be essential and more important than recognition of success for the individual or team concerned.  Although researchers may be disappointed if there are no statistically significant differences in outcome to report, such findings can be informative in their own right and may be worthy of publication, assuming that the study was well designed and executed properly.  Inconsistent data can therefore provide a stimulus for further scientific investigation. This might involve refinements of the research techniques used, or the construction and testing of new theories or new hypotheses in an attempts to eliminate possible alternative explanations. The application of scientific method

The external review process  Research reports are required to include sufficient detail to enable other researchers to repeat the study, and also so that others can assess the contribution of a particular study to the developing body of scientific knowledge.  To safeguard the quality of published research, all reputable academic journals employ a robust review process of all research papers before they are accepted for publication. The application of scientific method

The external review process (cont.)  The editor of the journal will be an internationally recognised expert in the field and will usually be supported by one or more associate editors and an editorial board f UK and international experts. The names of all these individuals usually appear at the front of every issuer of the journal because its credibility and reputation largely depend on the expertise, standing and networks of these key individuals.  The editorial board of a journal is supported by a ‘bank’ of external reviewers with expertise in a range of areas that reflect the aims and scope of the journal. Theses reviewers are expected to read the draft carefully and provide detailed commentary on all key aspects of the study. This is usually provided in the form of structured feedback to the editor. This will include commentary on:  The appropriateness of the overall research design, including the methods used to collect and analyse the data.  Any ethical issues.  The sampling technique used.  Any potential source of bias.  The operationalisation and control of key variables.  The reliability, validity and interpretation of the findings.  The appropriateness of any conclusions drawn.  An external reviewer is expected to assess all these issues carefully and arrive at an independent judgement about whether or not the research is worthy of publication in their particular journal. The application of scientific method

Double-blind peer review  Double-blind review indicates that at least two people were involved in the ‘blind’ independent review process, which is designed to minimise bias or the publication of research evidence that is flawed. The critical evaluation of research should be based on its intrinsic scientific merit rather than on the views and prejudices of any individual reviewer. In this context, peers are fellow research psychologists with expertise in the particular area of study or a specific methodological expertise.  Once the editor has received the reviews of a particular paper, they will respond to the author of the report. Just as the reviewers are not told whose work they are reviewing, so the authors of the report will not know who reviewed their paper. This is designed to maximise the objectivity of the review process. The application of scientific method

Research aims; alternative and null hypotheses AIMS AND HYPOTHESES Designing psychological investigations

Aims and research questions  The purpose of a research study can be expressed as an aim, research question and/or a hypothesis, which shapes the design of the investigation. The starting point for any psychological research study is for the researcher to think carefully about what the investigation is trying to discover, and then generate an appropriate aim that makes the focus of the study explicit. The aim of a study may be further refined into a research question and, in the case of most quantitative research, into a hypothesis.  Research questions differ in their breadth. Some are very precisely worded and indicate how the research study will focus on a specific issue. Others are deliberately worded in such a way that they allow the researcher to explore issues more widely. Designing psychological investigations

Formulating hypotheses  A hypothesis can be defined quite simply as a statement that is testable. This statement is made at the outset of an investigation and sums up what the researcher expects to find. This statement is based on the researcher’s knowledge of theory and previous research in a particular field. It is essential to phrase a hypothesis carefully so that it is precise, unambiguous and testable through the statistical analysis of quantitative data collected during the research study.  The process of formulating hypotheses highlights a fundamental issue relating to research. If the wording of the research hypothesis is too general, it will be difficult to test. On the other hand, when a hypothesis is clearly defined and testable, it may lack more general application, so a careful balance needs to be achieved.  Two different forms of hypothesis are important when analysing and interpreting the results of research – the alternative hypothesis and the null hypothesis. Designing psychological investigations

Hypothesis types Alternative hypothesisNull hypothesis  The alternative hypothesis states that the expected effect of the manipulated independent variable on the outcome is statistically significant.  The alternative hypothesis can be either directional or non-directional:  A directional hypothesis states the direction in which results are expected to occur.  A non-directional hypothesis does not state the expected direction of outcome.  Directional and non-directional hypotheses are sometimes referred to as one-tailed and two-tailed hypotheses.  The null hypothesis states that there is no effect in a study.  Statistical techniques enable the researcher to decide whether to retain or reject the null hypothesis. If the null hypothesis is rejected, then the alternative hypothesis is the most plausible explanation of the findings. Designing psychological investigations

Design issues and decisions; pilot studies DESIGNING EXPERIMENTS Designing psychological investigations

Design decisions  When planning a research study, a number of important design issues need to be considered, including:  Choosing an appropriate research method – it is important to decide which is the most appropriate research method to address the research aim. If the research design is purely qualitative, textual data might be gathered, or a study could generate quantitative data suitable for statistical analysis, or it could involve a mixture of both.  Deciding how many participants to study – Researchers conducting any kind of research have to decide how many participants to recruit for their study. This decision is based on practical and financial considerations, and, most importantly, on the number of participants needed to provide findings that can be trusted.  Using an appropriate sampling method – the researcher will have in mind a particular target population of individuals who would be suitable research participants. A sample of participants is usually selected and it is important to do this in such a way that the sample adequately represents the wider population from which it is drawn. Findings from a representative sample can be generalised to the target population. Designing psychological investigations

Design decisions (cont.)  Deciding how to brief participants – it is important to decide whether participants should be made aware, or remain unaware, of the specific nature of the investigation or even that they are taking part in research. This raises the important ethical issue of informed consent.  Deciding how to record the data and the techniques to be used – a written record may be made (often by the participants in the case of questionnaires and some experiments), or behaviour may be recorded for subsequent analysis. A combination of these methods may be used. The researcher also needs to decide which behaviour to record and which to ignore. If a written record is made, the researcher may needs to devise an appropriate coding system for recording behaviour. The technique used may be highly structured, or it may be unstructured. The main aim is to collect data that can be appropriately analysed. Designing psychological investigations

Pilot studies  An important step in designing a good research study is t undertake a pilot study. This is a small-scale trial run of a specific research investigation in order t test out the planned procedures and identify any flaws and areas for improvement, before time and money are invested in carrying out the main study. It is carried out on a small number of participants to find out whether there are any problems with:  The design  The clarity of any standardised instructions for participants and the procedures  The measuring instruments employed, including the use of behavioural categories in observational research  A pilot study also enables the researcher to practise carrying out the research task and provides information on how long it takes, which can be useful when recruiting and briefing participants and when creating a schedule for the actual study.  In the light of direct experience and feedback from the pilot study, the researcher will make changes to address any issues raised by the pilot study before the main study is conducted. Designing psychological investigations

Relationship between researcher and participants  A research investigation is liable to be influenced by those who are taking part. Research participants may be affected by demand characteristics, while investigators themselves may have unintended effects on the outcome of research. Designing psychological investigations

Demand characteristics  Demand characteristics are those elements in a research situation that lead participants to behave in accordance with what they perceive the research situation demands of them, which may be different from how they would typically behave outside of the research situation. Well-designed research aims to minimise the effects of demand characteristics as much as possible.  Demand characteristics might lead to any of the following forms of participant reactivity:  Faithfulness and faithlessness – participants try to guess the purpose of the research and act in a way they feel is helpful, or deliberately unhelpful, to the researcher.  Evaluation apprehension – participants may change their behaviour because the research situation makes them feel they are being evaluated in some way. This, in turn, could interfere with their performance or, alternatively, could trigger participants to try even harder; both would have a distorting effect on the data.  Social desirability bias – participants might be concerned about how others see them and so, rather than being completely honest, change their behaviour in order to create a favourable impression and, by so doing, distort the research findings. Designing psychological investigations

Investigator effects  An undesired effect of a researcher’s expectations or behaviour on participants or on the interpretation of data is known as an investigator effect. Participants’ behaviour could be affected by something as basic as their reactions to researchers who look a particular way or who have good social skills.  Many different features of the investigator could potentially influence the participants, including their age, gender, ethnicity, appearance, facial expression and communication style.  Investigator expectation effects can also occur if a researcher is committed to, or even unconsciously biased towards, interpreting the findings in a certain way. This can be a particular risk if events can be interpreted in more than one way. Designing psychological investigations

Operationalising variables; choosing designs; advantages and disadvantages of design choice EXPERIMENTAL DESIGN Designing psychological investigations

Experimental design  Experiments involved the manipulation of variables in order to investigate cause-and-effect relationships. Selecting an appropriate experimental design is essential for the success of any experimental research. It involves balancing the advantages and weaknesses of different designs. The aim of successful experimental design is to:  Provide an overall plan for all stages of the experiment.  Ensure high levels of control over the independent variable (IV) and the dependent variable (DV).  Eliminate all potential sources of ambiguity or bias.  Ensure appropriate and precise measurement of the key variables.  Enable the data collected to be analysed appropriately. Designing psychological investigations

Defining and operationalising variables  A variable is something that may vary or change in some way and which can either be categorised or measured.  The control, manipulation and measurement of variables are central to psychological research. Psychologists need to be able to define variables clearly if their research is to be scientifically credible and worthwhile.  Operational definitions are descriptions of variables phrased in terms that are sufficiently precise to enable them to be identified and measured.  In experimental research, the key variables are the IV and the DV. The IV is the variable that is systematically manipulated by the researcher in order to bring about changes in the DV. The DV is an outcome variable that is measureable. Operationalising variables usually results in narrowing the research focus.  The more precise the operational definition, the narrower the research focus and the more limited the extent to which results can be generalised. There is a balance to be achieved between precision and what is meaningful in the real world. Designing psychological investigations

Choosing an experimental design  When deciding on an appropriate design, the researcher must consider carefully:  The precise nature of the experimental task  How to control the relevant variables  The availability of participants  Three types of experimental design are: 1. An independent groups design – different participants are used in each condition of the experiment 2. Repeated measures design – the same participants are used in each condition of the experiment 3. Matched pairs design – each participant in one group or condition is carefully matched on all the variables considered to be relevant to the investigation with a participant in another group or condition. Designing psychological investigations

Choosing an experimental design (cont.)  Experiments using these designs typically involve a control condition and an experimental condition; the group of participants who receive the experimental treatment (IV) is referred to as the experimental group and the group that does not receive the experimental treatment I the control group. Results from the control group provide the baseline data against which to compare the effect of the IV on the experimental group.  Alternatively, an investigation might compare two experimental conditions. It is sometimes impossible or meaningless eradicate the IV from one condition.  The simplest form of experiment involves just two conditions, but it is possible to compare a control condition with a number of experimental conditions in more complex studies, or to compare two or more experimental conditions in situations where a control group cannot be used. Designing psychological investigations

Independent groups design  An independent groups design involves using different participants in each condition of the experiment.  Ideally, participants should be allocated randomly to the conditions. Random allocation aims to ensure that characteristics of the participants do not differ systematically between the conditions at the start of the study. If they did, any individual differences relevant to the investigation might become a confounding variable.  Random allocation cannot guarantee equivalent groups of participants. The researcher may still fail to eliminate individual differences. Fortunately, the likelihood of this occurring is minimal.  By randomly allocating participants to groups, a researcher also aims to avoid any conscious or subconscious bias in the allocation of participants to groups. When an independent groups design is used and there is a sufficient number of participants in each group, it is highly unlikely that individual differences between the groups will be a confounding factor. Designing psychological investigations

Independent groups AdvantagesDisadvantages  There is no issue of order effects which occur when participants’ performance is positively or negatively affected by taking part in two or more experimental conditions.  An independent groups design has a wide range of potential uses and can be used when problems with order effects would make a repeated measures design impractical.  There is the potential for error resulting from individual differences between the groups of participants taking part in the different conditions. Also, if participants are in short supply, then an independent groups design may represent an uneconomic use of those available to participate, since twice as many participants are needed to collect the same amount of data as would be required in a repeated measures design with two conditions. Designing psychological investigations

Repeated measures and matched pairs Repeated measures designMatched pairs design  This design involves exposing every participant to each of the experimental conditions, which means the participants are used as their own controls.  One of the conditions in a repeated measures design may be a control condition, which serves the same purpose as the control condition in an independent groups design.  A matched pairs design aims to achieve the key advantages of an independent groups design, and of a repeated measures design.  A matched pairs design involves matching each participant in one of the experimental conditions as closely as possible with another participant in the second condition on all the variables considered to be relevant to performance in the study. Designing psychological investigations

Repeated measures design  Advantages:  Individual differences between participants are removed as a potential confounding variable.  Fewer participants are required, since data for all conditions are collected from the same group of participants.  Disadvantages:  The range of potential uses is smaller than for the independent groups design.  Order effects may result when participants take part in more than one experimental condition. Designing psychological investigations

Matched pairs design Advantages:Disadvantages:  A matched pairs design combines the advantages of both independent groups and repeated measures designs.  The assumption is made that members of each pairing are sufficiently similar on the relevant variables that they can, for research purposes, be treated as if they are the same person. As the same time, participants only perform in one condition of the experiment, thereby eliminating the problem of order effects.  Achieving matched pairs can be difficult and time consuming. Complete matching of participants on all variables that may affect performance can rarely be achieved. Matched pairs designs are relatively uncommon, with their use restricted to specific situations where a matching process is highly desirable in order that experimental success can be achieved. Designing psychological investigations

Control of EVs  In experimental research there will always be a certain amount of interference from unwanted variables that cannot be fully controlled or about which the researcher may be unaware. Those variables are a source of unwanted ‘noise’ and are known as extraneous variables (EVs). These need to be controlled because they can obscure the effect that is being investigated so, wherever possible, the researcher will aim to minimise their influence through food experimental design. Extraneous variables may result from random error or constant error. Designing psychological investigations

Random and constant error Random errorConstant error  The effects of random error cannot be predicted. Possible sources include:  A participant’s state of mine.  A participant’s level of motivation.  Incidental noise.  Room temperature.  Previous experiences on the day of the study.  By allocating participants randomly to experimental conditions, psychologists usually assume that random errors balance our across the experimental conditions. Such errors might, however, result in some loss of sensitivity of the results.  Constant errors affect the dependent variable in a constant way and, therefore, create a more serious problem for the researcher than random error, as they may not affect all conditions of an experiment equally. Constant errors may include:  A failure to counterbalance or randomise the presentation order of experimental conditions.  Participant differences affecting one condition more than another.  Errors of measurement that affect one condition more than another.  An uncontrolled constant error in an experiment, which brings about a systematic change in a dependent variable, is known as a confounding variable. Designing psychological investigations

Naturalistic observation; questionnaires; interviews OTHER RESEARCH METHODS Designing psychological investigations

Naturalistic observation design  A key design issue with naturalistic observation studies is deciding how to sample the behaviour to be studied. The possibilities include:  Time interval sampling – observing and recording what happens in a series of fixed intervals.  Time point sampling – observing and recording the behaviour that occurs at a series of given points in time.  Event sampling – observing and recording a complete event, such as a teacher encouraging a pupil.  A further issue that needs careful consideration related to behavioural categories (ways in which data are organised and recorded). Possible methods include:  Preparing written notes.  Producing a checklist or tally chart.  Using a rating scale. Designing psychological investigations

Questionnaires  Questionnaires can be administered face to face, by post or online via the internet.  Once a questionnaire has been developed it should always be piloted. Piloting allows the researcher to check that all the questions can be answered and that they contribute to the purpose of the research. Any ambiguity or other issue that comes to light during piloting can be rectified before the questionnaire is used to gather data in an actual research study. Designing psychological investigations

Questionnaires (cont.) Closed or open questions Closed questions are frequently used ion questionnaires because they are easy to score and analyse. Open questions, on the other hand, do not constrain respondents, however they are more difficult to analyse. Number of questions and question order Only questions that are absolutely necessary for the purpose of the research should be included. Questions relating to demographic characteristics are usually included at the end. Highly sensitive questions should not be placed at the beginning. Use clear language Plain English should always be used, so that the wording of every question is clear and unambiguous. Jargon and technical language should be avoided where possible, or an explanation of any technical terms should be provided. Avoid leading, biased or value-laden questions Question wording should never lead the respondent towards a particular answer. Questions should not include any value judgements. Designing psychological investigations

Questionnaires (cont.) Ask one question at a time It is sometimes tempting to ask two separate questions rolled into one, but this does not allow the respondent to give two separate answers. Instead, two separate questions should be asked. Avoid using emotive language when asking questions Use of emotive language can bias the response. Ask questions that are clear and unambiguous It is important to avoid asking questions that are vague or ambiguous. All participants need to treat a particular question in the same way if the data collected are to be meaningful and produce useful results. Avoid making inappropriate or insensitive assumptions Avoid asking questions that incorporate an assumption and could therefore cause embarrassment to some respondents. Designing psychological investigations

Interviews The interviewer/researcher must have: Stage 1: Preliminaries to the interview Clearly described the research problem. Stated the aim of the interview. Linked the problem to an appropriate theory. Identified the general categories of data they will need to collect. Stage 2: The questions Generated an appropriate set of questions. Planned the order in which the questions will be presented. Planned the interview to obtain the required balance between structured and unstructured interviewing. Stage 3: The interview procedure Considered the issues of self-presentation. Identified and approached potential respondents. Planned the pre-interview meeting. Planned the post-interview debriefing. Decided how the information is to be recorded in the interview. Considered the ethical issues raised by the proposed research and sought advice if necessary. Designing psychological investigations

Sampling types; sample bias SAMPLING STRATEGIES Designing psychological investigations

Populations and samples  A target population is the total collection of people who share a given set of characteristics.  However, many target populations are too large for a researcher to study everyone in a way that is practical or financially feasible. For these reasons, a subset if the population – a sample – is typically investigated instead.  A representative sample is a selection of individuals from the target population, which shares all the main characteristics of the population despite its smaller size. If, and only if, a sample is truly representative, can it be used as a basis for generalising the results of the study, and any conclusions drawn, to the rest of the target population.  There will nearly always be some degree of sampling error that results in the sample differing in some way from the target population. If the sampling error is large, then generalisations to the target population are unlikely to be accurate. Researchers can minimise sampling error by choosing their sampling technique carefully in order to be able to generalise with confidence. Designing psychological investigations

Random sampling  In a random sample, every person or item in a given target population has an equal chance of being selected for inclusion. This means that it is necessary to have a list that identifies every person or item in the target population.  Random number tables or random number generators can be used to select a sample in an unbiased way.  However, selecting a random sample does not guarantee a sample that is totally representative of the population concerned. Nor does it mean that any two samples drawn from the same target population will share identical characteristics. By its very nature, a random sample can only guarantee that it has been selected in an entirely unbiased manner. Designing psychological investigations

Opportunity and volunteer sampling Opportunity samplingVolunteer sampling  Opportunity sampling involves the researcher selecting anyone who is available to take part in a study from a defined population such as the staff or students in a particular college. Opportunity sampling is a non-random method of sampling, widely used because it is convenient.  Its main weakness is that it is unlikely to generate a sample that is representative of the target population. This means that the findings gathered from an opportunity sample are unlikely to be representative of the target population and may be biased, so should not be generalised to the wider population.  Volunteer sampling is another non-random sampling technique that involves participants selecting themselves to take part in a research study, often by replying to an advertisement. This type of sampling has been widely used in psychological research.  Potential weaknesses of using a volunteer sample or self-selected sample are that the majority of a given target population are unlikely to respond to the request to participate. Those who do respond may have particular characteristics that makes them atypical of the target population.  Compared with random sampling, data gathered from a potentially biased, volunteer sample are less likely to be representative of the target population and so the findings of the study should not be generalised to all its members. Designing psychological investigations

Sample bias  The aim of sampling is to keep sampling error to a minimum in order to represent the target population accurately.  Sample bias can occur is some members of a target population are more likely to be selected than others. Three ways in which bias might be introduced are through choice of sampling technique, choice of target population and sample size. Designing psychological investigations

Choice of sampling technique  Recruitment depends of individuals agreeing to participate, so there is inevitably an element of bias towards self-selection. Potential participants have the right not to participate in research, which means that the characteristics of people who choose not to participate are lost, and the sample becomes biased towards those who are willing to take part.  Even when a sampling technique is carefully applied, a sample may still be biased if certain members of a population, who might be prepared to take part, are not represented. This can happen if there are minority subgroups.  The solution is to specify the characteristics of the subgroups in advance and then select from each group in the same proportions that appear in the target population. This is known as stratification. If the selection is then carried out using random sampling, the technique is known as stratified random sampling. If it is carried out using opportunity sampling, it is known as quota sampling. Designing psychological investigations

Choice of target population  No matter how careful a researcher is to select a representative sample in order to be able to generalise findings to the target population, sample bias can occur if certain populations are persistently targeted at the expense of others. This could lead to a body of psychological knowledge that is based on a limited subset of individuals who are alike in various ways but different from other people.  It is not necessarily wrong to target specific populations, as long as researchers recognise the limits that this places on their ability to generalise the findings. Designing psychological investigations

Small samples  Small samples may be biased if they happen to contain individuals with particular characteristics that are unlike the majority in the target population. A general principle is that the larger the sample, the more likely it is to provide a good approximation of the population from which it was drawn.  Determining sample size, therefore, depends on balancing the need to represent the target population accurately on one hand, and practical considerations, on the other. Statistical techniques can be used to decide on the sample size needed to achieve acceptable levels of sampling error in target populations of different sizes. It should be noted, however, that small samples should not automatically be discounted; they are sometimes used to good effect. Designing psychological investigations

Types of reliability; assessment and improvement of reliability; assessment and improvement of validity RELIABILITY AND VALIDITY Designing psychological investigations

Types of reliability  When carrying out psychological research it is essential that the chosen approach is used consistently. Any variation in how a researcher conducts a study and the tools used to collect data can introduce unwanted variation that can reduce the quality of the research evidence and, possibly, obscure the effects being investigated.  Researcher reliability refers to the extent to which a researcher acts entirely consistently when gathering data. In experimental research this is referred to as experimenter reliability and in non-experimental research, such as observational research or clinical assessment, where more than one researcher may be involved in data collection, it is known as inter-observer or inter-rater reliability.  Internal reliability refers to the consistency of the measures used in an investigation.  External reliability refers to the consistency of a measure from one occasion to another. Designing psychological investigations

Assessing researcher reliability Intra-researcher reliabilityInter-researcher reliability  The consistency of researchers’ behaviour is important in all research situations, irrespective of the research method being used. Intra-researcher reliability is achieved if a researcher performs consistently. Reliability is assessed by measuring the extent to which a researcher produces similar results when observing or scoring the same situations on more than one occasion. Intra-observer (intra-rater) reliability is achieved if there is a statistically significant positive correlation between the scores obtained on the different occasions.  This is important when there is more than one researcher working on a particular project. The researchers need to act in entirely similar ways. In experimental research, this would involve different researchers carrying out exactly the same procedures. In observation research, inter-observer (inter-rater) reliability is a measure of the extent to which different observers agree on what they have observed. Inter- observer reliability is achieved if there is a statistically significant positive correlation between the scores of the different observers. Designing psychological investigations

Improving researcher reliability  It is important to ensure high intra- and inter-researcher reliability. There are two main ways in which these can be improved:  Careful design of a study – taking into account design issues is critical to improving both intra- and inter-researcher reliability.  Careful training of researchers – this should take place prior to data collection, so that variability in their behaviour is reduced and reliability is improved both within and between researchers. Researchers should know exactly how to carry out the research procedures and how to record the data. From the outset, operational definitions of the key terms should be clear and fully understood by all those involved. In an experimental study, the researcher should know how to record participants’ responses in a consistent manner. Designing psychological investigations

Assessing internal validity  The split-half method can be used as a way of assessing the extent to which individual items in a particular psychological test of questionnaire are consistent with other items within the same test. The method involves splitting the psychological test or questionnaire into two pars after the data have been collected from the participants. This might be done by:  Comparing results obtained from odd-and even-numbered questions.  Comparing the results from the first half of the test with those from the second half.  Randomly splitting the test/questionnaire into two parts.  Whichever method is used, the two sets of responses are then correlated. A statistically significant positive correlation for the two sets of responses indicates that the test or questionnaire is reliable. Is the correlation is not statistically significant, the researcher would need to check individual test items by removing each one in turn and retesting to see if the overall reliability of the measuring instrument improves. Designing psychological investigations

Assessing external reliability  The test-retest method is used to assess the consistency or stability of a psychological test or questionnaire over time. This method involves presenting the same participants with the same test or questionnaire on two different occasions, with no feedback given after the first presentation. The time interval between presentations need to be selected carefully. If it is too short, participants may remember their previous answer, and if it is too long the participants may have changed in some way relevant to the test or questionnaire.  Correlational techniques are used to calculate the test stability. If there is a statistically significant positive correlation between the scores obtained in the test-retest phases, the test is deemed to be stable. If not, individual test items can be checked for consistency and/or the testing procedures can be revised and the reliability retested to see if a statistically significant correlation is obtained. Designing psychological investigations

Internal validity  Internal validity refers to the overall quality of a research design and is relevant to any kind of research.  Poor internal validity can result in failure to find an effect in a research study when one actually exists. It is important to pay close attention to all aspects of the research process, from formulating the research question, through planning, designing, conducting the study and analysing the data, so that the research community can have confidence in the findings. Designing psychological investigations

Internal validity (cont.)  Psychologists use various techniques to assess and improve the validity of specific tests and measure. Some of these are:  Face validity – this is the simplest type of validity and refers to whether a test or measuring instrument appears, on the surface, to be doing what it should and is self-evident.  Content validity – this is similar to face validity, focusing on whether the content of a test or measuring instrument covers the whole of the topic area.  Concurrent validity – this involves obtaining two sets of scores at the same time – one from the new procedure with unknown validity, and the other from an alternative procedure or test for which validity had already been established. The scores obtained from both the new and existing test will then be correlated to assess the validity of the new procedure. A statistically significant positive correlation coefficient would suggest that the new procedure is valid.  Predictive validity – this involves a similar strategy to that used to assess concurrent validity, but the two sets of scores are obtained at different points in time. Designing psychological investigations

External validity  The term ‘external validity’ is concerned with the extent to which results can be generalised across people, places and times. The three different types of external validity are:  Context validity – this refers to the extent to which research findings can be generalised to settings other than that of the original research. Psychologists often call this ‘ecological validity’.  Temporal validity – this is concerned with the ‘shelf-life’ of research findings and whether they are able to endure over time.  Population validity – this refers to the extent to which research findings can be generalised to people other than those actually involved in the original research. Designing psychological investigations

ETHICAL CONSIDERATIONS Designing psychological investigations

Code of ethics and conduct  When undertaking research, psychologists are expected to follow guidelines about ethics set out by whichever professional body they belong to. Since there are no absolutes about what is deemed to be ‘right’ or ‘wrong’, different groups determine what is considered to be acceptable or unacceptable for their members. In the United Kingdom, the British Psychological Society (BPS) is responsible for promoting ethical behaviour among psychologists and has developed ethical principles to protect all research participants from harm. The BPR acknowledges that psychologists owe a debt f gratitude to everyone who agrees to take part in research studies, and requires its members to treat participants with high standards of consideration and respect. The guidelines also protect psychologists as they go about their work.  The latest BPS Code of Ethics and Conduct (BPS 2006a) outlines the guiding principles that all psychologists should apple, including those who carry out research, and those in practice. Unfortunately, the mere existence of a code does not guarantee ethical practice; for this to happen, the code has to be implemented conscientiously. Good psychological research is only possible if there is mutual respect and confidence between the investigator and participants. For sound ethical reasons, some areas of human experience and behaviour may no longer be investigated. Designing psychological investigations

Respect  Informed consent – psychologists should ensure that all participants are helped to understand fully all aspects of the research that are likely to influence their willingness to participate, including the nature and objectives of the investigations, so they can give their fully informed consent to take part. If a study is carried out over an extended period of time or if there is any significant change in the focus of the study, it may be necessary to seek additional informed consent.  Confidentiality and anonymity – Participants have the right to expect that all data collected during a research study remain confidential and will be stored securely in accordance with the UK Data Protection Act, 1998. If the findings are published, the data should remain anonymous and should be presented in such a way that specific information cannot be linked to particular individuals. Participants should be warned at the start of a study if confidentiality and anonymity cannot be guaranteed, prior to giving their consent to take part.  Right to withdraw – At the outset it should be made clear to participants their right to withdraw from a research investigation at any time.  Deception – Withholding information or misleading participants about the purpose of a study is unacceptable if the participants subsequently become uneasy when they have been debriefed about its true purpose at the end of the study. Intentionally deceiving participants about the purpose and nature of the investigation should be avoided wherever possible, and is only ever deemed to be acceptable in very exceptional circumstances.  Observation research – Studies based on observation should always respect the privacy and psychological wellbeing of the individuals studied. Unless those being studied give their informed consent, observational research is only acceptable in public places where those being observed would expect to be observed by strangers. Designing psychological investigations

Competence  Psychologists should be committed to the Code of Ethics and Conduct, and to maintaining their levels of competence, while at the same time acknowledging any limits of their knowledge, skills, education and experience. This means that they should recognise and resolve the ethical dilemmas that arise out of a proposed research study and be able to defend all their decisions and actions on ethical grounds. Designing psychological investigations

Responsibility  Protection of research participants:  Risk of harm - investigators have a key responsibility to protect all participants from physical and mental harm during research – the risk of harm should normally be no higher than in their everyday life.  Understanding the implications of an investigation – The researcher may not have sufficient knowledge about the implications of an investigation for all participants, in which case it may be necessary to consult others who are more knowledgeable.  Protection from stress – Where research involved behaviour or experiences that are considered to be personal and private, the participants should be protected from undue stress, and given assurance that they need not answer personal questions.  Inducements – Financial incentives of other inducements should not be used to encourage individuals to take part in research.  Professional advice – A researcher has a responsibility to inform a participant of any psychological or physical problem that emerges during the course of research. If any professional advice is requested in the course of research, a referral should be suggested to someone who is suitably qualified to deal with the matter raised.  Non-human animals – when carrying out research with non-human animals the highest standards of animal welfare should be observed; the animals should not be subjected to any more pain, discomfort, suffering, fear, distress, frustration, boredom than is absolutely necessary.  Debriefing – the researcher should take time to discuss the study with participants. During the debriefing session the researcher should also discuss the participant’s experience of the study in order to identify any unseen discomfort, distress of other negative effect of the research. A participant has the right to withdraw any consent given, and to require that all their personal data be removed from the study and destroyed. Designing psychological investigations

Integrity  Psychologists should be honest, accurate, clear and fair in all their dealings with people, including research participants. They should also seek to promote integrity in all scientific activity. In the context of research, this should include being accurate and honest when recording and analysing data, and when reporting research findings and acknowledging any limitations of the results and the conclusions drawn.  It is also important to emphasise that participating in research is entirely voluntary and there should never be overt or covert coercion to do so. It needs to be clear that participation in research will not affect in any way the provision of resourced to which the individual is otherwise entitled. Designing psychological investigations

 Views of what constitutes ethical conduct and ethical research are not fixed, so codes of ethics are regularly reviewed and updated. Some recent additions include the ethical implications for psychologists working with the media on television (BPS 2007c). Psychologists sometimes need to find creative ways of applying the guidelines in order to carry out particular kinds of research while still ensuring that the rights of the participants are fully protected. Relating to the matter of informed consent, these include presumptive consent and prior general consent. Designing psychological investigations

Research ethics committees  Psychologists are required to apply the BPS guidelines when designing research and this provides the first line of defence in protecting the public from harm. However, researchers are not permitted to proceed to data collection until they have gained formal approval from an ethics committee. In universities and other institutions, this is the responsibility of a research ethics committee whose job is to scrutinise and rigorously evaluate ethical standards in research proposals and protect participants from harm. This provides a second line of defence.  Any university department running courses accredited by the BPS is required to apply these minimum standards, which indicate that psychological research carried out by staff and students must be approved by one or more of these committees:  A departmental ethics committee  An institutional ethics committee  An external ethics committee Designing psychological investigations

Enforcing ethical guidelines  The monitoring of ethical standards is the third and final means of protecting research participants. Should psychologists be found to be contravening the published guidelines, or a formal complaint is made about them, the BPS Investigatory Committee has the power to apply disciplinary procedures of varying severity. This committee can recommend whether a complaint should be dismissed or followed up by a disciplinary body. The psychologist may subsequently be officially reprimanded, have their Chartered Psychologist status suspended or removed or, in extreme cases, be expelled from the society. Designing psychological investigations

Means before ends/participants not subjects Means before endsParticipants not subjects  Changes in ethical guidelines over time illustrate how the can and do impact on what is done, how it is done, by whom and on whom it is done. Over 50 years have elapsed since the first official APA guidelines were published and, during that time, changes in the Zeitgeist and tremendous technological advances have occurred. There has been an important shift – the means is now regarded as being more critical than the end.  There is now far greater awareness of human rights and legal sanctions. Current BPS guidelines state that psychology undergraduates are no longer permitted to carry out any form of research on anyone under the age of 18 or vulnerable groups. In addition, research participants are not viewed more as collaborators in research and are no longer referred to as ‘subjects’. The drive towards equality has also led to the inclusion of guidance on how to avoid sexist or racist language in any aspect of their work in the BPS Code of Ethics and Conduct (BPS 2006a). Designing psychological investigations

Media and communications/bias and benefits Media and communicationsBias and benefits  Developments in information and communications technology has meant that greater openness and accountability to a wider audience is now possible and this has led to concerns about how research findings are communicated. There are not guidelines about how psychologists should conduct themselves with the media and what is expected of them (BPS 2007b), and about how to conduct Internet-based research (BPS 2007a). All the ethical guidelines produced by the BPS are primarily concerned with protecting participants, as well as protecting psychologists.  In addition to protecting psychologists and research participants, research ethics committees can act as gatekeepers, not only limiting the way in which research is carried out, but also what research is undertaken and the kinds of research question that are asked. Although members of ethics committees strive to maintain their neutrality, the dominant ideas and values of the time may still influence their decision-making. This awareness has led to the approval of more research for the benefit of women, people who are elderly, and other previously marginalised groups. In addition to safeguarding participants, ethics committees also have a responsibility to try to address any biases, so that the benefits of research are distributed equitably. Designing psychological investigations

Ethical issues in research proposals  Ethics committees focus, above all else, on the protection of research participants, and this demands detailed and rigorous evaluation of all aspects of a proposed study. A committee will therefore require extensive documentation, including the research proposal, a copy of the consent form, and any other material.  Once a researcher has prepared all the documentation that an ethics committee needs, it is circulated to the committee members so they can familiarise themselves with it. The committee then meets to consider the proposal and decide whether to reject it outright, approve it, or ask for further clarification and/or modifications before the study is permitted to proceed.  Once research has been approved and carried out, the findings will need to be analysed and reported. Formal approval by an ethics committee is just one of the elements of quality control that research must pass before it can be disseminated to the wider research community. Designing psychological investigations

Data Analysis and Reporting

APPROPRIATE SELECTION OF GRAPHICAL REPRESENTATIONS Data Analysis and Reporting

Graphs and frequency diagrams  Graphs tend to be used to show change over time or trials. Time or number of trials is usually plotted on the horizontal axis and the measure you are interested in is presented on the vertical axis.  Frequency diagrams give a visual impression of how often certain measures occur. There are rules to do with how the data are measured that determine which ones to use. Frequency is always recorded on the vertical axis and the frequency of the variable of interest is plotted on the horizontal axis. Data Analysis and Reporting

Bar charts and histograms Bar chartsHistograms  Bar charts are best for showing frequencies of nominal data or ordinal data, or for showing averages of different data sets on the same set of axes. Bar charts are also used to plot data in the form of percentages and means.  Histograms are most suitable for use with interval and ratio data and choosing between them may be determined by the nature of the scored to be presented. Discrete data are in the form of whole units. Such units are more meaningful if they are not sub-divided. Continuous data are measured in units that can theoretically be sub- divided ad infinitum. Data Analysis and Reporting

Scattergrams  Drawing a scattergram is an essential first step when carrying out any correlational analysis. It gives an initial indication of whether there is a relationship between two variables and, if so, wheher it is positive or negative. A scattergram also shows whether correlation analysis is suitable. If there are anomalies, such as unusual scores in the data set that stand apart from other points on the scattergram, the correlation might be unduly affected. As a general rule, if you think that one variable predicts another, the predictor variable is presented on the horizontal axis. Data Analysis and Reporting

PROBABILITY AND SIGNIFICANCE Data Analysis and Reporting

Inferential statistics  Inferential statistics enable psychologists to go further and use the samples of data they have collected to make inferences about the populations from which the samples were drawn. If you find that girls and boys perform differently in an academic test, how do you decide whether this difference is meaningful or significant? The difference is judged to be significant by carrying out an inferential test on the data that have been gathered. In this case, you would use a test that compared sample means and apply rules of probability and statistical significance to assess the result formally. If it is decided that a test result is statistically significant, practical consequences might well follow.  The concept of statistical significance is central to inferential statistics and to any decision about whether to retain or reject the null hypothesis. Hypotheses can be directional and specify the direction of difference or the type of correlation. They can also be non-directional and simply state that there is a difference or a correlation, but not indicate a direction. When testing statistical significance, a one-tailed test is applied to a directional hypothesis and a two- tailed test is applied to a non-directional hypothesis.  It is more accurate to say that assuming the null hypothesis is true, the probability of observing the test result obtained is very small and that other influences, such as experimental manipulation, are likely to be at work. Data Analysis and Reporting

Level of significance  The significance level for the result of any statistical test is expressed as a probability value, which can be anything from 0 to 1. this value indicates the probability that the null hypothesis is true and so it follows that a researcher would want a very small probability value in order to be able to claim that the test result is statistically significant. Probability values may be written as percentages or decimals:  A 5 per cent level of significance would be written as p=.05  A 1 per cent level of significance would be written as p=.01 (p = the probability of the result occurring if the null hypothesis were true)  Five per cent is usually regarded as being the minimum acceptable value for deciding whether a test result is statistically significant. When the 5 per cent significance level is achieved in quantitative research, providing that the study has been carefully designed and executed, a more likely explanation is that the result is due to the effects of the manipulated independent variable. The null hypothesis is therefore rejected. Data Analysis and Reporting

Level of significance (cont.)  The 5 per cent level of significance is by no means the only one that is used by psychologists. Sometimes more stringent significant levels are needed. Examples of more stringent levels include 1 per cent (p=.01), 0.5% (p=.005) and 0.1% (p=.001).  Statistical software will provide the exact p for a test statistic, but when you carry out statistical tests by hand, you may not know the exact figure. When this happens, the usual way of expressing significance is the use the, so would be followed with p>.05. Data Analysis and Reporting

Type 1 and 2 errors  Test statistics give an indication of what is true in the real world, but there is still a possibility that errors will be made when deciding what to do with the null hypothesis.  A Type 1 error occurs when a null hypothesis is rejected when it should not have been. The likelihood of making such an error is equal to the level of significance employed. For example, at p=.05 the risk of making a Type 1 error is 1 in 20. This type of error can occur when an insufficiently stringent significance level is adopted (for example p=.1)  A Type 2 error occurs when a null hypothesis is retained when it should not have been. There is a failure to detect an ‘effect’; this may occur when significance levels are too stringent.  It is often considered preferable to run a higher risk of making a Type 2 error than a Type 1 error, and this is why psychologists might choose a more stringent level of significance. It is better scientific practice to stay on the side of caution, especially when the potential consequences of applying a weak finding would be serious. Data Analysis and Reporting

FACTORS AFFECTING CHOICE OF TEST Data Analysis and Reporting

 There is a wide variety of tests to choose from, but it is important to apply the most appropriate one. If you do not, the test result, and the conclusions drawn from it, will not be trustworthy.  Strictly speaking, in a statistical context, the term ‘population’ refers to a complete data set rather than to a sample of the population. However, in the real world of research, psychologists always work with samples of data drawn from the populations of data for practical reasons. As long as the sample is carefully selected and the research study is well designed, it should be possible to generalise the findings fro the sample to the wider population from which the sample was drawn. In other words, statistical tests are applied to the data collected from the sample in order to infer the characteristics of the wider population. Data Analysis and Reporting

Factors affecting choice of test Type of research designType of data  This is determined by the research question that you want to answer. If you have conducted an experiment, you will need a test to detect differences between the sampled of data from the two or more conditions. The test you use will depend on whether the design of your experiment was independent or related. If you are looking for a correlation you will need a test that detects linear relationships in samples of paired data and if you are looking for a difference in frequency counts, you will need a test that will show if one exists.  The data will be measured on either a nominal, ordinal, interval or ratio scale. These scales differ in the qualities that they have and determine which type of graphical representation is appropriate. Along with the research design, they also determine which statistical test should be applied. Relatively complex statistical procedures can be applied to interval or ratio data. It is important to be able to distinguish interval/ratio data from ordinal and nominal data to which only relatively simple statistical procedures can be applied. Data Analysis and Reporting

Types of data Nominal data Data are allocated into categories such as male/female, smoker/non-smoker etc. Nominal data are sometimes referred to as ‘categorical data’ or ‘frequency data’ because once the categories have been set up, data are in the form of frequencies. The category label are only names, so there is no inherent order in them. This is the simplest of the four levels of measurement. Ordinal data A scale which consists of rankings or ratings is ordinal. This allow you to make statements about the relative magnitude of scores. For example, you can see if one value is higher, lower than, or equal to another. However, the extent of the comparison is limited, as the intervals between the units on the scale are not necessarily equal. Interval/ratio data An interval scale also involves measurements that can be compared in terms of magnitude, however, unlike ordinal data, there are equal intervals between the units on the scale because they are based on some standard unit of measurement. The zero point on interval scales is arbitrary and does not indicate absolutely nothing, for example the zero n a temperature scale is meaningless as temperatures can drop below zero. Ratio data are also measured on a scale that has magnitude and equal intervals but, in addition, has an absolute zero point. For example, in terms of weight, something which weighs 50kg is twice as heavy as something which weighs 25kg. This is because on a scale of weight, zero means absolute zero. The ratio scale is the most complex of the four. Data Analysis and Reporting

Deciding which test to use  The chi-squared test is used to analyse nominal data, and the Mann-Whitney U test, the Wilcoxon matched pairs signed ranks test and the Spearman’s rho test each require data that are at least on an ordinal scale. So how do you analyse data that are on an interval or ratio scale?  If you were to analyse samples of data such as the time taken in seconds for rats to run a maze, you would be working with data on a ratio scale. Rank orders could be allocated (such as rat C came first, rat A came second etc), converting the data to an ordinal scale. Alternatively, the scores could be grouped into categories such as the number of rats running the maze in under 30 seconds and the number of rats running the maze in over 30 but under 40 seconds etc. In this case, the data could be treated as nominal.  It is possible in most cases to make a more complex measurement scale into a simpler one, but not the other way around. Decision 1: What are you looking for? DifferencesCorrelation Decision 2: What level of measurement do the data represent? Spearman’s r s (Spearman’s rho) NominalOrdinal (or interval/ratio) Chi squared test Decision 3: What experimental design was used? Independent groupsRelated Mann Whitney U testWilcoxon matched pairs Data Analysis and Reporting

THE USE OF INFERENTIAL TESTS Data Analysis and Reporting

 In each inferential test, a formula is applied to the data to calculate a test statistic. The next step is to assess this statistic to find out whether it indicates a significant effect.  For each test there is a table of critical values. For certain tests, the calculated statistic must equal or exceed a particular critical value, but in other tests, the calculated statistic must be equal to or lower than a particular critical value in order to be deemed significant.  The critical value you need to use is determined by whether you are carrying out a one-tailed or two-tailed test, and by the level of significance you have decided on. A one-tailed test is used when the hypothesis is directional and a two- tailed test is used when the hypothesis is non-directional. Data Analysis and Reporting

Mann-Whitney U test  This is a test of difference that is suitable for comparing data gathered from two groups in an experiment using an independent groups design. It can be used with ordinal, interval or ratio data (since interval and ratio data can be converted to an ordinal level measurement by rank ordering the data as part of the test procedure.  In the following example which will be used to demonstrate how the test works, the data were obtained from a memory experiment. There are different people in each condition and no more than 20 observations in each condition. The directional hypothesis states that ‘more words are recalled in the experimental condition than in the control condition’. Data Analysis and Reporting

Mann-Whitney U test (cont.) Condition 1 (control)Condition 2 (experimental) Participant no. No. Of words recalled Rank orderParticipant no. No. Of words recalled Rank order 172102019 261111411 383.5121411 4128131817 595141513 61411151716 783.516139 8117171614.5 9106181918 191614.5 ∑ (sum of) ranks for condition 1 (the smaller sample) = 47 Data Analysis and Reporting

Mann-Whitney U test (cont.) 1. The data must be placed into the appropriate columns of a table. 2. The data must be ranked from the lowest value (rank 1) to the highest value (rank N). 3. The sum of ranks for the smaller of the two samples must then be calculated and the value named T. 4. The following formula must then be substituted in: U=N 1 N 2 +N 1 (N 1 =1)/2 –T U = The observed value of the Mann-Whitney statistic. N 1 = The number of values in the smaller sample (or in the sample for which the sum of the ranks has been calculated if both are the same size) N 2 = The number of values in the larger sample (or in the sample for which the sum of the ranks has not been calculated if both are the same size) U=(9x10)+9x(9+1)/2-47 U=(9x10)+90/2-47 U=90+45-47=88 U=88. This is the observed value of U. 5. Substitute in the following formula: U’=N 1 N 2 -U U’=(9x10)-88 U’=90-88 U’=2. This is the observed value of U’. Data Analysis and Reporting

Mann-Whitney U test (cont.) 6. Select the smaller value of U and U’. Whichever is the smallest value becomes the value of U; in this case the value of U’ is smallest (2). 7. Consult the table of critical values to obtain the critical values of U. In order to do this, you need to know:  The values of N 1 and N 2 (in this case 9 and 10).  Whether a one-tailed test (directional hypothesis used) or a two-tailed test (non-directional hypothesis used) is required. 8. Take the value of U (in this case 2) and use the correct table to compare the value of U with the critical value for a given significance level (in this case p=.05). The critical value of U for N1=9 and N2=10 for a one tailed test at p=.05 is 24. As the observed value of U (2) is less than the critical value (24), the probability of these results occurring if the null hypothesis is true is p<.05 (less than 5 per cent). In this case, the null hypothesis can be rejected and the experimental hypothesis is a more likely explanation of the results. Note: if the observed value had been greater than the critical value, then the probability of these results occurring if the null hypothesis is true would have been greater than 5 per cent. The null hypothesis would, therefore, have been retained. Data Analysis and Reporting

Wilcoxon matched pairs  The Wilcoxon matched pairs signed ranks test is a test of difference, suitable for use with data gathered from an experiment using a matched pairs or repeated measures design. It can be used on ordinal, interval and ratio data.  In the example, the data are taken from a memory experiment. The directional hypothesis states that more words are recalled in the experimental condition than in the control condition. Data Analysis and Reporting

Wilcoxon matched pairs (cont.) Participant no. No. Of words recalled (control condition) No. Of words recalled (experimental condition) DifferenceRank order 11720-34 21214-22.5 31614+22.5 41219-79 51615+11 61419-56 713 0(omitted) 81116-56 91319-68 101119-56 Data Analysis and Reporting

Wilcoxon matched pairs (cont.) 1. Place the data to be analysed into the appropriate columns of the table. 2. Calculate the difference between each pair of scores. It is essential that the direction of any difference is recorded. 3. Rank the data in the difference column, from the lowest value (rank 1) to the highest value (rank N). When you do this:  Any zero differences are disregarded.  Positive and negative signs are disregarded.  The ranks are shared for any scores which are tied. 4. Calculate the sum of ranks, which correspond to:  The differences with the + sign and;  The differences with the – sign.  Call the smaller of these values T. Sum of the ranks which correspond to the differences with the + sign = 2.5+1=3.5 Sum of the ranks which correspond to the differences with the – sign = 2.5+9+6+6+8+6=41.5 The observed value of T is therefore 3.5 (as this is the smallest of the two values). Data Analysis and Reporting

Wilcoxon matched pairs (cont.) 5. Consult the table of critical values in order to obtain the critical values of T. In order to obtain this, you need to know:  The value of N. Note that pairs of scores with a difference of zero are not included. In this example, the number of pairs of scores is 9.  Whether a one-tailed or a two-tailed test is required. 6. Take the observed value of T (in this case 3.5) and use the table to compare the value of T to the critical value for a given significance level (in this case p=.05). The critical value of T for N=9 for a one-tailed test at p=.05 is 8. As the observed value of T (3.5) is less than the critical value (8), the likelihood of the results occurring if the null hypothesis is true is p <.05 (less than 5 per cent). In this case the null hypothesis can be rejected and the experimental hypothesis is supported as a more likely explanation of the results. Note: if the observed value had been greater than the critical value, then the likelihood of results such as these occurring if the null hypothesis is true would have been greater than 5 per cent. The null hypothesis would have been retained. Data Analysis and Reporting

Spearman’s rank  Spearman’s rank order correlation coefficient (r s ) is a test of correlation suitable for use with pairs of scores. It can be used with ordinal, interval and ratio data. The test gives a correlation coefficient with a value between =1.00 and -1.00. The sign (+/-) indicates the direction of the relationship (positive or negative) and the number indicates the strength of the relationship, where 1 is a perfect correlation and a value between 0 and + or -1 is an imperfect correlation.  The example given uses data obtained from a study investigating the possible correlation between psychology test scores and biology test scores of a group of participants studying both subjects. The direction of the relationship is uncertain, so a non-directional hypothesis has been suggested. Data Analysis and Reporting

Spearman’s rank (cont.) Participant no. Psychology test score Rank order Biology test score Rank orderdd2d2 1951092911 227236200 347440311 468757524 55056161 694991811 73334141 826135100 9938 10-24 105967071 Data Analysis and Reporting

Spearman’s rank (cont.) 1. Draw a scattergraph of the data sets that you wish to correlate. This technique measures only straight-line relationships, and drawing a scattergraph can help you decide if this is the case. 2. Place the data to be analysed into the appropriate columns of a table. 3. Rank each set of scores separately, giving the lowest score rank 1 and the highest score rank N. Note: accuracy is diminished if this test is used when ranks are shared for any scores that are tied. 4. Find the difference (d) between each pair of rank order scores. 5. Square each of the d values. 6. Calculate the sum of (∑) the d 2 values. In this case, ∑d 2 =14. Data Analysis and Reporting

Spearman’s rank (cont.) 7. Substitute in the following formula: r s =1-(6∑d 2 /N(N 2 -1)) r s = the observed value of Spearman’s correlation coefficient. ∑d2 = the sum of the squared differences. N = the number of pairs of scores being correlated. r s =1-(84/10(100-1)) r s =1-84/10x99 r s =1-84/990 r s =1-0.0848 r s =0.9152. This is the value of r s which is more neatly expressed as.92. 8. Consult the table of critical values to obtain the critical value of r s. In order to do this you need to know:  The value of N.  Whether a one-tailed or two-tailed test is required. Data Analysis and Reporting

Spearman’s rank (cont.) 8. Take the observed value of r s (in this case.9152) and use the table to compare the value of r s to the critical value for a given significance level (in this case p=.05). The critical value of r s for a two-tailed test at p=.05 is.648. As the observed value of r s (.9152) is greater than the critical value (.648), the likelihood of results such as these occurring if the null hypothesis is true is less than 5 per cent. The null hypothesis can be rejected and the alternative hypothesis is a more likely explanation of the results. Note: if the observed value had been less than the critical value, the likelihood of such results occurring if the null hypothesis were true would be greater than 5 per cent. In which case, the null hypothesis could not have been rejected, and would have accounted for the results. Data Analysis and Reporting

Chi-squared test (x 2 )  The chi-squared test for independent samples (x2) is a test of association for use with data gathered from independent samples that are measured at a nominal level in the form of frequencies. It tests for differences by examining the association that exists between data categorized into rows and columns. It compares observed frequencies (those actually obtained) with expected frequencies (the frequencies which would be observed if the null hypothesis were true).  The following limitations can apply to the chi-squared test: 1. The chi-squared test should only be used in situations where each observation belongs in just one category. If an observation can go into more than one category, the data are not independent and the test cannot be used. 2. The observations used in the test must be actual frequencies of occurrence. Data such as averages, percentages or proportions should not be used. 3. The probability of making a Type 1 error is increased when there are expected frequencies of less than 5, especially when the total sample size is small. Using a larger sample size can minimize this potential problem.  The example shown is a specimen calculation of the chi-squared test for independent samples, using data from an investigation into children’s thinking. The non-directional hypothesis to be tested is that there is a difference in the problem solving ability of four- and five-year old children. Where RT1 and RT2 are row totals, CT1 and CT2 are column totals and GT is the grand total. Data Analysis and Reporting

Chi-squared test (cont.) No. of children able to solve problem No. Of children unable to solve problem Row total 4-year-old children Cell 1 8 Cell 2 12 RT1 20 5-year-old children Cell 3 17 Cell 4 3 RT2 20 Column total CT1 25 CT2 15 GT 40 1. Place the observed values to be analysed into the appropriate boxes in a table. This kind of table is called a ‘contingency table’. 2. Calculate the expected frequency for each cell, using the formula: Expected frequency (E)=(RTxCT/GT) 3. Subtract the expected frequency (E) from the observed frequency (O) for each cell: Cell 1: O-E=8-12.5=-4.5 Cell 2: O-E=12-7.5=4.5 Cell 3: O-E=17-12.5=4.5 Cell 4: O-E=3-7.5=-4.5 4. Calculate (O-E) 2 for each cell: Cell 1: -4.5 2 =20.25 Cell 2: 4.5 2 =20.25 Cell 3: 4.5 2 =20.25 Cell 4: -4.5 2 =20.25 5. Calculate (O-E) 2 /E for each cell: Cell 1: 20.25/12.5=1.62 Cell 2: 20.25/7.5=2.7 Cell 3: 20.25/12.5=1.62 Cell 4: 20.25/7.5=2.7 Data Analysis and Reporting

Chi-squared test (cont.) 6. Add the answers of (O-E) 2 /E for each cell to obtain the observed value of x 2 : 1.62+2.7+1.62+2.7=8.64 This is the observed value of x 2. Note: steps 2 to 6 can be represented by the following formula: x 2 =∑[(O-E) 2 /E] 7. calculate the number of degrees of freedom using the formula: (df)=(No. of rows – 1)x(No. of columns – 1) (df)=(2-1)x(2-1) (df)=1 8. Consult the table to obtain the critical values of x 2. In order to do this you need to know:  The number of degrees of freedom.  Whether a one-tailed or two-tailed test was used. 9. Take the observed value of x 2 (in this case 8.64) and use the table to compare this to the critical value for a given significance level (in this case p=.05). The critical value of x 2 for df=1 and a two-tailed test at p=.05 is 3.84. As the observed value for x 2 (8.64) is greater than the critical value (3.84), the likelihood of these results occurring if the null hypothesis is true is less than 5 per cent (p<.05). In this case, the null hypothesis would be rejected and the alternative hypothesis is a more likely explanation of the results. Note: if the observed value had been less than the critical value, then the probability of results such s these occurring if the null hypothesis were true would be greater than 5 per cent. In this case, the null hypothesis would have been retained. Data Analysis and Reporting

A complete presentation. Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE.

Similar presentations

Presentation on theme: "A complete presentation. Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

A complete presentation. Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE.

Similar presentations

Presentation on theme: "A complete presentation. Objectivity; replicability; the role of paradigms The application of scientific method MAJOR FEATURES OF A SCIENCE."— Presentation transcript:

Similar presentations

About project

Feedback