The meaning of Reliability and Validity in psychological research

Write down the meaning of reliability White board time Write down the meaning of reliability Clue: If your car is reliable what does this mean?

Now write down the meaning of validity The point she made was valid! Relevant has relevance

Obviously this means your new reading test isn’t any good! Imagine you’ve produced a new test to measures people’s reading ability but each time you take the test it gives different results. Obviously this means your new reading test isn’t any good! Why isn't this reading test any good?

What if…. You get one score when I mark the test but a completely different score when another person marks it! Clearly these scores are not very ____________________________?

So if the results are inconsistent they cannot be relied on because they are not trustworthy. Psychologists would say these results lack ‘reliability’.

Also The test doesn’t measure what it set out to measure so it lacks validity as well!

In order to conduct research, psychologists must find ways of measuring things. Many instruments of measurement we are already familiar with like bathroom scales, rulers, speedometers etc.

But for these things to be useful we need to be able to trust them But for these things to be useful we need to be able to trust them! The instrument of measurement has to be reliable It must measure what we want it to measure and it must be consistent, in other words you should measure the same height wherever and whenever you measure yourself – Your height will stay the same and should be consistent provided your tape measure is reliable that is!

Reliability refers to the consistency of a measuring instrument. If a measurement is reliable it will be consistent and stable. If a measurement is reliable we can trust the results.

How can we be sure if a measurement is reliable? Reliability can be checked using correlational techniques For example we could use the Test-Retest method Participants take a test twice and if the test is reliable the two scores will be highly correlated, so if they score high in one test they should score high in the other one. We say a piece of research is reliable if we can replicate and get similar results

Which is likely to be more reliable Which is likely to be more reliable? (show a high positive correlation between the two scores) The correlation of the marks from two examiners marking the same multiple choice test? OR The correlation of marks awarded to the same essay by two different examiners? Can you explain why?

The agreement on the multiple choice test should be high, that is there should be a high positive correlation between marks obtained by the two sets of scorers. On the other hand, if the two scorers were marking essays then there would probably not be such a high correlation.

Inter–rater reliability This type of reliability is concerned with how closely different people who are marking a test or observing a behaviour agree with each other. It is not therefore concerned with the reliability of the test itself (test-retest) – it is more to do with how much we can trust the person doing the marking!

In observation studies it is often necessary for training to take place in order to be able to establish the highest level of inter-rater/observer reliability as possible. Pilot studies are useful in this instance – observers can try out the observation schedule and then compare their results with another observer. If the observation schedule is a reliable measurement tool the observers will have similar results… and observers can ensure they are looking for the exact same things!

Validity The validity of a psychological measure is the extent to which it measures what it intended to measure – Sometimes even experienced researchers have accidently measured something else instead

If you can think of any other possible explanation for your findings then it could be that your research isn’t valid. For example does an IQ test actually tell us how intelligent a person is? Or is it actually measuring something else? Like how well a person can read English or how good their general knowledge is. If an IQ test doesn’t actually measure INTELLIGENCE then it is not VALID.

A tape measure may be a reliable measurement tool – but if we were using it to measure the circumference of peoples’ heads as a way of measuring intelligence it wouldn’t be very valid!

In your notes Draw a picture of a tape measure measuring a head explain why it is Reliable but not Valid

Testing validity The easiest way to discover whether a test is valid is to examine it and decide whether it looks as though it is. If a test which was supposed to measure intelligence contained a large memory test section then it would not have FACE VALIDITY (it looks wrong) Clearly it LOOKS more like a memory test than an intelligence test (assuming of course that intelligence is more than just having a good memory).

You can always ask an expert! If independent experts in the subject area agree that the test looks as though it does what it is supposed to do, then the test is said to have CONTENT VALIDITY.

Check your understanding The extent to which a test measures what was intended is known as what?

Answer: Validity

Face validity Content Validity Construct validity The extent to which the measurement object of a test is self-evident is known as what? Face validity Content Validity Construct validity

Answer: Face Validity

Valid Consistent Meaningful Useful Reliability means the measurement is? Valid Consistent Meaningful Useful

Answer: Consistent