1. Applied Quantitative Methods MBA course Montenegro
Peter Balogh
PhD

2. Significance testing

3. Significance testing
Significance testing aims to make statements about a population parameter, or parameters, on the basis of sample evidence.
It is sometimes called point estimation.
The emphasis here is on testing whether a set of sample results support, or are consistent with, some fact or supposition about the population.
We are thus concerned to get a 'Yes' or 'No' answer from a significance test; either the sample results do support the supposition, or they do not.
As you might expect, the two ideas of significance testing and confidence intervals are so closely linked, all of the assumptions that we have made in Chapter 11 about the underlying sampling distribution remain the same for significance tests.

4. Significance testing
Since we are dealing with samples, and not a census, we can never be 100% sure of our results (since sample results will vary from sample to sample and from the population parameter in a known way - see Figure 11.5).
However, by testing an idea (or hypothesis) and showing that it is very, very unlikely to be true we can make useful statements about the population which can form the basis for business decision making.

5. Significance testing
Significance testing is about putting forward an idea or testable proposition that captures the properties of the problem that we are interested in.
The idea we are testing may have been obtained from a past census or survey, or it may be an assertion that we put forward, or has been put forward by someone else, for example, an assertion that a majority of people support legislation to enhance animal welfare.
Similarly, it could be a critical value in terms of product planning; for example, if a new machine is to be purchased but the process is only viable with a certain level of throughput, we would want to be sure that the company can sell, or use, that quantity before commitment to the purchase of that particular type of machine.

6. Significance testing
Advertising claims could also be tested using this method by taking a sample and testing if the advertised characteristics, for example strength, length of life, or percentage of people preferring this brand, are supported by the evidence of the sample.
In developing statistical measures we may want to test if a certain parameter is significantly different from zero; this has useful applications in the development of regression and correlation models (see Chapters 15-17).

7. Significance testing
Taking the concept a stage further, we may have a set of sample results from two regions giving the percentage of people purchasing a product, and want to test whether there is a significant difference between the two percentages.
Similarly, we may have a survey which is carried out on an annual basis and wish to test whether there has been a significant shift in the figures since last year.
In the same way that we created a confidence interval for the differences between two samples, we can also test for such differences.

8. Significance testing
Finally we will also look at the situation where only small samples are available and consider the application of the t-distribution to significance testing; another concept to which we will return in Chapters 15 and 16.

10. 12.1 Significance testing using confidence intervals
Significance testing is concerned with accepting or rejecting ideas.
These ideas are known as hypotheses.
If we wish to test one in particular, we refer to it as the null hypothesis.
The term 'null' can be thought of as meaning no change or no difference from the assertion.
As a procedure, we would first state a null hypothesis; something we wish to judge as true or false on the basis of statistical evidence.

11. 12.1 Significance testing using confidence intervals
We would then check whether or not the null hypothesis was consistent with the confidence interval.
If the null hypothesis is contained within the con­fidence interval it will be accepted, otherwise, it will be rejected.
A confidence interval can be regarded as a set of acceptable hypotheses.
To illustrate significance testing consider an example from Chapter 11.

12. 12.1 Significance testing using confidence intervals
Case study
In the Arbour Housing Survey, it was found that the average monthly rent was £ and that the standard deviation was £89.48, on the basis of 180 responses (see Section ).

13. 12.1 Significance testing using confidence intervals
The 95% confidence interval was constructed as follows:

14. 12.1 Significance testing using confidence intervals
Now suppose the purpose of the survey was to test the view that the amount paid in monthly rent was about £200.
Given that the confidence interval can be regarded as a set of acceptable hypotheses, the null hypothesis (denoted by H0) would be written as:
H0 : µ = £200
As the value of £200 is not included within the confidence interval the null hypothesis must be rejected.
The values of the null hypothesis that we can accept or reject are shown on Figure 12.2.

16. 12.1 Significance testing using confidence intervals
In rejecting the view that the average rent is £200 we must also accept that there is a chance that our decision could be wrong.
There is a 2.5% chance that the average is less than £ (which would include an average of £200) and a 2.5% chance that the average is greater than £
As we shall see (Section 12.4) there is a probability of making a wrong decision, which we need to balance against the probability of making the correct decision.

17. 12.2 Hypothesis testing for single samples
Hypothesis testing is merely an alternative name for significance tests, the two being used interchangeably.
This name does stress that we are testing some supposition about the population, which we can write down as a hypothesis.
In fact, we will have two hypotheses whenever we conduct a significance test: one relating to the supposition that we are testing, and one that describes the alternative situation.

18. 12.2 Hypothesis testing for single samples
The first hypothesis relates to the claim, supposition or previous situation and is usually called the null hypothesis and labeled as H0.
It implies that there has been no change in the value of the parameter that we are testing from that which previously existed; for example, if the average spending per week on beer last year among year olds was £15.53, then the null hypothesis would be that it is still £15.53.
If it has been claimed that 75% of consumers prefer a certain flavour of icecream, then the null hypothesis would be that the population percentage preferring that flavour is equal to 75%.

19. 12.2 Hypothesis testing for single samples
The null hypothesis could be written out as a sentence, but it is more usual to abbreviate it to:
for a mean where µ0 is the claimed or previous population mean. For a percentage:
where π0 is the claimed or previous population percentage.

20. 12.2 Hypothesis testing for single samples
The second hypothesis summarizes what will be the case if the null hypothesis is not true.
It is usually called the alternative hypothesis (fairly obviously!), and is labelled as HA or as H1 depending on which text you follow: we will use the H1 notation here.
This alternative hypothesis is usually not specific, in that it does not usually specify the exact alternative value for the population parameter, but rather, it just says that some other value is appropriate on the basis of the sample evidence; for example, the mean amount spent is not equal to £15.53, or the percentage preferring this flavour of icecream is not 75%.

21. 12.2 Hypothesis testing for single samples
As before, this hypothesis could be written out as a sentence, but a shorter notation is usually preferred:
for a mean where µ0 is the claimed or previous population mean.
For a percentage:
where π0 is the claimed or previous population percentage.

22. 12.2 Hypothesis testing for single samples
Whenever we conduct a hypothesis test, we assume that the null hypothesis is true while we are doing the test, and then come to a conclusion on the basis of the figures that we calculate during the test.
Since the sampling distribution of a mean or a percentage (for large samples) is given by the Normal distribution, to conduct a test, we compare the sample evidence (sample statistic) with the null hypothesis (what is assumed to be true).
This difference is measured by a test statistic. A test statistic of zero or reasonably near to zero would suggest the null hypothesis is correct.
The larger the value of the test statistic the larger the difference between the sample evidence and the null hypothesis.

23. 12.2 Hypothesis testing for single samples
If the difference is sufficiently large we conclude that the null hypothesis is incorrect and accept the alternative hypothesis.
To make this decision we need to decide on a critical value or critical values.
The critical value or values define the point at which the chance of the null hypothesis being true is at a small, predetermined level, usually 5% or 1% (called the significance level).
In this chapter you will see the z-value being used as a test statistic.
It is more usual to divide the Normal distribution diagram into sections or areas, and to see whether the z-value falls into a particular section; then we may either accept or reject the null hypothesis.

24. 12.2 Hypothesis testing for single samples
However, it is worth noting that some statistical packages just calculate the probability of the null hypothesis being true when you use them to conduct tests.
The reason for doing this is to see how likely or unlikely the result is.
If the result is particularly unlikely when the null hypothesis is true, then we might begin to question whether this is, in fact, the case.
Obviously we will need to define more exactly the phrase 'particularly unlikely' in this context before we can proceed with hypothesis tests.

25. 12.2 Hypothesis testing for single samples
Most tests are conducted at the 5% level of significance, and you should recall that in the normal distribution, the z-values of and cut off a total of 5% of the distribution, 2.5% in each tail.
If a calculated z-value is between and +1.96, then we accept the null hypothesis; if the calculated z-value is below or above +1.96, we reject the null hypothesis in favour of the alternative hypothesis.
This situation is illustrated in Figure 12.3.

27. 12.2 Hypothesis testing for single samples
If we were to conduct the test at the 1% level of significance, then the two values used to cut off the tail areas of the distribution would be and
For each test that we wish to conduct, the basic layout and procedure will remain the same, although some of the details will change, depending on what exactly we are testing.
A proposed layout is given below, and we suggest that by following this you will present clear and understandable significance tests (and not leave out any steps).

29. 12.2 Hypothesis testing for single samples
While the significance level may change, which will lead to a change in the critical values, the major difference as we conduct different types of hypothesis test is in the way in which we calculate the z-value at step 4.

30. 12.2.1 A test statistic for a population mean
In the case of testing for a particular value for a population mean, the formula used to calculate z in step 4 is:
As we saw in Chapter 11, the population standard deviation (σ) is rarely known, and we use the sample standard deviation (s) in its place.
We are assuming that the null hypothesis is true, and so µ=µ0

31. 12.2.1 A test statistic for a population mean
The formula will become:
We are now in a position to carry out a test.

32. 12.2.1 A test statistic for a population mean
Example
A production manager claims that an average of 50 boxes per hour are filled with finished goods at the final stage of a production line.
A random sampling of 48 different workers, at different times, working at the end of identical production lines shows an average number of boxes filled as 47.5 with a standard deviation of 0.7 boxes.
Does this evidence support the assertion by the production manager at the 5% level of significance?

33. 12.2.1 A test statistic for a population mean
Step 1: The null hypothesis is based on the production manager's assertion:

34. 12.2.1 A test statistic for a population mean
Although there is only a 2.5 difference between the claim and the sample mean in this example, the test shows that the sample result is significantly different from the claimed value.
In all hypothesis testing, it is not the absolute difference between the values which is important, but the number of standard errors that the sample value is away from the claimed value.
Statistical significance should not be confused with the notion of business significance or importance.

35. 12.2.2 A test for a population percentage
Here the process will be identical to the one followed above, except that the formula used to calculate the z-value will need to be changed.
(You should recall from Chapter 11 that the standard error for a percentage is different from the standard error for a mean.)
The formula will be:
where p is the sample percentage and Π0 is the claimed population percentage (remember that we are assuming that the null hypothesis is true).

36. 12.2.2 A test for a population percentage
Example
An auditor claims that 10% of invoices for a company are incorrect.
To test this claim a random sample of 100 invoices are checked, and 12 are found to be incorrect.
Test, at the 5% significance level, if the auditor's claim is supported by the sample evidence.

37. 12.2.2 A test for a population percentage

38. 12.2.2 A test for a population percentage
Step 5: The calculated value falls between the two critical values.
Step 6: We therefore cannot reject the null hypothesis.
Step 7: The evidence from the sample is consistent with the auditor's claim that 10% of the invoices are incorrect.

39. 12.2.2 A test for a population percentage
In this example, when the calculated value falls between the two critical values.
The answer is to not reject the claim of 10% rather than to accept the claim.
This is because we only have the sample evidence to work from, and we are aware that it is subject to sampling error.
The only way to firmly accept the claim would be to check every invoice (i.e. carry out a census) and work out the actual population percentage which are incorrect.
When we calculated a confidence interval for the population percentage (in Section 11.4) we used a slightly different formulation of the sampling error.

40. 12.2.2 A test for a population percentage
There we used
because we did not know the value of the population percentage; we were using the sample percentage as our best estimate, and also as the only available value.
When we come to significance tests, we have already made the assumption that the null hypothesis is true (while we are conducting the test), and so we can use the hypothesized value of the population percentage in the calculations.

41. 12.2.2 A test for a population percentage
The formula for the sampling error will thus be
In many cases there would be very little difference in the answers obtained from the two different formulae, but it is good practice (and shows that you know what you are doing) to use the correct formulation.

42. 12.2.2 A test for a population percentage
Case study
It has been claimed on the basis of census results that 87% of households in Tonnelle now have exclusive use of a fixed bath or shower with a hot water supply.
In the Arbour Housing Survey of this area, 246 respondents out of the 300 interviewed reported this exclusive usage.
Test at the 5% significance level whether this claim is supported by the sample data.

43. 12.2.2 A test for a population percentage

44. 12.2.2 A test for a population percentage
Step 5: The calculated value falls below the critical value of
Step 6: We therefore reject the null hypothesis at the 5% significance level.
Step 7: The sample evidence does not support the view that 87% of households in the Tonnelle area have exclusive use of a fixed bath or shower with a hot water supply.
Given a sample percentage of 82% we could be tempted to conclude that the percentage was lower - the sample does suggest this, but the test was not structured in this way and we must accept the alternative hypothesis that the population percentage is not likely to be 87%.
We will consider this issue again when we look at one-sided tests.

45. 12.3 One-sided significance tests
The significance tests shown in the last section may be useful if all we are trying to do is test whether or not a claimed value could be true, but in most cases we want to be able to go one step further.
In general, we would want to specify whether the real value is above or below the claimed value in those cases where we are able to reject the null hypothesis.

46. 12.3 One-sided significance tests
One-sided tests will allow us to do exactly this.
The method employed, and the appropriate test statistic which we calculate, will remain exactly the same; it is the alternative hypothesis and the interpretation of the answer which will change.
Suppose that we are investigating the purchase of cigarettes, and know that the percentage of the adult population who regularly purchased last year was 34%.
If a sample is selected, we do not want to know only whether the percentage purchasing has changed, but rather whether it has decreased (or increased).
Before carrying out the test it is necessary to decide which of these two propositions you wish to test.

47. 12.3 One-sided significance tests
If we wish to test whether or not the percentage has decreased, then our hypotheses would be:
where π0 is the actual percentage in the population last year.
This may be an appropriate hypothesis test if you were working for a health lobby.

48. 12.3 One-sided significance tests
If we wanted to test if the percentage had increased, then our hypotheses would be:
This could be an appropriate test if you were working for a manufacturer in the tobacco industry and were concerned with the effects of improved packaging and presentation.

49. 12.3 One-sided significance tests
To carry out the test, we will want to concentrate the chance of rejecting the null hypothesis at one end of the Normal distribution.
Where the significance level is 5%, then the critical value will be (i.e. the cut off value taken from the Normal distribution tables) for the hypotheses H0: π = π0 , H1: π < π0; and for the hypotheses H0: π = π0, H1 : π > π0. (Check these figures from Appendix C.)
Where the significance level is set at 1%, then the critical value becomes either or
In terms of answering examination questions, it is important to read the wording very carefully to determine which type of test you are required to perform.

51. 12.3 One-sided significance tests
The interpretation of the calculated z-value is now merely a question of deciding into which of two sections of the Normal distribution it falls.

52. 12.3.1 A one-sided test for a population mean
Here the hypotheses will be in terms of the population mean, or the claimed value.
Consider the following example.

53. 12.3.1 A one-sided test for a population mean
Example
A manufacturer of batteries has assumed that the average expected life is 299 hours.
As a result of recent changes to the filling of the batteries, the manufacturer now wishes to test if the average life has increased.
A sample of 200 batteries was taken at random from the production line and tested.
Their average life was found to be 300 hours with a standard deviation of 8 hours.
You have been asked to carry out the appropriate hypothesis test at the 5% significance level.

54. 12.3.1 A one-sided test for a population mean
(Note that we are still assuming the null hypothesis to be true while the test is conducted.)

55. 12.3.1 A one-sided test for a population mean
Step 5: The calculated value is larger than the critical value.
Step 6: We may therefore reject the null hypothesis.
(Note that had we been conducting a two-sided hypothesis test, then we would have been unable to reject the null hypothesis, and so the conclusion would have been that the average life of the batteries had not changed.)
Step 7: The sample evidence supports the supposition that the average life of the batteries has increased by a significant amount.

56. 12.3.1 A one-sided test for a population mean
Although the significance test has shown that there has been a significant increase in the average length of life of the batteries, this may not be an important conclusion for the manufacturer.
For instance, it would be quite misleading to use it to back up an advertising campaign which claimed that 'our batteries now last even longer!'.
Whenever hypothesis tests are used it is important to distinguish between statistical significance and importance.
This distinction is often ignored. 

57. 12.3.1 A one-sided test for a population mean
Case study
It has been argued that, because of the types of property and the age of the population in Tonnelle, average mortgages are likely to be around £200 a month.
However, the Arbour Housing Trust believe the figure is higher because many of the mortgages are relatively recent and many were taken out during the house price boom.
Test this proposition, given the result from the Arbour Housing Survey that 100 respondents were paying an average monthly mortgage of £254.
The standard deviation calculated from the sample is £72.05 (it was assumed to be £70 in an earlier example).
Use a 5% significance level.

58. 12.3.1 A one-sided test for a population mean
The test statistic value of 7.49 is greater than the critical value of
On this basis, we can reject the null hypothesis that monthly mortgages are around £200 in favour of the alternative that they are likely to be more.

59. 12.3.1 A one-sided test for a population mean
A test provides a means to clarify issues and resolve different arguments.
In this case the average monthly mortgage payment is higher than had been argued but this does not necessarily mean that the reasons put forward by the Arbour Housing Trust for the higher levels are correct.
In most cases, further investigation is required.
It is always worth checking that the sample structure is the same as the population structure.

60. 12.3.1 A one-sided test for a population mean
In an area like Tonnelle we would expect a range of housing and it can be difficult for a small sample to reflect this.
The conclusions also refer to the average.
The pattern of mortgage payments could vary considerably between different groups of residents.
It is known that the Victorian housing is again attracting a more affluent group of residents, and they could have the effect of pulling the mean upwards.
Finally, the argument for the null hypothesis could be based on dated information, e.g. the last census.

61. 12.3.2 A one-sided test for a population percentage
Here the methodology is exactly the same as that employed above, and so we just provide two examples of its use.
Example
A small political party expects to gain 20% of the vote in by-elections.
A particular candidate from the party is about to stand in a by-election in Derbyshire South East, and has commissioned a survey of 200 randomly selected voters in the constituency.
If 44 of those interviewed said that they would vote for this candidate in the forthcoming by-election, test whether this would be significantly above the national party's claim.
Use a test at the 5% significance level.

62. 12.3.2 A one-sided test for a population percentage

63. 12.3.2 A one-sided test for a population percentage
Here there may not be evidence of a statistically significant vote above the national party's claim (a sample size of 1083 would have made this 2% difference statistically significant), but if the candidate does manage to achieve 22% of the vote, and it is a close contest over who wins the seat between two other candidates, then the extra 2% could be very important.

64. 12.3.2 A one-sided test for a population percentage
Example
A fizzy drink manufacturer claims that over 40% of teenagers prefer its drink to any other.
In order to test this claim, a rival manufacturer commissions a market research company to conduct a survey of 200 teenagers and ascertain their preferences in fizzy drinks.
Of the randomly selected sample, 75 preferred the first manufacturers drink.
Test at the 5% level if this survey supports the claim.
Sample percentage is:

65. 12.3.2 A one-sided test for a population percentage

66. 12.3.2 A one-sided test for a population percentage
Case study
The Arbour Housing Survey was an attempt to describe local housing conditions on the basis of responses to a questionnaire.
Many of the characteristics will be summarized in percentage terms, e.g. type of property, access to baths, showers, flush toilets, kitchen facilities.
Such enquiries investigate whether these percentages have increased or decreased over time, and whether they are higher or lower than in other areas.
If this increase or decrease, or higher or lower value is specified in terms of an alternative hypothesis, the appropriate test will be one-sided.

67. 12.3.3 Producer’s risk and consumers’ risk
One-sided tests are sometimes referred to as testing producers' risks or consumers' risks.
If we are looking at the amount of a product in a given packet or lot size, then the reaction to variations in the amount may be different from the two
groups.
Say, for instance, that the packet is sold as containing 100 items.
If there are more than 100 items per packet, then the producer is effectively giving away some items, since only 100 are being charged for.

68. 12.3.3 Producer’s risk and consumers’ risk
The producer, therefore, is concerned not to overfill the packets but still meet legal requirements.
In this situation, we might presume that the consumer is quite happy, since some items come free of charge.
In the opposite situation of less than 100 items per packet, the producer is supplying less than 100 but being paid for 100 (this, of course, can have damaging consequences for the producer in terms of lost future sales), while the consumers receive less than expected, and are therefore likely to be less than happy.
Given this scenario, one would expect the producer to conduct a one­sided test using H1: µ > µ0, and a consumer group to conduct a one-sided test using H1: µ < µ0.
There may, of course, be legal implications for the producer in selling packets of 100 which actually contain less than 100.
(In practice most producers will, in fact, play it safe and aim to meet any minimum requirements.)

69. 12.4 Types of error
We have already seen, throughout this section of the book, that samples can only give us a partial view of a population; there will always be some chance that the true population value really does lie outside of the confidence interval, or  that we will come to the wrong decision when conducting a significance test.
In fact these probabilities are specified in the names that we have already used - a 95% confidence interval and a test at the 5% level of significance.
Both imply a 5% chance of being wrong.

70. 12.4 Types of error
You might want to argue that this number is too big, since it gives a 1 in 20 chance of making the wrong decision, but it has been accepted in both business and social research over many years.
One reason is that we are dealing with a situation where we rely on people telling us the 'truth' which they may not always do!
Using a smaller number, say 1%, would also imply a spurious level of accuracy in our results.
(Even in medical research, the figure of 5% may be used.)

71. 12.4 Types of error
If you consider significance tests a little further, however, you will see that there are, in fact, two different ways of getting the wrong answer. You could throw out the claim when it is, in fact, true; or you could fail to reject it when it is, in fact, false.
It becomes important to distinguish between these two types of error.
As you can see from Table 12.1, the two different types of error are referred to as Type I and Type II.

72. 12.4 Types of error Table 12.1 Possible results of a hypothesis test
A Type I error is the rejection of a null hypothesis when it is true.
The probability of this is known as the significance level of the test.
This is usually set at either 5% or 1% for most business applications, and is decided on before the test is conducted.
A Type II error is the failure to reject a null hypothesis which is false.
The probability of this error cannot be determined before the test is conducted.

73. 12.4 Types of error Consider a more everyday situation.
You are standing at the curb, waiting to cross a busy main road.
You have four possible outcomes:
If you decide not to cross at this moment, and there is something coming, then you have made a correct decision.
If you decide not to cross, and there is nothing coming, you have wasted a little time, and made a Type II error.
If you decide to cross and the road is clear, you have made a correct decision.
If you decide to cross and the road isn't clear, as well as the likelihood of injury, you have made a Type I error.
The error which can be controlled is the Type I error; and this is the one which is set before we conduct the test.

74. Background theory

75. Background theory
Consider the sampling distribution of z (which we calculate in step 4), consistent with the null hypothesis, H0: µ = µ0 illustrated as Figure 12.5.
The two values for the test-statistic, A and B, shown in Figure 12.5, are both possible, with B being less likely than A.
The construction of this test, with a 5% significance level, would mean the acceptance of the null hypothesis when A was obtained and its rejection when B was obtained.
Both values could be attributed to an alternative hypothesis, H1: µ=µ1, which we accept in the case of B and reject for A.

76. Background theory
It is worth noting that if the test statistic follows a sampling distribution we can never be certain about the correctness of our decision.
What we can do, having fixed a significance level (Type I error), is construct the rejection region to minimize Type II error.
Suppose the alternative hypothesis was that the mean for the population, µ, was not µ0 but a larger value µ1.

77. 12.4.1 Background theory This we would state as:
H1 : µ= µ1, where µ1 > µ0 or, in the more general form:
H1 : µ > µ0 .
If we keep the same acceptance and rejection regions as before (Figure 12.5), the Type II error is as shown in Figure 12.6.

78. Figure 12.6 The Type II error resulting from a two-tailed test.

79. Background theory
As we can see, the probability of accepting H0 when H1 is correct can be relatively large.
If we test the null hypothesis H0: µ = µ0 against an alternative hypothesis of the form H1: µ < µ0 or H1: µ > µ0 , we can reduce the size of the Type II error by careful definition of the rejection region.
If the alternative hypothesis is of the form H1: µ < µ0, a critical value of z = will define a 5% rejection region in the left-hand tail, and if the alternative hypothesis is of the form H1: µ > µ0 a critical value of z = 1.645, will define a 5% rejection region in the right-hand tail.
The reduction in Type II error is illustrated in Figure 12.7.

80. 12.4 Types of error

81. 12.4 Types of error
It can be seen from Figure 12.7 that the test statistic now rejects the null hypothesis in the range as well as values greater than 1.96.
If we construct one-sided tests, there is more chance that we will reject the null hypothesis in favour of a more radical alternative that the parameter has a larger value (or has a smaller value) than specified when that alternative is true.
Note that if the alternative hypothesis was of the form H1: µ < µ0, the rejection region would be defined by the critical value z = and we would reject the null hypothesis if the test statistic took this value or less.

82. 12.5 Hypothesis testing with two samples
All of the tests so far have made comparisons between some known, or claimed, population value and a set of sample results.
In many cases we may know little about the actual population value, but will have available the results of another survey.
Here we want to find if there is a difference between the two sets of results.
This could be a situation where we have two surveys conducted in different parts of the country, or at different points in time.
Our concern now is to determine if the two sets of results are consistent with each other (come from the same parent population) or whether there is a difference between them.

83. 12.5 Hypothesis testing with two samples
Such comparisons may be important in a marketing context, for instance, looking at whether or not there are different perceptions of the product in different regions of the country.
They could be used in performance appraisal of employees to answer questions on whether the average performance in different regions is, in fact, different.
They could be used in assessing a forecasting model, to see if the values have changed significantly from one year to the next.

84. 12.5 Hypothesis testing with two samples
Again we can use the concepts and methodology developed in previous sections of this chapter.
Tests may be two-sided to look for a difference, or one­sided to look for a significant increase (or decrease).
Similarly, tests may be carried out at the 5% or the 1% level, and the interpretation of the results will depend upon the calculated value of the test statistic and its comparison to a critical value.
It will be necessary to rewrite the hypotheses to take account of the two samples and to find a new formulation for the test statistic (step 4 of the process).

85. 12.5.1 Tests for a difference of means
In order to be clear about which sample we are talking about, we will use the suffixes 1 and 2 to refer to sample 1 and sample 2.
Our assumption is that the two samples do, in fact, come from the same population, and thus the population means associated with each sample result will be the same.
This assumption will give us our null hypothesis:

86. 12.5.1 Tests for a difference of means
The alternative hypothesis will depend on whether we are conducting a two-sided test or a one-sided test.
If it is a two-sided test, the alternative hypothesis will be:
and if it is a one-sided test, it would be either

87. 12.5.1 Tests for a difference of means
We could also test whether the difference between the means could be assigned to a specific value, for instance that the difference in take home pay between two groups of workers was £25; here we would use the hypotheses:
The test statistic used is closely related to the formula developed in Section for the confidence interval for the difference between two means.

88. 12.5.1 Tests for a difference of means
It will be:
but if we are using a null hypothesis which says that the two population means are equal, then the second bracket on the top line will be equal to zero, and the formula for z will be simplified to:

89. 12.5.1 Tests for a difference of means
The example below illustrates the use of this test.
Example
A union claims that the standard of living for its members in the UK is below that of employees of the same company in Spain.
A survey of 60 employees in the UK showed an average income of £895 per week with a standard deviation of £120.
A survey of 100 workers in Spain, after making adjustments for various differences between the two countries and converting to sterling, gave an average income of £914 with a standard deviation of £ 90.
Test at the 1% level if the Spanish workers earn more than their British counterparts.

90. 12.5.1 Tests for a difference of means

91. 12.5.2 Tests for a difference in population percentage
Again, adopting the approach used for testing the difference of means, we will modify the hypotheses, using π instead of µ for the population values, and redefine the test statistic.
Both one- and two-sided tests may be carried out.
The test statistic will be:
but if the null hypothesis is π1 – π2 = 0, then this is simplified to:

92. 12.5.2 Tests for a difference in population percentage
Example
A manufacturer believes that a new promotional campaign will increase favourable perceptions of the product by 10%.
Before the campaign, a random sample of 500 consumers showed that 20% had favourable attitudes towards the product.
After the campaign, a second random sample of 400 consumers found favourable reactions amongst 28%.

93. 12.5.2 Tests for a difference in population percentage
Use an appropriate test at the 5% level of significance to find if there has

94. 12.6 Hypothesis testing with small samples
As we saw in Chapter 11, the basic assumption that the sampling distribution for the sample parameters is a Normal distribution only holds when the samples are large (n > 30).
Once we turn to small samples, we need to use a different sampling distribution: the t-distribution (see Section 11.7).
Apart from this difference, which implies a change to the formula for the test statistic (step 4 again!) and also to the table of critical values, all of the tests developed so far in this chapter may be applied to small samples.

95. Single samples
For a single sample, the test statistic for a population mean is calculated by using:
with (n - 1) degrees of freedom.
For a single sample, the test statistic for a population percentage will be calculated using:
and, again there will be (n - 1) degrees of freedom.
(Note that, as with the large sample test, we use the null hypothesis value of the population percentage in the formula.) |
Below are two examples to illustrate the use of the t-distribution in hypothesis testing.

96. 12.6.1 Single samples Example
A lorry manufacturer claims that the average annual maintenance cost for its vehicles is £500.
The maintenance department of one of their customers believes it to be higher, and to test this randomly selects a sample of six lorries from their large fleet.
From this sample, the mean annual maintenance cost was found to be £555, with a standard deviation of £75.
Use an appropriate hypothesis test, at the 5% level, to find if the manufacturer's claim is valid.

97. Single samples

98. 12.6.1 Single samples Example
A company had a 50% market share for a newly developed product last year.
It believes that as more entrants start to produce this type of product, its market share will decline, and in order to monitor this situation, decides to select a random sample of 15 customers.
Of these, six have bought the company's product.
Carry out a test at the 5% level to find if there is sufficient evidence to suggest that their market share is now below 50%.

99. Single samples

100. Two samples
Where we have two samples, we will again use similar tests to Section 12.5 but with formulae developed using the sampling errors developed in Chapter 11.
In the case of estimating a confidence interval for the difference between two means the pooled standard error (assuming that both samples have similar variability) was given by:

101. Two samples
Using this, we have the test statistic:

102. Two samples
Example
A company has two factories, one in the UK and one in Germany.
The head office staff feel that the German factory is more efficient than the British one, and to test this select two random samples.
The British sample consists of 20 workers who take an average of 25 minutes to complete a standard task.
Their standard deviation is 5 minutes.
The German sample has 10 workers who take an average of 20 minutes to complete the same task, and the sample has a standard deviation of 4 minutes.
Use an appropriate hypothesis test, at the 1% level, to find if the German workers are more efficient.

103. Two samples

104. Matched pairs
A special case arises when we are considering tests of the difference of means if the two samples are related in such a way that we may pair the observations.
This may arise when two different people assess the same series of situations, for example different interviewers assessing the same group of candidates for a job.
The example below illustrates the situation of matched pairs.

105. Marks given to candidate:
Matched pairs
Example
Seven applicants for a job were interviewed by two personnel officers who were asked to give marks on a scale of 1 to 10 to one aspect of the candidates' performance.
A summary of the marks given is shown below.
Marks given to candidate: I II
III IV V VI
VII
Interviewer A 8 7 6 9 5
Interviewer B 4

106. Matched pairs

107. 12.6.3 Matched pairs Interviewer Difference A B (X) (x-x)2 8 7 1 4 3 6A B
(X)
(x-x)2 8 7 1 4 3 6 9 5 2 -1 10

108. Matched pairs

109. Matched pairs
The use of such matched pairs is common in market research and psychology.
If, for reasons of time and cost, we need to work with small samples, results can be improved using paired samples since
it reduces the between-sample variation.

110. 12.7 Conclusions
The ideas developed in this chapter allow us to think about statistical data as evidence, if we consider the practice of law as an analogy, then a set of circumstances require a case to be made, an idea is proposed (that of innocence), an alternative is constructed (that of guilt) and evidence is construed to arrive at a decision.
In statistical testing, a null hypothesis is proposed, usually in terms of no change or no difference, an alternative is constructed and data is used to accept or reject the null hypothesis.
Strangely, perhaps, these tests are not necessarily acceptable in law.

111. 12.7 Conclusions
Given the size of many data sets, we could propose many hypotheses and soon begin to find that we were rejecting the more conservative null hypotheses and accepting the more radical alternatives.
This does not, in itself, mean that there are differences or that we have identified change or that we have followed good statistical practice.
We need to accept that with a known probability (usually 5%), we can reject the null hypothesis when it is correct - Type I error.

112. 12.7 Conclusions
We also need to recognize that if we keep looking for relationships we will eventually find some.
Testing is not merely about meeting the technical requirements of the test but constructing hypotheses that are meaningful in the problem context.
In an ideal world, before the data or evidence was collected, the problem context would be clarified and meaningful hypotheses specified.

113. 12.7 Conclusions
It is true that we can discover the unexpected from data and identify important and previously unknown relationships.
The process of enquiry or research is therefore complex, requiring a balance between the testing of ideas we already have and allowing new ideas to emerge.
It should also be noted that a statistically significant result may not have business significance.
It could be that statistically significant differences between regions, for example, were acceptable because of regional differences in product promotion and competitor activity.

114. 12.7 Conclusions
Since we are inevitably dealing with sample data when we conduct tests of significance, we can never be 100% sure of our answer - remember Type I and Type II error (and other types of error).
What significance testing does do is provide a way of quantifying error and informing judgement.