of the relationship between the area under the normal curve and the standard deviation."/>
Figure 1.27 Relationship between the area under the normal curve and the standard deviation.
1.17 Probability Distribution of Sample Means
The reason for the pattern of variation of sample means observed in Section 1.13 can easily be understood. We know that a mean is calculated by summing a number of observations on a variable and dividing the result by the number of observations. Normally, we look at the values of an attribute as observations from a single variable. However, we could also view each single value as an observation from a distinct variable, with all variables having an identical distribution. For example, suppose we have a sample of size 100. We can think that we have 100 independent observations from a single random variable, or we can think that we have single observations on 100 variables, all of them with identical distribution. This is illustrated in Figure 1.28. What do we have there, one observation on a single variable – the value of a throw of six dice – or one observation on each of six identically distributed variables – the value of the throw of one dice? Either way we look at it we are right.
So what would be the consequences of that change of perspective? With this point of view, a sample mean would correspond to the sum of a large number of observations from variables with identical distribution, each observation being divided by a constant amount which is the sample size. Under these circumstances, the central limit theorem applies and, therefore, we must conclude that the sample means have a normal distribution, regardless of the distribution of the attribute being studied.
Because the normal distribution of sample means is a consequence of the central limit theorem, certain restrictions apply. According to the theorem, this result is valid only under two conditions. First, there must be a large number of variables. Second, the variables must be mutually independent. Transposing these restrictions to the case of sample means, this implies that a normal distribution can be expected only if there is a large number of observations, and if the observations are mutually independent.
In the case of small samples, however, the means will also have a normal distribution provided the attribute has a normal distribution. This is not because of the central limit theorem, but because of the properties of the normal distribution. If the means are sums of observations on identical normally distributed variables, then the sample means have a normal distribution whatever the number of observations, that is, the sample size.
Figure 1.28 The total obtained from the throw of six dice may be seen as the sum of observations on six identically distributed variables.
1.18 The Standard Error of the Mean
We now know that the means of large samples may be defined as observations from a random variable with normal distribution. We also know that the normal distribution is completely characterized by its mean and variance. The next step in the investigation of sampling distributions, therefore, must be to find out whether the mean and variance of the distribution of sample means can be determined.
We can conduct an experiment simulating a sampling procedure. With the help of the random number generator of a computer, we can create a random variable with normal distribution with mean 0 and variance 1. Incidentally, this is called a standard normal variable. Then, we obtain a large number of random samples of size 4 and calculate the means of those samples. Next, we calculate the mean and standard deviation of the sample means. We repeat the procedure with samples of size 9, 16, and 25. The results of the experiment are shown in Figure 1.29.
As expected, as the variable we used had a normal distribution, the sample means also have a normal distribution. We can see that the average value of the sample means is, in all cases, the same value as the population mean, that is, 0. However, the standard deviations of the values of sample means are not the same in all four runs of the experiment. In samples of size 4 the standard error is 0.50, in samples of size 9 it is 0.33, in samples of size 16 it is 0.25, and in samples of size 25 it is 0.20.
Figure 1.29 Distribution of sample means of different sample sizes.
If we look more closely at these results, we realize that those values have something in common. Thus, 0.50 is 1 divided by 2, 0.33 is 1 divided by 3, 0.25 is 1 divided by 4, and 0.20 is 1 divided by 5. Now, can you see the relation between the divisors and the sample size, that is, 2 and 4, 3 and 9, 4 and 16, 5 and 25? The divisors are the square root of the sample size and 1 is the value of the population standard deviation. This means that the standard deviation of the sample means is equal to the population standard deviation divided by the square root of the sample size. Therefore, there is a fixed relationship between the standard deviation of the sample means of an attribute and the standard deviation of that attribute, where the former is equal to the latter divided by the square root of the sample size.
In the next section we will present an explanation for this relationship, but for now let us consolidate some of the concepts we have discussed so far.
The standard deviation of the sample means has its own name of standard error of the mean or, simply, standard error. If the standard error is equal to the population standard deviation divided by the square root of the sample size, then the variance of the sample means is equal to the population variance divided by the sample size.
Now we can begin to see why people tend to get confused with statistics. We have been talking about different means and different standard deviations, and students often become disoriented with so many measures. Let us review the meaning of each one of those measures.
There is the sample mean, which is not equal in value to the population mean. Sample means have a probability distribution whose mean has the same value as the population mean.
There is a statistical notation to represent the quantities we have encountered so far, whereby population parameters, which are constants and have unknown values, are represented by Greek characters; statistics obtained from samples, which are variables with known value, are represented by Latin characters. Therefore, the value of the sample mean is represented by the letter m, and the value of the population mean by the letter μ (“m” in the Greek alphabet). Following the same rule for proportions, the symbol for the sample proportion is p and for the population proportion is π (“p” in the Greek alphabet).
Next, there is the sample standard deviation, which is not equal in value to the population standard deviation. Sample means have a distribution whose standard deviation, also known as standard error, is different from the sample standard deviation and from the population standard deviation. The usual notation for the sample standard deviation is the letter s, and for the population standard deviation is the letter σ (“s” in the Greek alphabet). There is no specific
