where fi,(xi) is the probability density function of Xi if Xi is continuous and pi(Xi is the probability mass function of Xi if Xi is discrete. The µi is also called the population mean of Xi because it is the mean of Xi over all possible values in the population. Similarly, the mean vector µ is the population mean vector of X.
To further explain the relationship and difference between the population mean and the sample mean introduced in Section 2.2, we first consider a univariate random variable X and its population mean μ. Consider a random sample of observations from the population, say, X1, X2,…, Xn. The sample mean is a random variable because the observations X1, X2,…, Xn are all random variables with values varying from sample to sample. For example, let X represent the measured intensity of the current of a wafer produced by a semiconductor manufacturing process. Then we take a random sample of n = 10 wafers from this process and compute the sample mean of the measured intensities of the current and get the result x̄ = 1.02. Now we repeat this process, taking a second sample of n = 10 wafers from the same process and the resulting sample mean is 1.04. The sample means differ from sample to sample because they are random variables. Consequently, the sample mean, and any other function of the random observations, is a random variable. On the other hand, the population mean µ does not depend on the samples and is a (usually unknown) constant. When we take a sample with very large sample size n, the sample mean will be very close to the population mean µ with high probability. As the sample mean X̄ is a random variable, we can evaluate its mean and variance. It is easy to see that E(X̄) = µ and var (X̄) = σ2/n, where β2 is the variance of X. An estimator of a parameter is called unbiased if its mean is equal to the true value of the parameter. X̄ is a commonly used estimator of µ because it is unbiased and has a smaller variance for a larger sample size n.
This concept can be extended to a p-dimensional random vector X with mean vector µ. Consider a random sample X1, X2,…, Xn from the population of X. The sample mean vector X̄ is a random vector with population mean E(X̄) = µ and population covariance matrix , where Σ is the population covariance matrix of X. The population covariance matrix is defined shortly. The sample mean vector X̄ is an unbiased estimator of the population mean vector μ.
The (population) covariance matrix of a random vector X is defined as
The ith diagonal element of Σ is the population variance of Xi:
The (j,k)th off-diagonal element of Σ is the population covariance between Xj and Xk:
where fjk(xj, xk) and pjk(xj, xk) are the joint density function and joint probability mass function, respectively, of Xj and Xk. The population covariance measures the linear association between the two random variables. It is clear that σi = σkj and the covariance matrix Σ is symmetric. The same as the sample covariance matrix, the population covariance matrix Σ is always positive semidefinite.
