useful property of MLE is the invariance property. In general, let Then based on the invariance property, the MLE of the standard deviation √σjj is
The MLE has some good asymptotic properties and usually performs well for data sets of large sample sizes. For example, under mild regularity conditions, MLE satisfies the property of consistency, which guarantees that the estimator converges to the true value of the parameter as the sample size becomes infinite. In addition, under certain regularity conditions, the MLE is asymptotically normal and efficient. That is, as the sample size becomes infinite, the distribution of MLE will converge to a normal distribution with variance equal to the optimal asymptotic variance. The details of the regularity conditions are beyond the scope of this book. But these conditions are quite general and often satisfied in common circumstances.
3.4 Hypothesis Testing on Mean Vectors
In this section, we study how to determine if the population mean μ is equal to a specific value μ0 when the observations follow a normal distribution. We start by reviewing the hypothesis testing results for univariate data. Suppose X1, X2,…, Xn are a random sample of independent univariate observations following the normal distribution N(μ, σ2). The test on μ is formulated as
where H0 is the null hypothesis and H1 is the (two-sided) alternative hypothesis. For this test, we use the following test statistic:
where X̄ is the sample mean
The test based on a fixed significance level α, say α = 0.05, has the disadvantage that it gives the decision maker no idea about whether the observed value of the test statistic is just barely in the rejection region or if it is far into the region. Instead, the p-value can be used to indicate how strong the evidence is in rejecting the null hypothesis H0. The p-value is the probability that the test statistic will take on a value that is at least as extreme as the observed value when the null hypothesis is true. The smaller the p-value, the stronger the evidence we have in rejecting H0. If the p-value is smaller than α, H0 will be rejected at the significance level of α. The p-value based on the t statistic in (3.18) can be found as
where T(n − 1) denotes a random variable following a t distribution with n − 1 degrees of freedom.
We can define the 100(1 − α)% confidence interval for μ as
It is easy to see that the null hypothesis H0 is not rejected at level α if and only if μ0 is in the 100(1 − α)% confidence interval for μ. So the confidence interval consists of all those “plausible” values of μ0 that would not be rejected by the test of H0 at level α.
To see the link to the test statistic used for a multivariate normal distribution, we consider an equivalent rule to reject H0, which is based on the square of the t statistic:
We reject H0 at significance level α if t2>(tα/2,n−1)2.
For a multivariate distribution with unknown mean μ and known Σ, we consider testing the following hypotheses:
