Development of a Survey Sampling Theory
The report of the commission presented to the ISI Congress in 1925 marked the official recognition of the use of survey sampling. Most of the basic problems had already been posed, such as the use of random samples and the calculation of the variance of the estimators for simple and stratified designs. The acceptance of the use of partial data, and especially the recommendation to use random designs, led to a rapid mathematization of this theory. At that time, the calculation of probabilities was already known. In addition, statisticians had already developed a theory for experimental statistics. Everything was in place for the rapid progress of a fertile field of research: the construction of a statistical theory of survey sampling.
Jerzy Neyman (1894–1981) developed a large part of the foundations of the probabilistic theory of sampling for simple, stratified, and cluster designs. He also determined the optimal allocation of a stratified design. The optimal allocation method challenges the basic idea of the quota method, which is the “representativeness”. Indeed, depending on the optimal stratification, the sample should not be a miniature of the population as some strata must be overrepresented. The article published by Neyman (1934) in the Journal of the Royal Statistical Society is currently considered one of the founding texts of sampling theory. Neyman identified the main fields of research and his work was to have a very important impact in later years. We now know that Tschuprow (1923) had already obtained some of the results that were attributed to Neyman, but the latter seems to have found them independently of Tschuprow. It is not surprising that such a discovery was made simultaneously in several places. From the moment that the use of random samples was considered a valid method, the theory would arise directly from the application of the theory of probability.
1.5 The US Elections of 1936
During the same period, the implementation of the quota method contributed much more to the development of the use of survey sampling methods than theoretical studies. The 1936 US election marked an important turning point in the handling of questionnaire surveys. The facts can be summarized as follows. The major American newspapers used to publish, before the elections, the results of empirical surveys produced from large samples (two million people polled for the Literary Digest) but without any method to select individuals. While most polls predicted Landon's victory, Roosevelt was elected. Surveys conducted by Crossley, Roper, and Gallup on smaller samples but using the quota method gave a correct prediction. This event helped to confirm the validity of the data provided by opinion polls.
This event, which favored the increase in the practice of sample surveys, was made without reference to the probabilistic theory that had already been developed. The method of Crossley, Roper, and Gallup is indeed not probabilistic but empirical, therefore validation of the adequacy of the method is experimental and absolutely not mathematical.
1.6 The Statistical Theory of Survey Sampling
The establishment of a new scientific consensus in 1925 and the identification of major lines of research in the following years led to a very rapid development of survey theory. During the Second World War, research continued in the United States. Important contributions are due to Deming & Stephan (1940), Stephan (1942, 1945, 1948) and Deming (1948, 1950, 1960), especially on the question of adjusting statistical tables to census data. Cornfield (1944) proposed using indicator variables for the presence of units in the sample. Cochran (1939, 1942, 1946, 1961) and Hansen & Hurwitz (1943, 1949) showed the interest of unequal probability sampling with replacement. Madow (1949) proposed unequal probability systematic sampling (see also Hansen et al., 1953a,b). This is quickly established that an unequal probability sampling with fixed size without replacement is a complex problem. Narain (1951), Horvitz & Thompson (1952), Sen (1953), and Yates & Grundy (1953) presented several methods with unequal probabilities in two articles that are certainly among the most cited in this field. Devoted to the examination of several designs with unequal probabilities, these texts are mentioned for the general estimator (expansion estimator) of the total, which is also proposed and discussed. The expansion estimator is, in fact, an unbiased general estimator applicable to any sampling design without replacement. However, the proposed estimator of variance has a default. Yates & Grundy (1953) showed that the variance estimator proposed by Horvitz and Thompson can be negative. They proposed a valid variant when the sample is of fixed sample size and gives sufficient conditions for it to be positive. As early as the 1950s, the problem of sampling with unequal probabilities attracted considerable interest, which was reflected in the publication of more than 200 articles. Before turning to rank statistics, Hájek (1981) discussed the problem in detail. A book of synthesis by Brewer & Hanif (1983) was devoted entirely to this subject, which seems far from exhausted, as evidenced by regular publications.
The theory of survey sampling, which makes abundant use of the calculation of probabilities, attracted the attention of university statisticians and very quickly they reviewed all aspects of this theory that have a mathematical interest. A coherent mathematical theory of survey sampling was constructed. The statisticians very quickly came up against a difficult problem: surveys with finite populations. The proposed model postulated the identifiability of the units. This component of the model makes irrelevant the application of the reduction by sufficiency and the maximum likelihood method. Godambe (1955) states that there is no optimal linear estimator. This result is one of the many pieces of evidence showing the impossibility of defining optimal estimation procedures for general sampling designs in finite populations. Next, Basu (1969) and Basu & Ghosh (1967) demonstrated that the reduction by sufficiency is limited to the suppression of the information concerning the multiplicity of the units and therefore of the nonoperationality of this method. Several approaches were examined, including one from the theory of the decision. New properties, such as hyperadmissibility (see Hanurav, 1968), are defined for estimators applicable in finite populations.
A purely theoretical school of survey sampling developed rapidly. This theory attracted the attention of researchers specializing in mathematical statistics, such as Debabrata Basu, who was interested in the specifics of the theory of survey sampling. However, many of the proposed results were theorems of the nonexistence of optimal solutions. Research on the question of the foundations of inference in survey theory was becoming so important that it was the subject of a symposium in Waterloo, Canada, in 1971. At this symposium, the intervention of Calyampudi Radhakrishna Rao (1971, p. 178), began with a very pessimistic statement:
I may mention that in statistical methodology, the existence of uniformly optimum procedures (such as UMV unbiased estimator, uniformly most powerful critical region for testing a hypothesis) is a rare exception rather than a rule. That is the reason why ad hoc criteria are introduced to restrict the class of procedures in which an optimum may be sought. It is not surprising that the same situation is obtained in sampling for a finite situation. However, it presents some further complications which do not seem to exist for sampling from infinite populations.
This introduction announced the direction of current research.
In survey sampling theory, there is no theorem showing the optimality of an estimation procedure for general sampling designs. Optimal estimation methods can only be found by restricting them to particular classes of procedures. Even if one limits oneself to a particular class of estimators (such as the class of linear or unbiased estimators), it is not possible to obtain interesting results. One possible way out of this impasse is to change the formalization of the problem, for example by assuming that the population itself is random.
1.7 Modeling the Population
The absence of tangible general results concerning certain classes of estimators led to the development of population modeling by means of a model called “superpopulation”. In this model‐based approach, it is assumed that the values taken by the variable of interest on the observation units of the population are the realizations of random
