Gallup correctly predicted Franklin Roosevelt to be the new president, whereas Literary Digest predicted that Alf Landon would beat Franklin Roosevelt. The prediction based on the very large sample size turned out to be wrong. The explanation was that the sampling technique of Literary Digest did not produce representative samples. In the 1930s, cars and telephones were typically owned by middle‐ and upper‐class people. These people tended to vote Republican, whereas lower‐class people were more inclined to vote Democrat. Consequently, Republicans were overrepresented in the Literary Digest sample.
As a result of this historic mistake, opinion researchers learned that they should rely on more scientific ways of sample selection. They also learned that the way a sample is selected is more important than the size of the sample.
The classical theory of survey sampling was more or less completed in 1952. Horvitz and Thompson (1952) developed a general theory for constructing unbiased estimates. Whatever the selection probabilities are, as long as they are known and positive, it is always possible to construct a useful estimate. Horvitz and Thompson completed the classical theory, and the random sampling approach was almost unanimously accepted. Most of the classical books about sampling were also published by then (Cochran, 1953; Deming, 1950; Hansen, Hurvitz, and Madow, 1953; Yates, 1949).
1.2.2 TRADITIONAL DATA COLLECTION
There were three modes of data collection in the early days of survey research: face‐to‐face interviewing, mail interviewing, and telephone interviewing. Each mode had its advantages and disadvantages.
Face‐to‐face interviewing was already used for the first censuses. Thus, it is not a surprise it was also used for surveys. Face‐to‐face interviewing means that interviewers visit the persons selected in the sample. Well‐trained interviewers will be successful in persuading reluctant persons to participate in the survey. Therefore, response rates of face‐to‐face surveys are usually higher than surveys not involving interviewers (for example, mail surveys). Interviewers can also assist respondents in giving the right answers to the questions. This often results in better data. However, the presence of interviewers can also be a drawback. Research suggests that respondents are more inclined to answer sensitive questions properly if there are no interviewers present.
Survey agencies often send a letter announcing the visit of the interviewer. Such a letter can also give additional information about the survey, explain why it is important to participate, and assure that the collected information is treated confidentially. As a result, the respondents are not taken by surprise by the interviewers.
The response rate of a face‐to‐face survey is usually high and so is quality of the collected data. But a price has to be paid literally: face‐to‐face interviewing is much more expensive. A team of interviewers has to be trained and paid. They also have to travel, which costs time and money.
Mail interviewing is much less expensive than face‐to‐face interviewing. Paper questionnaires are sent by mail to persons selected in the sample. They are invited to answer the questions and to return the completed questionnaire to the survey agency. A mail survey is not interviewers based. Therefore, it is a cheaper mode of data collection than face‐to‐face survey. Data collection involve mailing costs (letters, postage, envelopes) both for sending the questionnaire and for delivering the questionnaire back; similar costs have to be considered for each reminder. Therefore, costs for stamps and questionnaire printing could be not completely irrelevant. Another advantage of mail survey is that the absence of interviewers can be experienced as less threatening for potential respondents. Therefore, respondents are more inclined to answer sensitive questions properly.
The absence of interviewers also has a number of disadvantages. There are no interviewers to explain questions or assist respondents in answering them. This may cause respondents to misinterpret questions, which has a negative impact on the quality of the collected data. Furthermore, it is not possible to use show cards. A show card is typically used for answering closed questions. Such a card contains the list of all possible answers to a question. Respondents can read through the list at their own pace and select the answer corresponding to their situation or opinion. Mail surveys put high demands on the design of the paper questionnaire. For example, it should be clear to all respondents how to navigate through the questionnaire and how to answer questions.
Since the persuasive power of the interviewers is absent, response rates of mail surveys tend to be low. Of course, reminder letters can be sent, but this is often not very successful to let the response rate become very high. More often survey questionnaire forms end up in the pile of old newspapers.
In summary, the costs of a mail survey are relatively low, but often a price has to be paid in terms of data quality: response rates tend to be low, and also the quality of the collected data is also often not very good. Dillman (2007) believes, however, that good results can be obtained by applying his Tailored Design Method. This is a set of guidelines for designing and formatting mail survey questionnaires. They pay attention to all aspects of the survey process that may affect response rates or data quality.
Face‐to‐face interviewing was preferred in the early days of survey interviewing in the Netherlands. The idea was in the 1940s that poor people had poor writing skills and they were not interested in the topics of the surveys. Therefore, they had a lower probability to complete mail questionnaires. People completing and returning questionnaire forms were assumed to be more interested in the survey topics because their intelligence and socioeconomic position was above average.
A third mode of data collection is telephone interviewing. Interviewers are needed to conduct a telephone survey, but not as many as for a face‐to‐face survey, since they do not have to travel from one respondent to the next. They can remain in the call center of the survey agency and conduct more interviews in the same amount of time. Therefore, interviewer costs are less high. An advantage of telephone interviewing over face‐to‐face interviewing is that respondents may be more inclined to answer sensitive questions, because the interviewer is not present in the room. A drawback in the first days of telephone surveys may be that telephone coverage in the population was low. Not every respondent could be contacted by telephone.
Telephone interviewing has some limitations. Interviews cannot last too long, and question answer may not be written. Obviously, no show cards can be used; lists can be presented by reading them out loud (by the interviewers).
This implies a possible recency effect in the answers. Another problem may be the lack of a proper sampling frame. Telephone directories may suffer from severe under‐coverage because many people do not want their phone number to be listed in the directory. Another new development is that increasingly people replace their landline phone by a mobile phone. This fact increases under‐coverage in the telephone directories. For example, according to Cobben and Bethlehem (2005), only between 60% and 70% of the Dutch population can be reached through a telephone dictionary. However it has to be advised that mobile phone numbers are not listed in directories in many countries. Thus, a problem arises.
Example 1.2 is about the first telephone survey in the Netherlands.
EXAMPLE 1.2 The first telephone survey in the Netherlands
The first telephone survey took place in the Netherlands on June 11, 1946. See NIPO (1946) for a detailed description. An interview to a few hundred owners of telephones in Amsterdam asked a few questions about listening to the radio. The call to the people was between 20:00 and 21:30 hours on a Tuesday night. Some results are in Table 1.1.
