involved at the various steps of the process, and the related risks of errors. The flowchart will help the surveyor2 in properly planning a web survey and will help researchers highlight the steps of the survey process for their respective research. Starting from the flowchart presented by Thorsdottir and Biffignandi (Webdatanet Conference, 2015), the flowchart presented in this chapter is slightly different since mobile extension of the web survey is also considered (i.e., mobile web survey). Considering the wide diffusion of mobile devices (especially smartphones), mobile web survey is becoming a very frequent, say, mostly a standard, situation. When a web survey is administered if no disallow is activated, the survey may be received and completed not only using a PC but even from a mobile device. This situation is called mobile web survey, i.e., no prevention toward mobile devices in a web survey (for a discussion of the mobile surveys, see Chapter 2). For a detailed description of the choices for smartphone participation in web surveys, see Peterson et al. (2017). Mobile web survey implies several methodological challenges; nevertheless the survey process requires only a few differences with respect to the web‐only survey.
The flowchart divides the mobile web (or the web‐only survey) process into the following six major steps:
1 Determine the survey objective.
2 Metadata definition.
3 Designing the mobile web or only web survey (deciding on the mode, the sampling frame, and the sampling approach, designing the questionnaire, designing the paradata methodology, and selecting the sample, software, or programming language).
4 Collecting the data.
5 Creating the database.
6 Processing data.
The use of the flowchart brings a total survey error (TSE) perspective, which is an effective approach for understanding error sources in a comprehensive way (Weisberg, 2005). Biemer TSE approach aims at helping researchers make design decisions that maximize data quality within the constraints of a limited budget. TSE perspective disaggregates overall error into component, mainly distinguishing between sampling and non‐sampling errors. The TSE paradigm (see, for example, Platek and Särndal, 2001 and the ensuing discussions) refers to the concept of optimally allocating the available survey resources to minimize TSE for key estimates. In principle, to apply the TSE paradigm, the major sources of error should be identified so that the survey resources can be allocated to reduce their errors, within specified constraints on costs. An overview of the TSE history and recent research results on the relationships between different types of errors is given in Biemer et al. (2017). The idea of the TSE is of optimally balancing the dimensions of survey quality within the survey budget and schedule. It is possible to distinguish between TSE and total survey quality. This considers the fitness for use of an estimate, that is, quality from both the producer and user perspectives. The “fitness for use” concept implies not only accuracy is considered; for instance, attributes such as the timeliness, accessibility, and comparability of the data have to be granted in the survey process, as well as various types of errors (see Chapter 5), such as measurement, nonresponse, and so forth (Groves, 1989).
The message in this chapter is that both TSE and total quality survey perspective embody the need to consider different error sources and that errors occur at every step of the survey process and, sometimes, they are interrelated.
In this chapter, a framework of the web survey process is proposed. The framework's main purpose is to provide a clear overview of the necessary decisions when organizing a mobile web survey or web‐only survey and to create a shared overview of the survey process steps. Practitioners, as well as researchers, can refer to the flowchart and obtain a clear understanding of the procedure to follow, the choices to do, and the type of errors that might occur. Thus, there is a more complete, integrated perspective in studying and interpreting errors. Understanding them becomes more easily.
3.2 Theory
The mobile web survey or the web‐only survey process differs in some respects from a survey based on traditional modes (paper and pencil, telephone, and fax). Errors arising in mobile web surveys and only mobile surveys have some specific characteristics due mainly to the coverage aspects of the target population and to various other possibilities (the availability of auxiliary variables, the time required to deliver the questionnaire and to receive the completed questionnaire, technical skills, the equipment needed, and many other situations). Therefore, a well‐defined outline of the survey research steps and of the errors that might occur at each step is vital for the surveyor. Furthermore, in many cases, the errors at different steps have some relationships, even if the relationship is not clearly defined and formalized. Thus, decisions pertaining to the reduction of an error at a specific step could increase the errors occurring at other steps of the survey process. For instance, increasing response rate by stressing the interviewee with a high number of solicitations might decrease the quality of the data (more item nonresponse, less accuracy, and so forth). Therefore, the overall quality of the survey process could be either improved or deteriorated depending on the decision made.
The flowchart mentioned in the previous section is in Figure 3.1. It shows the main steps and sub‐steps for a web‐only or mobile web survey based on probability sample.
Figure 3.1 Flowchart of a probability‐based web‐only or mobile web survey
The six main steps (gray highlighted) and their related sub‐steps (hell gray and white highlighted) are listed in the previous section. Hereunder, detailed comments of the flowchart steps are reported.
The first step is Determine the survey objectives, and it is a preliminary point of every survey, independent from the mode. Objectives need to be clearly stated and the questions to be identified. Since this flowchart considers web mode (either web only or mobile web), it should be critically evaluated if the objectives and the structure of the survey are adequate for that mode. Web mode implies a self‐completed questionnaire; thus, for example, self‐completed surveys are better in capturing sensitive topics. If the questionnaire is long self‐completion, especially using web mode, it is not in favor of high participation rates and smaller measurement error. This becomes especially critical in mobile web surveys, where the participation situation has a high probability to be disturbed from other activities or tasks the interviewee is doing.
The second step is Metadata definition. Recent literature and empirical research brought significant attention to metadata, showing a set of information describing the survey (that is, elements for evaluating the data quality, like the list and description of the variables, the target population, the sample units, and so forth). The metadata refer to variables and activities occurring in every step of the process. The International Organization for Standardization (ISO) defines metadata as “data describing and defining other data in a certain context.” In such a case, the contest is the condition under which the data collection and processing takes place. The definition of metadata is an important step, recommended for all surveys, independently from the mode. Details to consider in mobile web surveys (and in web‐only surveys) differ from traditional surveys due to the characteristics of various steps and sub‐steps. Metadata are mainly semantic based; a bad or imprecise metadata description affects the quality of the questionnaire and could cause errors in the answers (questions not correctly understood, questions that each interviewed could interpret in a different way). Metadata are useful for the selection
