necessary to test a large number of people by means of a comprehension test. This test should contain multiple texts and different questions for each of them. In general, the material used in an experiment is defined as a set of items (the texts or the questions in our example are items). The ideal number of participants, as well as the number of items necessary to undertake proper research, is a complex question, which we will address in Chapter 6.
Furthermore, experimental research is generally carried out by recruiting naive participants, who ignore the goals of the experiment and who have zero expertise in the subject under study. This precaution aims to try to control certain cognitive biases that could influence research results. The first bias is related to the fact that the participants who know the research hypothesis may try to base their answers on this hypothesis. Should this happen, the results obtained could suffer from what is called confirmation bias. Rather than answering naturally, participants could provide answers based on the hypothesis to confirm it, not because the assumption is correct, but rather because it seems adequate to them (even if this is not the case). The second bias is related to the fact that participants may want to help the researcher. If the participants know or suspect the goal of an experiment beforehand, the results obtained in this second scenario may not correspond to reality, but rather to the answers that the participants presume are expected.
Finally, in experimental research, participants are generally assigned to conditions in a random manner. This means that every person has the same chances of being included under one condition of the experiment or another. This random assignment offers additional protection against the effect of uncontrolled external variables. In addition to testing a large number of people, randomly distributing them to the different conditions reduces the probability that external variables could systematically influence the results. However, this random assignment is only feasible when all variables are manipulated. When one or more variables are simply observed, participants must be included in one condition or another on the basis of their own characteristics, such as gender or age, for instance. In this case, we speak of quasi-experimental research, since it is not possible to control all the variables. Leaving this question aside, experimental and quasi-experimental research is very similar, and the elements developed in the following chapters apply to both types of research.
1.2.5. Use of statistics and generalization of results
The last essential characteristic of experimental research concerns the way in which data is analyzed. Experimental research aims to collect quantitative data that can be statistically analyzed. As we will see in Chapter 7, quantitative data can be described using different indicators, such as the mean, for example. Based on these descriptive indicators, it is possible to obtain an overview of the data collected, to summarize and illustrate them, in order to communicate the results with simplicity.
At the second stage, data is used to draw conclusions about the research hypotheses. In experimental linguistics, the aim is to study and understand a linguistic phenomenon for a specific population. Since it is impossible to test an entire population, researchers collect data from a representative sample. Through the use of inferential statistics, it is possible to determine whether the results of a particular sample are applicable to the whole population. This process is called generalization.
1.3. Types of experiment in experimental linguistics
Experimental research can be applied to all areas of linguistics, even if historically some areas have used such a methodology more consistently than others. Research questions vary widely between linguistic fields, meaning that many different methods and measures can be used in experimental linguistics. In this book, we do not aim to offer a detailed presentation of every research field and the methods associated with each, but rather to provide an overview of the principles of experimental methodology and the available techniques for linguists. Here, we will introduce some major classes of experiments that can be carried out in linguistics, and we will then develop these in every dedicated chapter.
In general, the experimental studies carried out in linguistics can be classified depending on the aspect of the language under study. Alternately, we will discuss studies on linguistic production and those relating to language comprehension. We will see that the study of comprehension poses many challenges, since this process is not directly observable. For this reason, research on language comprehension is based on the observation of indirect measures, which can be explicit or implicit. We will also see that it is possible to study comprehension by observing different stages of this process, either while it is in progress or once it has been completed.
1.3.1. Studying linguistic productions
The first type of linguistic experiment aims to investigate language production, all the manifestations of language that are produced by individuals in a certain language. Although these manifestations can be collected from diverse corpora and then studied through corpus analysis (see Zufferey (2020) for a detailed presentation of these methods), in some cases, the data contained in the corpus is not enough for studying a linguistic phenomenon. Some rare phenomena practically do not appear, if at all, in a corpus. What is more, the use of observation of naturally produced data is not suitable for showing the influence of a variable on the emergence of a specific linguistic phenomenon, as we have already seen. To counter this, different experiments can be implemented in order to study the production of linguistic phenomena. In these experiments, the goal is to purposefully elicit the emergence of certain linguistic structures, while controlling the context in which such structures appear. The experimental study of linguistic production will be described in further detail in Chapter 3.
1.3.2. Explicit and implicit measures of comprehension
The second type of experiments used in experimental linguistics include studies conducted on the mechanisms involved in language processing and comprehension. Such processes are numerous and range from the organization of the lexicon, to the comprehension of a text or a discourse. It is therefore the most broadly studied aspect in experimental linguistics. Unlike some aspects of the production component, the language comprehension component is unique, in that it cannot be directly assessed through mere observation. It is outright impossible to directly observe the processes involved in the comprehension of a text, for example. This is why it is necessary to find a way to measure these processes indirectly, based on indicators that can be associated with them.
The first way of collecting these indicators requires the use of explicit tasks in which participants have to reflect upon certain linguistic aspects. For example, this is the case for metalinguistic tasks such as grammaticality or acceptability judgments. This type of task could be used to test the participants’ grammatical knowledge, by showing them syntactically correct or incorrect sentences in compliance with grammatical standards, and asking them to identify errors and justify their choice. While these tasks have the advantage of providing direct access to speakers’ knowledge, they also have the defect of being based on their reflexive skills and their subjective appreciation of their own understanding. These tasks are also particularly complex for certain types of people, especially for children or people with language impairments, for whom it is often very difficult to explain the reasoning behind their decision. Other tasks make it possible to circumvent these problems, by setting up experiments in which the participants have to choose between several illustrations matching a linguistic stimulus. For example, Durrleman et al. (2015) tested the comprehension of relative sentences in people with autism spectrum disorder (ASD), asking them to point to the image corresponding to sentences such as “show me the little boy running after the cat”. Making use of such tasks offers the possibility of studying language comprehension in children and populations suffering from linguistic
