Alternatively, methods for studying comprehension in an implicit manner (without asking the participants directly for a judgment or an explanation of their reasoning) have also been developed. This is the case in action tasks, in which some kinds of behavior adopted on the basis of a linguistic stimulus can be observed. For example, Pouscoulous et al. (2007) tested the understanding of scalar implicatures triggered by words such as quelques (roughly equivalent to some), by asking French-speaking children to arrange tokens in boxes so as to match statements like “quelques cases ont des jetons (some boxes have tokens)”. It is also possible to understand comprehension skills using recall or recognition tasks, in which questions are asked at the end of a reading exercise or after listening to a text or speech fragment. For example, Zufferey et al. (2015a) tested the comprehension of causal relations in children aged 5–8 years, by asking them to answer why questions after every page, when reading a story with them.
1.3.3. Offline and online measures of comprehension
The various tasks listed above, as well as the tasks proposed in the examples presented so far in this chapter, enable access to comprehension once the word, sentence or text has been processed and understood. These measures are described as offline, in that they affect the final interpretations resulting from the comprehension process. On the other hand, online measures allow us to study the processes that come into play in comprehension itself. Such processes have the characteristic of being extremely fast, transient and occurring out of people’s consciousness, therefore remaining inaccessible to traditional offline measures.
Borrowing scientific methods and paradigms from other disciplines, such as psychology, has allowed the study of online processes involved in language comprehension. The majority of online measurement techniques have something in common: they observe the time required for a process, by measuring the reading time or reaction time. These techniques are based on the idea that the time required to complete a process reflects certain characteristics of this process, particularly in terms of complexity. Longer reaction times and reading times are generally associated with a more in-depth processing of the linguistic stimulus. Tasks using these time measures typically involve asking participants to name words, read or produce sentences, or decide whether or not a series of letters matches a word in their language. Studies that have employed such tasks have shown that, at the word level, response times and reading times are influenced by properties such as frequency, length and predictability. Similarly, at the sentence level, reading is influenced by properties such as syntax complexity or the need to produce inferences (Just and Carpenter 1980; Rayner 1998; Smith and Levy 2013).
Studies based on time measures have benefited from significant technological developments since the 1970s, so that today, anyone can easily conduct research from their computer. In addition, new techniques have been developed to enable the recording of eye movement whilst reading or when observing an image. It is thus possible to gain an insight, not only into the time required to read certain words or sentences, but also the exact movements made by the eyes during reading. This data provides additional information, such as the time allotted for different words, the order in which words are fixated or even the eye movements associated with reading certain passages. These eye movement measures can be applied to the study of reading as well as to the study of spoken speech production or comprehension.
Finally, the methods used in the field of neuroscience have also been transferred to experimental linguistics. These methods provide access to the brain activity involved in language-related processes. Using small electrodes placed on the scalp, the electroencephalogram (EEG) records the activity of neurons on the surface of the brain. This technique gives an accurate temporal overview of the activity of neurons associated with a specific linguistic process. Functional magnetic resonance imaging (fMRI) aims to measure the activity of neurons based on their oxygen consumption. It thus provides a precise spatial overview of the brain areas involved in a specific linguistic process.
As we can infer by reading these lines, offline methods are the most accessible to researchers, since they require few technical means. In most cases, offline measures can be collected using paper and pencil tasks. A simple spreadsheet available on every computer can be used for organizing and analyzing the data from such studies. For some statistical tests, a program must be added to the list of necessary tools. Online methods for observing reaction time or reading performance require special software for programming experiments. Things get more complicated when you want to record eye movements. These recordings require the use of expensive tools, that also take time to control. Furthermore, the data from studies on eye movements is much more complex to process. Finally, EEG or fMRI studies are generally reserved for people benefiting from access to such techniques, which are extremely costly in terms of equipment and necessary skills for processing recorded signals. For this reason, such techniques will not be discussed in this book.
Finally, we should point out that the offline and online measures do not provide answers to the same type of research questions. It is therefore important to consider them as complementary measures, which shed different light on the same phenomenon. There are no good or bad measures in experimental linguistics; the choice must be made on the basis of the goals and hypotheses of the research project. More and more often, offline and online measurements are used in parallel in the same study. We will return to these measures, their specific characteristics and the means for combining them, in detail, in Chapters 4 and 5.
1.3.4. Research designs and experimental designs
Whether for the purpose of studying production or comprehension, research can be categorized according to the general framework in which data collection takes place or, in other words, the experimental design. On the one hand, there are longitudinal designs, in which the same subjects are observed on several occasions, following varying time intervals. This type of design is generally used in studies where a variable cannot be manipulated, but its effect can be observed through time. For example, to study the influence of age on the ability to distinguish sounds between the different languages spoken in the environment of babies growing up in bilingual homes, one possibility would be to test the same bilingual babies at 2 months, 4 months, then 8 months old. Another example of longitudinal design would be the study of the relationship between language development and the development of theory of mind. In this case, language skills and individual differences in theory of mind could be measured in children aged 3 and a half, 4, and 4 and a half, for example.
The major interest of longitudinal studies is that they make it possible to observe changes in real time. However, they also have two significant disadvantages. First, such studies imply that participants must be tested on several occasions in relatively short periods of time. It is thus inevitable to lose participants during the study, due to motivation and availability reasons. Secondly, these studies generate significant costs, since it is necessary to find and then test people repeatedly, and above all, keep in touch with them and convince them to return to the following test sessions.
In order to work around these problems, cross-sectional designs observe different people, who are subjected to different conditions. To use the example of bilingual babies, instead of testing the same babies at different ages, we could simultaneously test groups of babies of different ages. This method would imply making a sort of picture of a situation at a given moment, which would offer indications on the relationship between age and sound perception in bilingual babies. Cross-sectional designs are typically used in quasi-experiments, where the independent variable is not manipulated.
When the independent variable can be manipulated, it is possible to allocate the participants to different conditions, in which manipulation can either be present or absent. Two types of experimental designs can be constructed in this case. In the first,
