the treatment and control groups are apart. Positive scores indicate that children in the treatment group fared better than children in the control group and therapy was helpful. Negative scores indicate that children in the control group experienced better outcomes than children in the treatment group and therapy was harmful (Hoyt & Del Re, 2018).
We can combine the results of multiple studies by calculating the weighted average effect size. Studies are weighted based on their number of participants, so large studies influence the average more than smaller studies. As a rule of thumb, effect sizes of .2 or less are considered “small,” .5 are “medium,” and .8 or greater are “large” (Ferguson, 2017).
Examples of Meta-Analysis
Figure 3.4 shows the results of meta-analyses examining the effects of neurofeedback, behavior therapy, and medication on children with ADHD. These meta-analyses combine the results of hundreds of studies involving thousands of children with this disorder. Consequently, the results can be used to make decisions regarding which form of treatment is most likely to help youths with ADHD (Cortese et al., 2017; Fabiano et al., 2010; Faraone & Buitelaar, 2010).
Figure 3.4 ■ Meta-Analysis for the Treatment of ADHD in Children
Note: Meta-analysis is used to combine the results of many studies into a single effect size. This meta-analysis shows the effects of various treatments for ADHD compared to placebo. Whereas neurofeedback has a small effect on children’s symptoms, behavior therapy and medication have medium to large effects.
The bars in Figure 3.4 show the average weighted effect size for each form of treatment compared to placebo. Overall, neurofeedback has a small (and nonsignificant) effect on ADHD symptoms, behavior therapy has a medium effect, and medication has a large effect. Consequently, evidence-based practice indicates that clinicians should use behavior therapy and/or medication to treat children with ADHD, because these treatments are most likely to help (Evans et al., 2019).
Review
Meta-analysis is a statistical technique that researchers use to combine the results of many studies into a single, numerical result.
Meta-analysis yields an effect size (ES) that tells us how much of an effect the treatment had on children’s outcomes. It often reflects the number of standard deviations the treatment and control groups are apart at the end of the study.
As a general rule, effect sizes of .2 or less are small, .5 are medium, and .8 or greater are large.
What Are Quasi-Experimental Studies?
Experiments allow us to infer causal relationships between variables because participants are randomly assigned. Sometimes, however, researchers are not able to conduct true experiments because random assignment is not possible. Instead, researchers conduct quasi-experiments. In a quasi-experimental study, researchers manipulate an independent variable (e.g., provide treatment) and note changes in a dependent variable (e.g., children’s outcome). However, they do not randomly assign participants to different groups, so they cannot infer that the treatment caused those outcomes. The term “quasi” means “looks like.” A quasi-experimental study looks like a true experiment, but it lacks an experiment’s essential ingredient: random assignment.
Let’s look at three of the most common types of quasi-experimental studies used in the field of abnormal child psychology: pretest-posttest studies, nonequivalent groups studies, and single case studies.
Pretest-Posttest Studies
A pretest-posttest study is a quasi-experimental study in which the same group of participants is measured at least twice: at baseline (before treatment) and at the end of the study (after treatment). Because all participants receive treatment, there is no control group and, therefore, random assignment is not possible.
For example, researchers conducted a pretest-posttest study investigating the effects of stimulant medication on children’s ADHD symptoms. They administered stimulant medication to a large sample of children with ADHD for approximately 12 weeks. To assess outcomes, they asked clinicians, parents, and teachers to rate children’s ADHD symptoms at the beginning and end of the study. Overall, 75% of children showed a significant decrease in symptoms (Döpfner, Görtz-Dorten, Breuer, & Rothenberger, 2011).
Because the study lacked a control group, the researchers could not conclude that the medication caused this reduction in children’s symptoms. It is possible that other factors, besides medication, might better explain the results of this study. Internal validity refers to the degree to which we can say that manipulation of an independent variable (e.g., treatment) causes a corresponding change in a dependent variable (e.g., children’s outcomes). When other factors can explain children’s outcomes, researchers say these factors threaten the internal validity of the study. There are several threats to internal validity that limit the causal inferences we can make from quasi-experimental research (Kazdin, 2017).
First, maturation can compromise the internal validity of a study. Maturation refers to changes in the child that occur because of the passage of time. For example, as children’s brains mature, they show greater capacity for attention, concentration, and impulse control. It is possible that all children, even those who do not receive treatment, will show a reduction in ADHD symptoms simply due to this brain maturation. Unless researchers compare children who receive treatment with children in a control group who do not, the effects of treatment cannot be distinguished from maturation alone.
Second, environmental factors can threaten the internal validity of pretest-posttest studies. Environmental factors include changes in the child’s family (e.g., divorce), school (e.g., a new teacher), or peer group (e.g., best friend moves away). Environmental factors also include major events (e.g., an economic downturn, the COVID-19 pandemic) or more subtle changes in the child’s surroundings. For example, if researchers assessed children’s symptoms at pretest during the school year and at posttest during the summer, parents might report fewer attention problems over time. However, this apparent improvement in attention might be explained by the fact that inattention is less problematic during summer vacation than during the academic year. Without a control group for comparison, it is possible that these environmental changes might explain some of the study’s results.
A third threat to internal validity is repeated testing. The act of repeatedly assessing children can cause them to show improvement over time. For example, if children know that their parents and teachers are monitoring their behavior, they might try to act more attentive or obedient. Similarly, if parents and teachers are repeatedly asked to rate children’s behavior, they might pay more attention to signs of improvement. Without a comparison group, it is possible that some of the benefits of treatment are simply due to the fact that children were monitored and tested multiple times.
Fourth, attrition can threaten the internal validity of a study. Attrition refers to the loss of participants over the course of the study. Attrition usually occurs because participants decide to withdraw from the study or simply stop attending treatment sessions. When a large percentage of participants in the treatment group withdraw from the study, it threatens the study’s internal validity. For example, Döpfner and colleagues (2011) found that 75% of children who completed their study
