Two well-known examples of qualitative studies in composition studies are Janet Emig’s case study The Composing Processes of Twelfth Graders and Shirley Brice Heath’s ethnography Ways with Words.
Quantitative research differs from qualitative research in that it typically contrasts a treatment and a control group to test the validity of a hypothesis. As a simple example, a teacher may teach one classroom of students as she typically does, while she teaches a different classroom of students using an experimental pedagogy, the “treatment” that she is testing. While carefully controlling for possible interferences to the study, known as threats to validity, the researcher collects data from both groups of students to determine whether the new pedagogy has a significant effect on students’ performance. Findings in quantitative studies are typically reported numerically (rather than descriptively, as in qualitative studies) and often depend on rigorous statistical analyses. Two journals known for publishing reports of quantitative research in composition studies are Research in the Teaching of English and Written Communication. Yet another form of quantitative research is a meta-analysis, which is a study that selects from many prior quantitative studies according to carefully chosen criteria and then statistically consolidates their findings. The most well-known meta-analysis in composition studies is George Hillocks’ Research on Written Composition.
Examples of Empirical Research
For examples of how empirical research contributes to knowledge in composition studies, let us return to the topic of writing across the curriculum. WAC has been the subject of both qualitative and quantitative empirical research studies, and the published results of these studies extend the knowledge about WAC beyond what can be learned from scholarship. Much of the empirical research about WAC is qualitative. Beaufort’s article “Developmental Gains of a History Major: A Case for Building a Theory of Disciplinary Writing Expertise” (2004) is just one example of several case studies that have been undertaken to identify how an individual student learned the complexities of writing in a particular discipline. Other qualitative studies about WAC have focused not on student learning but on faculty development, such as Walvoord et al.’s In the Long Run: A Study of Faculty in Three Writing-Across-the-Curriculum Programs (1997), which examines the impact of WAC on several faculties’ teaching philosophies and attitudes, teaching strategies, and career patterns. A characteristic shared by many of the qualitative studies about WAC is that they are longitudinal studies, i.e., studies in which data is collected over multiple years.
Although quantitative research studies about WAC are less common, some are available. One example is Beason’s article “Feedback and Revision in Writing Across the Curriculum Classes,” published in Research in the Teaching of English in 1993. This study did not entail a treatment and control group; instead, a total of twenty students were randomly selected from writing classes in four disciplines, and the first and final drafts of these students’ multi-draft writing assignments were analyzed by multiple raters, who coded the feedback students received on their drafts and the revisions that students made. All of the data was then quantified so that precise conclusions could be drawn about the differences in teacher and student feedback on drafts, as well as the relationship between comments and the revisions that students made. Beason also compared the data from this study of feedback and revision in WAC courses to data from other studies about feedback and revision in traditional composition courses.
Beason explains in his article that he chose a quantitative design for his study because of its unique potential to contribute to the discipline’s knowledge about WAC, especially in contrast to the many qualitative studies about WAC that were already available:
Although [prior qualitative WAC studies] are insightful studies, a focused quantitative approach (besides helping create a needed balance in WAC research) allows a researcher to isolate and scrutinize selected phenomena that are affected by many classroom factors but that can still be singled out and examined in and of themselves. Coding such phenomena provides, moreover, a sense of order for complex behaviors and products that seem to be without patterns . . . (406)
He further comments that his study contributes not only to knowledge about WAC but also to knowledge about feedback on writing and revision. Because quantitative studies often analyze data about multiple variables, it is common for a single quantitative study to contribute to the discipline’s knowledge about more than one topic.
Another example of a quantitative research about WAC is a meta-analysis. Bangert-Drowns et al. published “The Effects of School-Based Writing-to-Learn Interventions on Academic Achievement: A Meta-Analysis” in Review of Educational Research in 2004. These authors quantitatively analyzed the reports of forty-eight previously published research studies about writing-to-learn curricula at a range of grade levels and in a range of disciplines (this latter variable is what makes this meta-analysis relevant to WAC). After consolidating the results of all these studies, using methods appropriate for meta-analyses, the authors concluded that “writing can have a small, positive impact on conventional measures of academic achievement” (29). Other conclusions of this meta-analysis are that the effects are greater if the writing prompts are used for a longer period of time and if the writing prompts require metacognition, but the effects are lessened if these writing curricula are implemented in middle school grades or if the writing assignments are too long. The results of meta-analyses are more noteworthy than the results of a single research study because they help to combine and appropriately weight the findings of many studies, thereby contributing substantially to knowledge in the discipline.
Advice for Locating Empirical Research
When you are conducting bibliographic research, do not dismiss empirical research that has relevance to your topic simply because its methodology and reporting may seem unfamiliar to you. Empirical studies provide valuable knowledge to the discipline, so much so that when Richard Haswell, one of the founders of CompPile (a major database in composition studies), noticed that two of the main professional organizations in composition studies have discouraged the publication of empirical research, he charged those organizations with waging a war on disciplinary knowledge. Haswell argues that the consequences of dismissing empirical research are severe: “when college composition as a whole treats the data-gathering, data-validating, and data-aggregating part of itself as alien, then the whole may be doomed” (219). Haswell cites others who share his stance, including Stephen Witte, who states, “A field that presumes the efficacy of a particular research methodology, a particular inquiry paradigm, will collapse inward upon itself” (qtd. in Haswell 220). To limit this threat, Haswell has coined the term “RAD research” to refer to research in composition studies that is replicable, aggregable, and data supported; he has also restricted CompPile’s use of the search term “data” to refer to “any study that systematically collects and reports facts usable in further study, through whatever research method (interview, ethnography, experimentation, descriptive measurement, case study, etc.)” (Haswell, CompPile Glossary).
When investigating an issue for your bibliographic project, you can use several of the databases that you will learn more about in chapter five—CompPile, WorldCat, ERIC and JSTOR—to search specifically for reports of empirical research. When using the CompPile database, you can locate empirical research by using the search terms for your topic in conjunction with the search terms “RAD research” or “data.” To find any book-length research reports published on your topic, conduct an advanced search of the WorldCat database, using the keywords for your topic along with the following Library of Congress subject descriptor: “English language—Composition and Exercises—Research” (it must be typed using two hyphens to represent each dash). Using the ERIC database, you can more easily identify reports of empirical research on your topic if you limit your search to just the two journals that publish the greatest number of empirical research reports about composition studies: Research in the Teaching of English and Written Communication. Additional journals that publish empirical research reports about topics in education more broadly are indexed in the JSTOR database; the journal Review of Educational Research, which published the meta-analysis about WAC discussed as an example here, is one such journal. After you have read chapter five to learn more about
