the other tornado analysis is less clear‐cut, however: some injuries may be more severe and consequential than others. And neither of those metrics is applicable for impacts outside of human health. A starting point might be money, considering that so much of our lives is spent using it as a universal currency. But can we put a monetary value on a species going extinct? Or on various aspects of livelihoods and culture?
A partial way around this challenge is to use a qualitative measure of relative change instead of a quantitative metric (Cramer et al., 2014; Oppenheimer et al., 2014; Smith et al., 2001, 2009). For instance, in their synthesis assessment of the detection and attribution of changes in risk associated with extreme weather, Cramer et al. (2014) synthesized only across like systems (e.g., bleaching/stress/mortality of warm water corals) when assigning a level of confidence to the evaluation of whether observed climate trends had played a major or minor role in an observed change. Hence, their summary statement highlighted “High‐temperature spells have impacted one system with high confidence (coral reefs), indicating Risks Associated with Extreme Weather Events. Elsewhere, extreme events have caused increasing impacts and economic losses, but there is only low confidence in attribution to climate change for these” (Cramer et al., 2014, p. 983) but included no cross‐system synthesis. However, these system‐specific conclusions were then aggregated into a past‐to‐future assessment of the qualitative change in risk by Oppenheimer et al. (2014). Synthesizing across qualitative, rather than quantitative, outputs of detection and attribution analyses means that the synthesis is more flexible in the types of detection and attribution analyses it can include. For instance, a multiple linear regression analysis may be appropriate for a system that behaves fairly linearly to external perturbations, but another type of analysis may be required for a system with a highly nonlinear response. In a quantitative synthesis it would be hard to include the output parameters of both analyses in a consistent way. Similarly, being able to include more disparate types of analyses of each component input (e.g., different studies of butterfly range shifts using different techniques) means that a qualitative synthesis can incorporate a more robust representation of uncertainty. However, the trade‐off is a lack of transparency over technical details that may be important.
An alternative approach is to convert results of individual studies into a binary metric, such as “predictions consistent with observations” versus “predictions inconsistent with observations” (Rosenzweig et al., 2007, 2008; Savo et al., 2016). For predictions of future risks, a possible binary metric might be based on a threshold for losses or damages or based on a threshold for relative importance in relation to predicted effects of other factors. With some loss of information about severity, this approach can in practice produce a single synthesis measure. However, it has several important assumptions (Stone et al., 2013). Most important, by assuming that each unit of study (for which a binary result is assigned) is equivalently important, it is still assigning value. Such an approach has yet to be applied specifically to impacts related to extreme weather.
1.2.4. The Arithmetic of Synthesis
There are two possible dimensions in which one can conduct a synthesis analysis: horizontally, across like systems, or vertically, along the causative chain. Figure 1.2 shows a simple example from Cramer et al. (2014) in which both dimensions were explicitly invoked in developing a synthesis conclusion of the detection and attribution of “increased erosion of Arctic coasts.” Vertically, synthesis assessments of individual steps in the causal chain, from “decreasing Arctic sea ice cover in summer” through “lack of sea ice protection from wind storms” were used to build the final assessment.
Alternatively, the final assessment can be seen as the horizontal synthesis across multiple like systems, in this case across the Arctic regions of Asia, Alaska, and Canada. Although the various causative steps of the regional assessments were not listed in the published report, they were necessarily implicit in the development of the regional assessments; similarly, the various Arctic‐wide assessments were developed from regional information. Thus in fact this figure should appear more as a grid, with only certain cells having published assessments.
Figure 1.2 Synthesis assessments from the IPCC AR5 concerning the attribution of increased erosion of Arctic coasts. In Cramer et al. (2014) synthesis assessments were made for various aspects of the information feeding the overall assessment. The overall assessment can be viewed as being developed through a causative chain or as aggregation across regional assessments. Confidence is given for the existence of a trend for “decreasing sea ice cover in summer” and for a “major role” in causing trends along the arrows from one box to another.
The nature of synthesis across the two dimensions differs. Sensibly, confidence along the vertical causal chain, in the existence of a trend in the first step and of causation in the last two steps, decreases as the assessment proceeds through the impact chain. Along the horizontal regional dimension, though, confidence in the Arctic‐wide assessment is the same as for the regional assessments. This is sensible enough, but what if the assessment for Asia had been for “very low confidence”? Basing the Arctic‐wide assessment on the more or less confident result would mean that the existing synthesis assessment would not be representative of the entire Arctic (Stone et al., 2013). However, taking some qualitative average (i.e., “low confidence”) would hide the existence of “medium confidence” in at least some impacts. Cramer et al. (2014) attempted to deal with this issue by adopting the practice of assigning confidence to carefully worded synthesis statements, with the explanation that “the confidence statements refer to a globally balanced assessment” (p. 1014). So for instance, the assessment of “changes in flood frequency and magnitude in non‐snowmelt‐fed rivers” referred to changes of any nature, not applicable to all non‐snowmelt‐fed rivers around the planet but rather to the existence of such changes in at least a major river in most continents.
This issue of “horizontal arithmetic” does not only apply to the confidence measure used by Cramer et al. (2014). For the binary synthesis approach previously described, Rosenzweig et al. (2007) consider if one assessment concluded no impact or an impact in the opposite sense of another region (e.g., decreased erosion for the preceding example). A high “no‐impact” count implies a lesser overall combined impact, even though this is by no means necessarily the case. However, given uncertainty in the assessments, picking the most extreme case would be biased, because it would produce a large combined impact estimate even in the absence of climate change. At the other extreme, the fact that one particular system is not being affected may have little overall relevance, and so it should not be selected as representative (Stone et al., 2013).
1.2.5. Is There Power in Numbers?
A final concern is in understanding the uncertainty in any final synthesis measure. This depends not only on the described factors but also on interdependence of the individual studies contributing to the synthesis (Cramer et al., 2014). For example, in synthesis studies of shifts in the geographic ranges of multiple species it is assumed that each species shifts its range independently of others (e.g., Hockey et al., 2011; Parmesan et al., 2011; Rosenzweig & Neofotis, 2003). In that case the addition of observations of the range shift of an additional species adds substantial new information to the synthesis. However, the independence is hard to confirm when species are shifting their ranges as part of a general relocation of an entire ecosystem: observations for a species that is simply following its food (with the observations of that species already included) will lend confidence to the observations of its food but will not truly add a new item within the synthesis.
1.3. SYNTHESIZING ACROSS EVERYTHING
In
