The Jedi Mind Tricks in the New Climate Reference Guide for Judges
Part 3: Climate science would be fundamentally misrepresented if judges followed the guide’s advice on deciding admissibility of scientific evidence
Note: Part 3 follows Part 1 and Part 2 of this series.
A judge reading the Reference Guide on Climate Science could not be blamed if he came away with the impression that deciding critical scientific issues in climate change cases is relatively straightforward and easy. A judge looking for such advice would find a simple recipe he could follow.
In the section that advises on how to identify credible scientific evidence, judges are told that the reports from the IPCC, “the leading scientific body for the assessment and synthesis of research on climate change,” are the primary sources for information on climate science, with other synthesis and government reports filling in as necessary.
Judges are also advised that individual peer-reviewed studies may also be consulted on occasion, but they don’t have the weight of IPCC reports
In the next section on expert testimony and reports, judges are told that the role of the expert witness is to summarize IPCC findings, and where necessary, the findings of other organizations. Experts may also be needed to support or refute any claims not already covered by the IPCC reports or those of similar organizations.
Experts, judges are advised, may even draw conclusions without targeted studies by reasoning deductively from general IPCC reports to specific harms claimed by plaintiffs.
The recipe is another Jedi mind trick
Happily, deciding on the admissibility of scientific evidence in climate science cases seems to be a much simpler process than in other scientific fields. A judge can just follow a very simple recipe
Obtain credible scientific evidence mostly from IPCC reports and those of other official organizations
Use experts to interpret and explain those reports if you have any questions
Of course, it’s not really that easy. This simple recipe will lead to serious misrepresentations of climate science that will favor plaintiffs in climate lawsuits. To see how, we’ll apply the recipe to an important climate litigation issue—how to decide if a plaintiff’s injuries were really caused by man-made climate change.
Example: Juliana v United States
Climate attribution scientific questions are at the heart of climate cases. Climate attribution science studies how extreme weather events and changes in climate variables may be attributed to underlying causes. In climate cases, the critical scientific question is to what extent extreme weather events may be attributed to human-caused climate change.
In the 2015 case Juliana v United States, 21 youth plaintiffs sued the United States for alleged violations of their constitutional rights for failing to protect them from extreme weather events produced by greenhouse gas (GHG) emissions.
The defense expert, John Weyant, professor at Stanford and a lead IPCC author, argued that climate models are not sufficiently developed to attribute to GHG emissions the localized harms the plaintiffs were alleging. On the other hand, the plaintiff’s expert, Kevin Trenberth of the National Center for Atmospheric Research and a lead IPCC author, submitted an expert report that essentially just interpreted climate evidence based on official reports, and then concluded from those reports that human-induced climate change was necessarily the cause of the plaintiff’s harms.
Since the case was dismissed for standing, the key climate attribution issue was never decided. However, if a judge followed the recipe advocated in the climate reference guide, he would automatically accept the plaintiff’s argument that attribution questions can be decided by experts interpreting reports.
Trenberth’s Argument in Juliana Is Just The Recipe in the Climate Reference Guide
Trenberth’s expert process follows the recipe advocated by the climate reference guide. Trenberth starts by reviewing and interpreting climate science findings, just as the climate reference guide recommends. In the section on attribution, Trenberth explains the importance of IPCC findings in a footnote:
Consistent with the recipe in the climate reference guide, he will occasionally reference the IPCC, other organizations, or even individual studies. For example, in the discussion of heat waves, Trenberth displays an IPCC chart:
He also references the National Oceanic and Atmospheric Association (NOAA), a government agency.
And, of course, he references individual studies.
Essentially, Trenberth’s process follows the playbook that the reference guide instructed judges to follow.
This process may seem reasonable and logical to a judge not familiar with climate science. However, Trenberth knows there is a fundamental problem with his methodology: it’s not consistent with how climate attribution scientists actually attribute climate change to extreme weather events.
In his expert report, Trenberth acknowledges what he calls the “conventional” climate science attribution methodology, which involves, among other things, running counterfactual simulations of the General Circulation Models (GCM) and other models with and without climate change to isolate the effect of GHG emissions on climate variables as well as extreme weather events. In his expert report, Trenberth criticizes the “conventional approach” in favor of the methodology he is known for developing, the conditional or “storyline” approach.
In his storyline approach to climate attribution, Trenberth conditions on the extreme weather event having already happened. Given that an extreme weather event, such as a heat wave, did happen, the conditional approach asks what’s the evidence that climate change made it worse. It’s often called the storyline approach because the conditional methodology is conducive to telling the kind of causal stories that Trenberth and the climate reference guide suggest judges should think to be acceptable scientific evidence in a climate change trial.
For example, suppose a plaintiff claims that he suffered injuries from a severe hurricane. The conventional approach would attempt to model the full dynamics of a hurricane within a model under a counterfactual of no climate change and with climate change to discover the effect of climate change. (I’m simplifying here of course. There are many other attribution techniques as well). The conventional approach is of course very hard since there are many aspects of hurricanes that are not well understood. It’s hard; but then science is hard.
Trenberth’s conditional approach simplifies the problem by essentially ignoring all that complexity. Instead of simulating a hurricane, the conditional approach assumes a hurricane occurred and then asks how climate change made it worse. The basic story is as follows: the plaintiff suffered harm from a severe hurricane. Given that the hurricane occurred, an expert can examine IPCC reports and find that the IPCC has deemed it “likely,” as discussed in Part 1, that hurricanes have already become more severe as a result of climate change. Therefore, the expert and the judge can conclude that climate change caused the plaintiff’s harm.
This last step—that climate changed caused the plaintiff’s harm— is exactly the step Trenberth takes in his expert report. Having established that the extreme weather event occurred and that consensus science says that climate change made it worse, he immediately concludes that the plaintiffs were harmed by climate change.
Trenberth’s methodology in support of the plaintiffs in Juliana is the same process the climate reference guide instructs judges to follow.
The climate reference guide makes clear that Trenberth’s methodology can be used in cases in which a climate expert could infer causation of an extreme weather event even without a targeted study. To support that claim, they cite Climate Change Attribution and Legal Contexts: Evidence and the Role of Storylines. This paper attacks Weyent’s methodology in Juliana and argues for Trenberth’s, with explicit support of Trenberth’s storyline approach. The recipe in the climate reference guide is clearly Trenberth’s argument for the plaintiff in Juliana.
The recipe that the reference guide advocates makes climate causation very easy for plaintiffs to prove. The judge just needs to see an official report from the IPCC or some other organization. If the judge has any questions, the plaintiff can bring in an expert to answer them.
The recipe in the climate reference guide gives the defense little to counter with. The defense would be allowed to bring in their own expert to dispute the plaintiff’s expert’s interpretation of the official report, much as religious scholars argue over the meaning of a sacred text. Alternatively, the defense expert could point to new research that has not been incorporated into the official report yet. But beyond that, the climate reference guide would tie the defense’s hands.
Defense Expert Weyant’s Counter-Arguments
Before laying out Weyant’s counter-arguments for the defense, its important to dispense with an objection that will likely be made. In climate cases, experts who testify for the defense are often labeled climate deniers or cranks. Weyant is neither. He is a noted climate scientist and a lead author at the IPCC. Weyant is an expert in Integrate Assessment Modeling (IAM), a class of climate models that IPCC uses extensively for policy work.
Weyant is not disputing Trenberth’s assessment of the state of climate research—he agrees with it. But his main point is that general statements about findings in climate research from official reports, even though correct, can’t be used to prove the localized harms claimed by plaintiffs in climate cases.
Weyant makes three points countering Trenberth’s analysis
Climate models aren’t sufficiently developed to reliably support a climate plaintiff’s contentions
Confounding factors bedevil any attribution analysis of the sources of harms from climate change
The conditional or “storyline” approach, relied upon in Trenberth’s expert report is not a legitimate alternative to the climate attribution models
We’ll take up these points in turn.
Climate models aren’t sufficiently developed to support climate plaintiff’s claims
Trenberth essentially agrees with Weyant that the conventional climate models, the GCMS for example, are not sufficiently developed to support specific claims of climate harms. That realization was what motivated Trenberth to develop the storyline approach in the first place. But Weyant’s point is that the conventional models were not developed specifically to measure harms (or benefits) from climate change. For that purpose, climate scientists have developed two other classes of models, models ignored by Trenberth and the climate reference guide, that the IPCC uses extensively for policy analysis:
climate impact models that combine climate and socio-economic drivers to make predictions of impacts on the economy and the environment at regional and sectoral scales
integrated assessment models (IAM) that combine simplified climate models with models of the economy and impact models.
Both model types are used extensively by the IPCC for policy purposes. They are also the models that climate scientists have developed to understand the benefits and harms from climate change, making them relevant in court cases. However, they are not even mentioned by Trenberth or the climate reference guide. Weyant’s key point is that these models (and keep in mind that Weyant was a lead IPCC author on IAMs) are not well-developed enough at this point to support plaintiff claims in climate litigation.
Of course, Weyant’s claim might be disputed. But the recipe would have omitted discussion of these two important model classes.
Confounding factors bedevil climate attribution studies
Weyant points out that there is an essential difference between the claim that climate change made an extreme weather event worse and the claim that the harm a plaintiff suffered during an extreme weather event was caused by climate change. The latter claim is relevant in the courtroom, but analyzing the latter claim is bedeviled by the presence of confounding factors.
For example, wildfires can happen more often for all sorts of non-climate change reasons. Wildfires are started by humans with alarming frequency. Policies can also make wildfires more frequent and intense. The controlled burns that are necessary to mitigate wildfires may have been reduced by forest management regulatory policy, or the policy may be have been changed to burn areas that lead to more natural wildfire ignitions.
Heat stress is similarly complicated. If a plaintiff lives in a highly populated area, urbanization may have created the “urban heat island effect” in which temperatures are higher in the city compared to the surrounding areas. Demographics matter too. An older population is more susceptible to heat stress, whether or not climate change has occurred.
In general, there is a long list of confounding factors in any extreme weather event that could have caused the harm alleged by climate change. Establishing causation is not easy because it’s necessary to examine a plaintiff’s particular circumstances and then sort through the confounding factors and climate change explanations, quantifying the relative contributions of each. Trenberth and the climate reference guide recipe assume that process away.
The storyline approach is not a legitimate alternative to climate models that were developed specifically for the purposes of attribution
As Weyant points out, the storyline approach is designed to replace just one of the three types of models that have been developed for climate attribution. It replaces the GCM and other models by assuming away the dynamical properties of the phenomenon, focusing on the thermodynamics. As Trenberth explained in his justification for the storyline approach, climate scientists understand the thermodynamics of weather events better than they understand the dynamics. The storyline approach holds fixed the dynamics, therefore, and concentrates on the effects of climate change on the thermodynamic properties.
To many readers, the storyline approach, when explained in terms of its underlying scientific assumptions—thermodynamics versus dynamics—may seem too abstract to follow, so let me offer a concrete example.
Suppose a plaintiff living in Houston alleges that he was harmed by a bad hurricane. The plaintiff sues the federal government, as in Juliana, or perhaps he sues an oil major. The plaintiff brings in an expert who follows the playbook advocated in the climate reference guide. The plaintiff’s expert refers to the findings I discussed in Part 1. The expert, reading reports, tells the judge it is “likely” that precipitation in Houston from extreme storms has been worsened by climate change and it is “likely” that that proportion of hurricanes in categories 3-5 has already gotten worse as a result of climate change. Thus, the expert concludes that climate change did indeed cause the plaintiff’s harms.
For the moment, let’s set aside the problems with using IPCC and other ratings in courtrooms for which they were not designed. Let’s also set aside that the models that climate attribution scientists have developed for attribution analysis—impact models and IAMs—are being ignored here. And let’s also set aside the potential presence of confounding factors. Does the expert’s argument make sense if we ignore these issues?
It does not. This argument depends on the storyline approach, which focuses on the thermodynamics of the hurricane. The thermodynamics are easier for scientists to analyze. A hurricane is like a giant heat engine that sucks in and uses heat from higher sea surface temperatures. The higher the sea surface temperature, other things equal, the greater the strength of the hurricane. Increased precipitation is also basically thermodynamics. Warmer air holds more water vapor (the Clausius-Clapeyron relation) and so rising temperatures should lead to more precipitation. The expert is talking about the thermodynamic properties of hurricanes in his testimony.
But implicit in this argument is that the dynamical properties of the hurricane don’t change in a way that matters with climate change. The dynamical properties are where and how the hurricane forms, the “genesis” points, and how the hurricane tracks as it builds strength and moves along the ocean. What if climate change also modified the dynamical properties of hurricanes so that they are less likely to strike the Houston area, even though those that do form are stronger and have more precipitation?
Sound far-fetched? The 2021 paper U.S. Tropical Cyclone Activity in the 2030s Based on Projected Changes in Tropical Sea Surface Temperature estimates a statistical model of the full lifecycle of a hurricane and then uses it to project the effects of hurricanes in 2030. The paper finds that hurricanes indeed get stronger from rising sea surface temperatures but also that the dynamical properties change. The genesis points move eastward, further away from the United States. The tracks also curve North more. The net effect is that hurricanes get stronger but they become less likely to strike the Houston area and the panhandle of Florida and more likely to strike the east coast of Florida. If the plaintiff lives in Houston, then climate change reduced his harms.
It is important to realize that Weyant’s argument against the storyline approach does not depend on knowing that the dynamical properties have changed. I only referred to the 2021 paper to show that it is plausible to think the dynamics of extreme events could change with climate change. The point is that climate scientists know much less about the dynamical properties of extreme weather events than they do about the thermodynamics. Trenberth agrees, which is why he developed the storyline approach in the first place. But Weyent’s point is that you can’t just assume the problem away. Any expert who comes into a courtroom interpreting IPCC reports is making a gigantic background assumption about the dynamics of extreme weather events for which he has little or no evidence.
The climate reference chapter is constantly saying that the IPCC should be the pre-eminent authority, but then it recommends a recipe for judges to follow that implicitly assumes the storyline approach in attribution questions that the IPCC doesn’t endorse. Ironically, the IPCC agrees with Weyant that the storyline approach can’t fully solve the attribution problem. In the IPCC AR6, in the chapter on extreme weather events, the IPCC had this to say about the storyline approach (the fixed effects it refers are the dynamics that are assumed to be unaffected by climate change):
Conclusions
If a judge followed the recipe in the climate reference guide, he would deeply misrepresent the climate science attribution debate, probably the most important unsettled issue in climate science cases. In climate attribution cases, a scientific admissibility recipe in which experts interpret official reports ignores
the relevance of impact models and IAMs that climate scientists have developed specifically for the purpose of measuring the harms of climate change
the views of a lead IPCC author and expert on IAMs and impact models who said these models are not developed enough to infer causation in climate cases
the importance of separating out confounding factors
the gigantic background assumption, for which there is little to no evidence, that dynamical properties of extreme weather events aren’t affected by climate change
In fact, a recipe in which an expert’s role is primarily to interpret official reports will lead to deeply misleading conclusions on all climate-related scientific questions. The advice in the climate reference guide that judges can decide admissibility of scientific evidence by having experts explain and interpret official reports is gravely flawed. If there is a re-write of the guide, the recipe should be removed. In the meantime, judges should ignore the reference guide’s unsound advice.