Opinion

In defense of the traditional nullhypothesis: remarks on theTrenberth and Curry WIREsopinion articlesMyles Allen∗

In response to their respective opinion articles, I argue that Kevin Trenberth’sproposal to reverse the burden of proof in attribution studies is misguided, butJudith Curry’s counter proposal to abandon hypothesis tests as useless is worsestill. Some observed weather events will have been made more likely by humaninfluence on climate, some less likely, and it is a legitimate and very important fieldof scientific enquiry to work out which are which. The appropriate null hypothesisto use in such studies is that human influence has not increased the probability ofoccurrence of a particular weather event unless the evidence suggests otherwise. 2011 John Wiley & Sons, Ltd.

How to cite this article:WIREs Clim Change 2011, 2:931–934. doi: 10.1002/wcc.145

The WIREs Climate Change opinion articlesby Kevin Trenberth1 and Judith Curry2 were

commissioned to provoke debate in response toKevin Trenberth’s address to the annual meeting ofAmerican Meteorological Society (AMS) in Seattle inJanuary 2011.3 They certainly succeed, in that I findmuch to disagree with in both of them, but I will beginwith the points with which I agree.

Kevin Trenberth’s key concern1,3 is that the over-all approach of the climate science community towardattribution, particularly when it comes to assessingcausal factors behind individual weather events, maybe too conservative. The result may be too many ofwhat statisticians call ‘type-2’ errors, where we con-clude that we cannot reject a null hypothesis (becauseof too little data, or weak or inappropriate statisticaltests) even though that null hypothesis is, in fact, false.He lists extreme climate events that occurred in 2010alone and points out that, in a closely coupled system,human influence on climate must have played somerole in all of them. This is perfectly correct (there is,of course, a caveat, which I will elaborate below).

∗Correspondence to: myles.allen@ouce.ox.ac.uk

School of Geography and the Environment, University of Oxford,Oxford, UK

Trenberth’s underlying concern is a valid one:type-2 errors do no particular harm to climate scien-tists as a group. An individual might miss out on ahigh-profile paper, but that would be a small pricecompared to the reputational harm of claiming a pos-itive result that subsequently turns out to the false.Indeed, the whole system of peer review often seemsgeared toward the avoidance of type-2 errors. Buttype-2 errors can still do harm. Suppose (hypotheti-cally) a small island state is being battered by stormsthat are in fact being made more frequent by humaninfluence on climate, but because of inadequate mod-els and the short length of data records in that regionwe cannot formally reject the null hypothesis of nohuman influence on these storms. If we reassure thegovernment of that island that these storms are (as faras we can tell) entirely natural, they may fail to takeappropriate measures to adapt.

On the other hand, blaming all weather events onclimate change is not helpful either. At present, inter-national adaptation funds are to be disbursed on thebasis of vulnerability4: but vulnerability to what? Toany form of harmful weather, or only those weathertypes that are being exacerbated by anthropogenic cli-mate change? As Trenberth states, ‘all storms developin a changed environment’, but some, like polar lows,may be being made less frequent.5 It is hard to see how

Volume 2, November/December 2011 2011 John Wiley & Sons, Ltd. 931

Opinion wires.wiley.com/climatechange

addressing vulnerability to this class of weather eventcould be called adaptation to anthropogenic climatechange. If attribution becomes an issue in the alloca-tion of adaptation resources (and the question surelycannot be begged forever), type-2 errors do harm; ifwe falsely say a particular weather event has not beenexacerbated by climate change, vulnerable regionsmight miss out on funds they deserve to receive.

It is no coincidence that the few studiesthat have addressed the problem of attribution ofcauses of individual harmful weather events6–9 havefocused exclusively on mid-latitude regions wherelong homogenous data records exist and for whichweather models were originally developed. Yet themost potential for actual harm may well be in poorcommunities in the Arctic and tropics. At a recentIntergovernmental Panel on Climate Change (IPCC)Expert Meeting on attribution, Stephen Schneidermade this point to me with characteristic clarity: ‘weshouldn’t be sacrificing the interests of the world’spoor just to keep you comfortable’ (I’d like to thinkhe didn’t mean it personally; he was referring to meas a representative of the traditional, conservative,type-2-error-averse ‘Working Group I’ attributioncommunity). We should clearly be aware of the dangerof a global bias in attribution claims toward well-monitored, well-understood phenomena which (as ithappens) tend to be those that affect the rich whosetaxes paid for that monitoring and understanding.

In her response to Trenberth’s AMS address,Judith Curry2 objects to his proposed solution, whichis to reverse the traditional burden of proof andrequire those arguing that human influence has notcontributed to a particular harmful weather event toprove their case, rather than vice versa. She pointsout, perfectly correctly, that this could be deeplymisleading, since the null hypothesis that humaninfluence has not affected a weather event in any wayis trivially false, and hence impossible to disprove.

Curry goes on to distil the issue down to our atti-tude to the Precautionary Principle. A strong interpre-tation of precaution does indeed amount to reversingthe burden of proof, obliging those who assert that anactivity does no harm to prove their case. She correctlypoints out that our attitude to precaution is a political,not a scientific, issue, and hence that Trenberth’s essaydoes not, ultimately, turn on a scientific question. Thisis indeed a cogent counterargument, since there is awell-developed literature (e.g., Ref 10) arguing thatthe Strong Precautionary Principle is incoherent.

Now to proceed to points on which I dis-agree—along with most climate scientists I know.Curry refers to ‘other proponents of his arguments’, asif Trenberth’s position were somehow representative

of a new school of scientific thought. My impressionis that it is still very much a minority view. In partic-ular, reversing the burden of proof would represent amassive (and, frankly, inconceivable) policy shift forthe IPCC which, in successive assessments, returns totesting the null hypothesis of no human influence onclimate.

Curry argues that because framing a scientificquestion in terms of hypothesis tests makes it veryimportant where the burden of proof lies and decidingthat is not a purely scientific question, the solution is toabandon hypothesis tests. This seems to be throwingthe baby out with the bathwater. She questionswhether the IPCC’s statement ‘most of the observedincrease in global average temperatures since the mid-20th century is very likely due to the observed increasein anthropogenic greenhouse gas concentrations’ is‘useful’ because it assigns a probability (‘very likely’,meaning less than 10% chance that the statement isfalse) to an imprecise statement, ‘most of the observedwarming’.

There is nothing imprecise about ‘most’: it meansmore than half. As it happens, this wording was intro-duced to replace the (vaguer but more evocative)phrase ‘contributed substantially’ in a nice example ofthe IPCC review process making its conclusions bothmore specific and less emotive. As Curry observes, aninfinite number of statements could have been made,ranging from ‘it is extremely likely that the anthro-pogenic increase in greenhouse gases has caused somewarming’ (not very informative, since an infinitesi-mally small warming is of no policy relevance) to ‘itis about as likely as not that greenhouse-gas-inducedwarming exceeds the total observed warming’ (whichindicates the size of the greenhouse signal, but under-states our confidence in attribution). Far from beinga ‘poor choice’, in Curry’s words, ‘most’ was chosenfor precisely the reasons she advocates: large enoughto be policy relevant, while small enough for the nullhypothesis ‘not most’ to be rejected at an informativeconfidence level.

The idea of giving numerical ranges forthe size of the greenhouse-gas-induced warmingwas indeed discussed in the preparation of theIPCC Third Assessment (‘likely somewhere betweenhalf and double the total observed warming’ wasmy effort, as I recall—met with the derision itdeserved as incomprehensible to anyone other thana climate nerd). Both Third and Fourth Assessmentschose to focus headline results on anthropogenicgreenhouse-gas-induced warming, as opposed to totalanthropogenic warming (a point Curry seems to havemissed), partly because we have much less confidencein the magnitude of anthropogenic aerosol-induced

932 2011 John Wiley & Sons, Ltd. Volume 2, November/December 2011

WIREs Climate Change Defending the traditional null hypothesis

cooling, and partly because the magnitude of thegreenhouse component is most relevant for the longer-term (50 years plus) future.

The need for greater flexibility in the communi-cation of uncertainty is reflected in the latest guidanceto IPCC authors,11 recognizing the need for calibratedlanguage to qualify statements that cannot be framedin terms of a numerical hypothesis test. However,abandoning the old-fashioned hypothesis tests entirelywould be a step too far.

If we are to carry on testing hypotheses, what ofTrenberth’s suggestion that we should assume humaninfluence unless proved otherwise? I doubt this willfind much support in the scientific community. Thekey point that Trenberth ignores is that, while humaninfluence is now ‘playing some role’ in any weatherevent that occurs (at least trivially, in the sense thata seagull in Brazil affects the timing of a tornado inTexas), in around half of these events this role willhave been to have made them less likely to occur ratherthan more.5 Only when the signal of anthropogenicinfluence on extreme weather becomes overwhelming(which looks to be a long way off at present) will itmake sense to assume human influence has increasedthe odds of any weather event that occurs.

Trenberth’s Figure 1 implies that changes inoccurrence frequency are dominated by rare eventsthat have been made more likely by the shift in meanclimate. This is partly an artifact of presentation; onthis scale, a 100% reduction in risk looks smaller thana 300% increase, although it might have more pro-found implications (compare the impact on a tropicalisland of a complete cessation of hurricane activityversus the impact of a three-fold increase). On a moretechnical level, the use of Gaussian distributions inthe example implies that relative risk increases rapidlytoward the upper tail, whereas those studies that

have been done either assume6 or find8 a more uni-form change in relative risk expected from a Paretodistribution.

So we still have plenty of interesting hypothesesto test: did human influence on climate increase therisk of this event at all? Did it increase it by more thana factor of two, often regarded by lawyers as a thresh-old for a ‘substantial’ contribution to risk?12 These arenot easy questions to answer, but not impossible, andsince most of the impacts of climate change will arisefrom changing risks of extreme weather, the answersare critically important. It is scandalous, in my opin-ion, that there is still no systematic effort to compilea global inventory of the impacts of climate change,complementing the United Nations Framework Con-vention on Climate Change (UNFCCC) inventory ofgreenhouse gas emissions. There are, of course, globalinventories of weather-related disasters, but which ofthese disasters are due to events made more (or less)likely by anthropogenic climate change? In the absenceof agreed standards of attribution, many countries andcommunities are claiming to be adversely affected.Which of these claims are valid? And whose interestsare being ignored simply because they do not have thescience base or political nous to complain?

For almost anyone I talk to who is not a cli-mate scientist or environmentalist, the question ofwhether they are being adversely affected by climatechange today is far more interesting than the ques-tion of what the climate will be like in 2100. For fartoo long, the climate science and policy communitieshave dismissed this attitude as short-sighted: the over-whelming focus of the latest model inter-comparisonthat will inform the IPCC Fifth Assessment13 is on pro-jection, not attribution. It is time for this to change:these people pay our salaries, we should be testing thehypotheses they care about.

REFERENCES1. Trenberth KE. Attribution of climate variations

and trends to human influences and natural vari-ability. WIREs Clim Change 2011, 2:925–930.doi:10.1002/wcc.142.

2. Curry J. Nullifying the climate null hypothesis. WIREsClim Change 2011, 2:919–924. doi:10.1002/wcc.141.

3. Trenberth KE. Communicating climate science andthoughts on Climategate. American MeteorologicalSociety Joint Presidential Session on CommunicatingClimate Change, January 23–27, 2011, Seattle, WA.Available at: http://ams.confex.com/ams/91Annual/webprogram/Paper180230.html. (Accessed October26, 2011).

4. Klein RJT, Mohner A. The political dimension of vul-nerability: implications for the Green Climate Fund.IDS Bull 2011, 42:15–22.

5. Zahn M, von Storch H. Decreased frequency of NorthAtlantic polar lows associated with future climatewarming. Nature 2010, 467:309–312.

6. Stott PA, Stone DA, Allen MR. Human contributionto the European heatwave of 2003. Nature 2004,432:610–614.

7. Perlwitz J, Hoerling M, Eischeld J, Xu T, Kumar A. Astrong bout of natural cooling in 2008. Geophys ResLett 2009, 36:L23706.

Volume 2, November/December 2011 2011 John Wiley & Sons, Ltd. 933

Opinion wires.wiley.com/climatechange

8. Pall P, Aina T, Stone DA, Stott PA, Nozawa T,Hilberts AGJ, Lohmann D, Allen MR. Anthropogenicgreenhouse gas contribution to flood risk in England andWales in autumn 2000. Nature 2011, 470:382–385.

9. Dole R, Hoerling M, Perlwitz J, Eischeld J, Pegion P,Zhang T, Quan X-W, Xu T, Murray D. Was there abasis for anticipating the 2010 Russian heat wave?Geophys Res Lett 2011, 38:L06702.

10. Peterson M. The precautionary principle is incoherent.Risk Anal 2006, 26:595–601.

11. Mastrandrea MD, Field CB, Stocker TF, Edenhofer O,Ebi KL, Frame DJ, Held H, Kriegler E, Mach KJ,Matschoss PR, et al. Guidance note for lead authors

of the IPCC Fifth Assessment Report on Consis-tent Treatment of Uncertainties. IntergovernmentalPanel on Climate Change (IPCC), 2010. Available at:http://www.ipcc.ch. (Accessed October 26, 2011).

12. Grossman D. Warming up to a not-so-radical idea: tort-based climate change litigation. Colum J Environ Law2003, 28:1–62.

13. Taylor K, Stouffer RJ, Meehl G. A summaryof the CMIP5 experimental design. Bull AmMeterol Soc 2011. Available at: http://cmip-pcmdi.llnl.gov/cmip5/docs/Taylor_CMIP5_design.pdf.(Accessed October 26, 2011).

934 2011 John Wiley & Sons, Ltd. Volume 2, November/December 2011