A tutorial on using AccessIPCC’s FAR_OUT

Donna Laframboise has created an excellent introduction and guide to using AccessIPCC’s FAR_OUT. I am reproducing her post below, with her kind permission.

A Powerful New Research Tool

December 7, 2010


Canadian blogger Hilary Ostrov and Australian computer programmer Peter B. have given the climate change world a gift this week. Since March they’ve been hyperlinking and annotating the 3,000-page Intergovernmental Panel on Climate Change (IPCC) report released in 2007 . The result is AccessIPCC.com.

Those of us who’ve been taking a close look at the 2007 report (also known as AR4) have identified numerous concerns. Now we have a tool to analyze it more comprehensively than ever before.

The more than 18,000 references on which the IPCC builds its case have been coded via an automated process. This means one can now visually scan a chapter and see which citations are of potential concern. To gain a clean designation (and thus remain untagged by AccessIPCC), a reference must meet stringent criteria:

  • it is apparently peer-reviewed
  • it doesn’t violate the IPCC’s cut-off dates (that issue is discussed here, here, here, and here)
  • it wasn’t written by an author of that IPCC chapter (which would make it a self-reference)
  • it wasn’t written by an author of another IPCC chapter (an indicator of the incestuous nature of the IPCC-affiliated community)
  • it doesn’t base its conclusions on a model or a simulation (versus real-world observations)
  • it wasn’t written by a Person of Concern
  • it wasn’t published by a Journal of Concern

Persons of Concern are defined here (the actual list of names appears at the “Persons of Concern” tab at the bottom left of your screen). These are a group of activist-oriented scientists known as the Hockey Team – as well as a handful of people closely associated with them in the Climategate emails.

Journals of Concern are those that have published papers authored by Persons of Concern. One of the most disturbing things we learned from Climategate is that academic peer-review can be startlingly superficial. Phil Jones, the director of the University of East Anglia’s Climatic Research Unit (the source of the Climategate e-mails), told a UK parliamentary committee in March that, in all his years of publishing papers in reputable journals such as Nature and Science, no one has ever asked to examine his raw data or his computer code.

In drug trials, raw data and computer codes are inspected as a matter of course. A new kind of aspirin receives orders of magnitude more scrutiny than does the sort of research routinely cited by the IPCC. This means that governments are making trillion-dollar climate decisions based on IPCC reports that rely on data no one has ever double-checked. The mere fact that an academic journal published the research findings seems to be good enough for the IPCC. (This approach then empowers author cliques to highlight papers that support certain arguments while ignoring the existence of contrary ones. The IPCC’s approach to the species extinction question provides a classic illustration of this.)

AccessIPCC doesn’t imply that every article published in a Journal of Concern is flawed. It doesn’t imply that every research paper that depends on modeling is invalid. It merely alerts us to the fact that we should proceed with caution. (If your brain, like my own, benefits from cheat sheets here’s one you can print-out until you become familiar with the AccessIPCC codes.)

In the IPCC version of its report, citations are cryptic and a good deal of messing about is required before one is able to examine a reference in full. AccessIPCC solves this problem. For example, a section in which the IPCC discusses urban air quality looks like this on the IPCC website (click the image to enlarge or click here to go to the original):

click to enlarge

Below is a cut-and-paste of the same section from AccessIPCC: Urban air quality
Background levels of ground-level ozone have risen since pre-industrial times because of increasing emissions of methane, carbon monoxide and nitrogen oxides; this trend is expected to continue over the next 50 years ( Fusco and Logan, 2003 [MoS] ; Prather et al., 2003 [JoC] ). Changes in concentrations of ground-level ozone driven by scenarios of future emissions and/or weather patterns have been projected for Europe and North America ( Stevenson et al., 2000 [JoC, MoS] ; Derwent et al., 2001 [JoC] ; Johnson et al., 2001 [JoC, MoS] ; Taha, 2001 [NPR, MoS] ; Hogrefe et al., 2004 [MoS, ARC] ). Future emissions are, of course, uncertain, and depend on assumptions of population growth, economic development, regulatory actions and energy use ( Syri et al., 2002 [MoS] ; Webster et al., 2002a [MoS] ). Assuming no change in the emissions of ozone precursors, the extent to which climate change affects the frequency of future ‘ozone episodes‘ will depend on the occurrence of the required meteorological conditions ( Jones and Davies, 2000 [MoS] ;( Sousounis et al., 2002; ) Hogrefe et al., 2004 [MoS, ARC] ;( Laurila et al., 2004; ) Mickley et al., 2004 [JoC, MoS] ). Table 8.4 summarises projections of future morbidity and mortality based on current exposure–mortality relationships applied to projected ozone concentrations. An increase in ozone concentrations will affect the ability of regions to achieve air-quality targets. There are no projections for cities in low- or middle-income countries, despite the heavier pollution burdens in these populations.

Hover your cursor over any of the above citations and voilà – full details appear. The red-coloured font tells us that 12 of the 14 citations in this paragraph may be of concern. The MoS after the first one tells us these research conclusions were derived via simulations. Since a simulation’s output cannot be accurate if its inputs are suspect, it’s important to receive this heads-up. With just a glance we now see that 10 of these 14 citations rely on modeling rather than on concrete, real-world data to reach their conclusions.

We also see that 5 citations have been tagged JoC – which means they were published in Journals of Concern. One is tagged NPR – which tells us it is not peer-reviewed. And two are tagged ARC – which indicates this particular research was written by a person who authored another chapter in the same IPCC report.

In a different part of the report, a cut-and-paste from near the top of the AccessIPCC version of the chapter shows us how many cited papers were produced by the people who wrote the IPCC chapter (see it online here):

Coordinating Lead Authors:
Pete Smith (UK) [SRC:15], Daniel Martino (Uruguay) [SRC:2],

Lead Authors:
Zucong Cai (PR China) [SRC:8], Daniel Gwary (Nigeria) [SRC:2], Henry Janzen (Canada) [SRC:8], Pushpam Kumar (India) [SRC:3], Bruce McCarl (USA) [SRC:13], Stephen Ogle (USA) [SRC:7], Frank O’Mara (Ireland) [SRC:8], Charles Rice (USA) [SRC:6], Bob Scholes (South Africa), Oleg Sirotenko (Russia) [SRC:3], Mark Howden (Australia) [SRC:2], Tim McAllister (Canada) [SRC:3], Genxing Pan (China) [SRC:5], Vladimir Romanenkov (Russia) [SRC:3], Steven Rose (USA) [SRC:1], Uwe Schneider (Germany) [SRC:5], Sirintornthep Towprayoon (Thailand) [SRC:2], Martin Wattenbach (UK) [SRC:2],

This means that this chapter’s arguments rely on research conducted by 19 out of its 20 authors. Coordinating lead author Pete Smith felt it was appropriate to cite 15 of his own publications. Meanwhile, 13 papers authored by lead author Bruce McCarl are also cited.

As those familiar with academia are aware, one earns brownie points if one’s papers are widely cited elsewhere. In this case, we see that such citations have been orchestrated by the authors themselves. The above results also lend support to economist Ross McKitrick’s view (see page 3 here) that one needn’t wait for the IPCC to write its report. All one needs to do is look at the people who’ve been assigned to a particular chapter and the outcome can be predicted in advance.

This is Version 1 of AccessIPCC. And remember, it has been produced by volunteers. Hilary cautions us that the project is not yet complete and is “far from perfect.” Some of the codes will no doubt need correcting. (There’s a dedicated blog where discussions of that nature can take place.)

That being said, AccessIPCC represents an enormous leap forward. We now have a more user-friendly – and immeasurably richer – way of evaluating the IPCC’s Nobel-winning report.


Full disclosure: AccessIPCC is the brainchild of Peter B., who contacted me in March while I was coordinating the Citizen Audit of AR4 references. He and Hilary became acquainted and joined forces via that initiative.

AccessIPCC’s FAR_OUT Version 1 Released

I am pleased report that thanks to Peter’s tremendous efforts, Version 1 of FAR_OUT (Fourth Assessment Report – Objectively, Uniformly Tagged) is now ready for prime-time live :-)

In this version, all 44 Chapters of AR4’s Working Groups 1, 2 and 3 have been annotated by AccessIPCC. In the months ahead we will be incorporating the Technical Summaries, FAQs, Synthesis Reports and Summary for Policymakers – all of which require somewhat different programming for annotation purposes.

Once again, we thank those who have contributed their very valuable input and feedback as this project evolved. (And your comments have been preserved for the record and virtual posterity in our new History page).

We know that this version is far from perfect, but we hope that you will find the “pictures” emerging at the different levels of exploration (from Chapter, to Working Group, and Overall) will be helpful.

Both Peter and I are committed to continuous improvement and we would welcome your comments and feedback here. Please do take your mouse for a stroll, and do come back let us know what you think.

Update 2010 Dec 07: We have made a few minor revisions. Details can be found on our Version History page – on which we will track changes from now on.

People Behind AccessIPCC

AccessIPCC was started by Peter B., an Australian programmer and Hilary Ostrov who got in contact via the CitizenAudit project.

Other people are assisting but all criticism should be directed at us.

By Hilary Ostrov:

British-born (more years ago than I care to remember) and proudly Canadian-bred, I reside in a suburb of Vancouver, British Columbia – along with my very vocal cat, Amber.

My background – which includes a BA in English & Psych. from the University of Western Ontario – and current vocation (computer training, database design and management, website design and other computer related stuff) are such that I would never have imagined – even a year ago – that so much of my time would be consumed by matters climatic.

In fact, it wasn’t until about a week or so “BC” (Before Climategate) that I even became aware of any controversy. Certainly, on more than one occasion in recent years you could have heard me blithely muttering “must be global warming” when confronted by unusual and/or extreme weather conditions (although on the rare occasion that I gave the matter more than a passing thought, the theory that it must be our CO2 that is the primary cause always struck me as being somewhat far-fetched). My initial interest was sparked by a link I decided to follow from a response to an article in Canada’s National Post. And the rest, as they say, is history!

By December 2009, I was sufficiently informed, and concerned by so much of what I had learned, that I decided to start my own blog: The View From Here. Not that I expect anything I write to change the world, but at least it gives me a platform for airing my thoughts – and indulging my whimsical penchant for playing with words :-)

When Peter introduced himself and his project to me, in mid-March of this year, I was astounded at the potential power of Peter’s program as a self-directed research and educational tool; it will be a virtual goldmine of information at one’s fingertips!

I’m honoured and delighted to be working with Peter on this project … for the magnificent fee of $0.00 per hour ;-)

By Peter B.:

I am an ex-Royal Australian Navy engineering officer with a BSc(Eng) from London University. I spent most of my navy time programming computers.

I was nudged into looking into Climate Change by my cousin. It quickly became apparent that the Fourth Assessment Report of the IPCC was underpinning much climate policy. The ClimateGate emails became available but it was difficult if not impossible to see what relation (if any) they had to the FAR. This effort started in an attempt to make it easier to access the FAR and its interrelated data.