History of AccessIPCC

This blog was initiated on May 23, 2009. At that time, FAR_OUT was in its very early stages – and the blog was not public, but rather open to a few individuals from whom we sought input. These were initially blog posts and comments, the content of which has now been reproduced (without formatting or links which are now irrelevant) on this page, just for the record.

New sample chapters now available
May 26, 2010 by hro001

Reminder, please keep in mind that at this time we would ask that you not share this link (or any AccessIPCC links) with others.

We have received some helpful feedback from some who have viewed our initial sample chapter, which we have incorporated into this latest iteration. And we now have a real index page.

In addition to Working Group (WG) 1, Chapter (CH) 4, we have also made available WG1 CH3 and WG1 CH6. All 3 chapters may be accessed from the index page – and there are links from each chapter back to the index page.

UPDATE 11/4/2010 – links to original demo have been updated, and as of today, sample includes all chapters of Working Group 1. [hro]

Some caveats regarding chapters 3 and 6: the conversion from .pdf to .html is not always as smooth as one would like: inconsistencies in the original pdf unfortunately result in anomalies in the output which present “tagging” (and many other!) challenges that Peter is working on overcoming!

While all our tagging, numbers and totals are still preliminary (primarily due to the above challenges!), we have expanded the Table of Reviewer Comments so that it now includes an indication of Reviewers who might have other roles in other chapters (e.g. Contributing Author), as well as an indication of the number of papers authored (or co-authored) by the Reviewer which are cited in AR4.

Currently, in the Table of Reviewer Comments, we show the total number of comments for each Reviewer, as well as the number of “Rejected” Comments. However, as I discovered a few days ago, while attempting to determine how we might obtain a more accurate quantification of the “responses” to the Reviewer Comments, this does not reflect the true number of comments that were actually “Accepted”. So we will eventually be replacing the “Rejected” Comments column with an “Accepted” Comments column. As I had noted in my post:

I did find it somewhat surprising that on a first pass, so to speak, of the 34,000+ Reviewer Comments [on the Second Order Draft] a mere 7,664 (22%) responses were unambiguously identifiable as “Accepted”.

[…] One could add to the above the 668 that were unambiguously (well, after a careful process of prior elimination) identifiable as “Agreed” (even though “Agree(d)” was not an [IPCC designated] option). Assuming that the IPCC would view “Agree(d)” as “Accepted” (which may or may not be a reasonable assumption), this would bring the Accepted total to 8,332 (23.9%).

We have also added some navigation links to different parts of each annotated chapter page, at the top of each page.

Once again, please do come back and let us know what you think!
=======

Coming soon ….

May 23, 2010 by hro001

Reminder, please keep in mind that at this time we would ask that you not share this link (or any AccessIPCC links) with others.

UPDATE May 26: this post is now somewhat obsolete, as we now have an index page and other features!

There will, of course, be an index page which will contain links to each of the annotated chapters – as well as some summary information – but it’s not quite yet ready for semi-prime-time review. Later today (or perhaps tomorrow), I shall do some screen captures and append to this post so you will have an idea of what this might look like.

Update (07:20 PM) Here are some screen captures of early version of future pages you can expect to see. Imagine that this material (particularly persons and journals) is all hyperlinked to more detail (click each image for larger version):

Our table of contents (index) could include percentage of tagged references

The Positions column indicates authorship roles within AR4

An example of information the database contains regarding the Journals


====

Welcome to AccessIPCC

May 23, 2010 by hro001 6 Comments

Reminder, please keep in mind that at this time we would ask that you not share this link (or any AccessIPCC links) with others.

If you are viewing this page, you have been invited to provide us with some feedback on your impressions of the sample Chapter (Working Group I, Chapter 4) that has been annotated by AccessIPCC.

Please do read the About AccessIPCC page which will give you more information about this project, and then follow the link to the sample chapter:

FAR_OUT Fourth Assessment Report – Objectively Uniformly Tagged

A few things to keep in mind …

The version of the chapter that you will see was converted from the .pdf version of the 4th Assessment Report – and there are some layout differences between the online version of the original and the .pdf version.

This is still very much a work in progress, so that any numbers you may see are still preliminary (but they’re getting quite close to reality!)

Some images, tables and figures got lost or warped in the conversion from .pdf to html. We have not replaced or “fixed” these anomalies, because for the most part they do not affect our results (I know, I know … sounds like I must be a “climate scientist” in training!) Each annotated page, however, does contain a link to the relevant chapter index of the AR4 online version.

Please do come back and let us know what you think :-)

Comments

willr2010 says:
May 24, 2010 at 5:04 am

Just had a quick glance. Looks good so far.

No real surprises at the inter-linking. It was quite obvious when we were auditing.

How will you handle all the references to UN documents?

hro001 says:
May 24, 2010 at 3:35 pm

willr, thanks for your comments :-)

At this point, any reference to UN documents is covered by the “Non Peer Reviewed” “tag” (just as it was during our citizen audit). But we’re open to suggestions! Did you have anything particular in mind?!

Donna Laframboise says:
May 27, 2010 at 2:47 pm

This looks so great. Peter & Hilary, you both should take a bow. A lot of work has gone into this and it has a nice, clean look to it. What a treasure trove. I can’t wait to start using it :-)

In a brainstorming spirit, the following are some thoughts. Not sure if other people will see things this way or not. I’m just throwing out some ideas.

I’m wondering how it would work if JoC (journal of concern) were substituted for the longer phrase currently in use: CRUsted By Journal. That would be shorter and more neutral. (The fact that, if you sound it out, it sounds like “joke” can be our little secret *wink*.)

Ideally, if it isn’t too much trouble, each appearance of that term would be hyper-linked back to a single page where users would find – at the top of the page – a brief, general explanation of the possible reasons why these journals have been so designated. If, further down the same page, a list of the journals that are deemed to be of concern by this project could appear, with a brief explanation of what those concerns are in each specific case, that would be truly awesome.

In my view, it’s critical that the criteria used to arrive at the JoC designation be stated clearly. Otherwise, the entire project could be at risk of being dismissed by folks who say something along the lines of: “Nature and Science are the most prestigious scientific journals in the world. Anyone who’d put them on that list is being biased and unreasonable and shouldn’t be paid any mind.”

So the page that discusses the JoC designation could talk about the following things:

1. According to Phil Jones’ testimony, none of the journals that published his work ever asked for his raw data during the peer-review process. This is surely a legitimate cause for concern. Once the user/reader has been advised of this, it’s up to them to draw their own conclusions. (We’d just need to provide a link to a news article in which the Phil Jones testimony is reported, to back ourselves up.)

2. Some journals have (for many years, now) written editorials that have taken advocacy positions re: the AGW debate. Moreover, in the case of Nature, this UK science publication owned by a German publishing company somehow felt it was appropriate to write an editorial endorsing Barack Obama in the last Americanelection. In other words, journals that have indulged in advocacy should, in my view, also be designated as a JoC. If we cite/link to the offending editorials this invites users/readers to become more thoughtful about whether or not these journals are behaving professionally. (A person or two might be found who could track down and make a list of those sorts of editorials. I already have a short list of them myself.)

3. Some journals have policies re: data sharing, but don’t enforce those policies. This would be another (in my view) appropriate reason to designate a journal as a JoC.

Of all the tags, it seems that the CRUsted By Journal is the most widespread throughout the currently available chapters. If it would be possible to put this particular tag in an alternate colour (I know there’s a limit to how many colours one wants to use – too many becomes confusing), it might be helpful.

Perhaps, in a similar spirit, Non Peer Reviewed could be shortened to NPR (again, hyper-linked back to a single page that explains the criteria used, explains why IPCC reports are not peer-reviewed according to the traditional definition, and also states that this designation could well be in error and should be confirmed by further research on the part of the user/reader).

And, just because I’m on a roll, perhaps AR4 Author This Chapter could be shortened to Chapter Author.

Just some thoughts…

hro001 says:
May 30, 2010 at 12:24 pm

Thanks, Donna … this is all very helpful :-)

The two versions of the sample chapters that we have “published” so far contain very preliminary numbers [the main reason that we are not even going “quasi public” at this point in time!]

We know that we need to refine and define our criteria – particularly for JoC [a naming convention that we are adopting – and extending to replace CRUsted, so that we will have PoC (Person of Concern) and JoC (Journal of Concern). And we know that our definitions of each tag have to be “robust” and defensible!

My vision of the “Definition of Tags Used to Annotate References” table at the top of each Chapter is that the “Explanation” column will be a brief summary of the criteria, with a link to a page of longer explanations. There will probably be 2 additional columns for each tag: one with the link I just described, and another with a link which will actually display the details of the items comprising the “Occurrences” of that tag in the Chapter being viewed.

Btw, I’ve now seen a “first pass” using the revised tags [currently: NPR, PoC, JoC, Chapter Author, AR4 Author, 2007, Ambiguous, Not Found] – and the pages are looking much tidier!

In the next version we publish, you will also see that our narrative (above table) has been modified accordingly – and now contains an additional paragraph: “We recognize that a computer program’s power is not perfect, and we shall be making provision for users to provide verifiable input for correction to any given ‘tagged’ item.”

baahumbug says:
May 30, 2010 at 3:47 am

I’d like to second Donnas following comments…

“In my view, it’s critical that the criteria used to arrive at the JoC designation be stated clearly. Otherwise, the entire project could be at risk of being dismissed by folks…”

Regarding Peer Reviewed, I’d like you to consider adding something along the lines of the following…

“Some Non-Peer Reviewed items listed do not necessarily need to be peer reviewed. e.g. population statistics. Discretion is left up to the user/reader”

I’m sure better wording can be utilised.

Regards

oggi

hro001 says:
May 30, 2010 at 12:50 pm

Hi Oggi, thanks for coming back :-)

Pls. see my reply to Donna re JoC. As for Non Peer Reviewed, we are currently using the same “definition” that we used in Citizen Audit.

But there is much that can (and probably should!) be said about Non Peer Reviewed in the “longer explanation” page for this tag – as well as much that needs to be said about “Peer Reviewed” [Montford covers this ground quite well, IMHO, in The Hockey Stick Illusion (p.374-383)]

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: