CBS: East Anglia CRU covered up bad data, computer modeling

posted at 2:55 pm on November 24, 2009 by Ed Morrissey

Declan McCullough dives into the East Anglia CRU exposure and delivers a well-researched and fair look at the controversy for CBS News.  McCullough looks at the various e-mails, including portions that have not yet gotten much attention from the media, and concludes that the CRU has acted without transparency.  He also shows why the data itself has become suspect, as well as the modeling on which anthropogenic global-warming activists rely for stoking public demand for action:

As the leaked messages, and especially the HARRY_READ_ME.txt file, found their way around technical circles, two things happened: first, programmers unaffiliated with East Anglia started taking a close look at the quality of the CRU’s code, and second, they began to feel sympathetic for anyone who had to spend three years (including working weekends) trying to make sense of code that appeared to be undocumented and buggy, while representing the core of CRU’s climate model.

One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: “I feel for this guy. He’s obviously spent years trying to get data from undocumented and completely messy sources.”

Programmer-written comments inserted into CRU’s Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: “Apply a VERY ARTIFICAL correction for decline!!” and “APPLY ARTIFICIAL CORRECTION.” Another, quantify_tsdcal.pro, says: “Low pass filtering at century and longer time scales never gets rid of the trend – so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!”

While much of the attention has focused on CRU director Phil Jones and his messages about “hiding the decline,” McCullough focuses more attention on this:

I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation – apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective – since we’re using an off-the-shelf product that isn’t documented sufficiently to say that. Why this wasn’t coded up in Fortran I don’t know – time pressures perhaps? Was too much effort expended on homogenisation, that there wasn’t enough time to write a gridding procedure? Of course, it’s too late for me to fix it too. Meh.

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that’s the case? Aarrggghhh! There truly is no end in sight… So, we can have a proper result, but only by including a load of garbage!

One thing that’s unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country’s met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!

Knowing how long it takes to debug this suite – the experiment endeth here. The option (like all the anomdtb options) is totally undocumented so we’ll never know what we lost. 22. Right, time to stop pussyfooting around the niceties of Tim’s labyrinthine software suites – let’s have a go at producing CRU TS 3.0!since failing to do that will be the definitive failure of the entire project.

Ulp! I am seriously close to giving up, again. The history of this is so complex that I can’t get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections – to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?…

If that looks like the CRU relied on unreliable data sets, bad computer models, and a desire to reach a conclusion rather than do actual science, you’re not alone.  McCullough concludes:

The irony of this situation is that most of us expect science to be conducted in the open, without unpublished secret data, hidden agendas, and computer programs of dubious reliability. East Anglia’s Climatic Research Unit might have avoided this snafu by publicly disclosing as much as possible at every step of the way.

Most critics of AGW advocacy have never considered it science in the first place.  This just shows that we were right; it’s a religious belief, and its high priests apparently have few scruples about rigging the models and the data to show what they want, rather than pursue science and the scientific method.  Be sure to read all of Declan’s lengthy and excellent article.

Now — when will CBS put this on their Evening News?  Or will they have to bump this for Katie Couric’s Poetry Corner?

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Trackbacks/Pings

Trackback URL

Comments

Comment pages: 1 2 3

How many Pounds, Euros and Dollars were invested in this travesty ?

J_Crater on November 25, 2009 at 3:18 PM

I strongly suggest you read the original code documents, esp. “HARRY_READ_ME”. All the links are at WattsUpWithThat.com

It’s not too difficult to understand, even for someone with scant computer experience like me… It’s a poor gov’t programmer saying, ‘What the f*** do I do with all these bogus datasets?? There’s no way to match them up! We’ll have to ignore the entire period before year 1400, and stop plotting at 1960 to hide the decline…’

It’s one thing to read the CBS story, but a whole other thing to read the words of the programmer actually charged with the task of cooking the data.

Ralph64 on November 25, 2009 at 5:34 PM

These immoral, unethical thieves in lab coats cost the public more than 50 Bernie Madoffs. This is Lysenko-level scientific malfeasance. The frauds deserve jail time, and the fools who believed their alarmism deserve scorn.

theCork on November 25, 2009 at 5:46 PM

Whoever had the FOIA request in for this data needs to go to court immediately and insist on getting it from the CRU. Just to make them do it and to make sure it can be used in court.

And the UK government needs to drop a very big hammer on them.

darwin-t on November 25, 2009 at 6:32 PM

So far the left-wing media is not even covering this story and even Fox News Tonight is treating it like the Valerie Plame fiasco: “Oh, well. Never mind!”

BHO is even traveling to Copenhagen to lend his diminished credibility to the cause–Devine Leftist that he is.

This needs to stay on the front burner turned up very hot.

I know we can count on MM to do so!

Stepan on November 25, 2009 at 6:41 PM

This needs to stay on the front burner turned up very hot.

Amen, Stepan.

And by MM, I know you don’t mean MainstreamMedia!!

We’ve gotta reiterate this story after the holiday, in a big way.

Ralph64 on November 25, 2009 at 6:57 PM

60 teachers at my school are getting the AWG hoax message LOUD and clear. Thanks HotAir.

*Precious Pup laugh* hee, hee, hee, hee, hee…

Mojave Mark on November 25, 2009 at 7:36 PM

I want clever T-shirt sayings!!

“So if you believe in AGW if have some greenhouse gas for you”

barnone on November 24, 2009 at 4:03 PM

AGW. Who’s the bitter clingers, now, Sport

darwin-t on November 25, 2009 at 10:19 PM

Hey, climate fraud’s breakin’ out all over!

New Zealand climate agency accused of data manipulation

Django on November 26, 2009 at 12:15 AM

Most critics of AGW advocacy have never considered it science in the first place. This just shows that we were right; it’s a religious belief, and its high priests apparently have few scruples about rigging the models and the data to show what they want, rather than pursue science and the scientific method.

Color elgeneralisimo as one of those not so surprised by this latest revelation…

The problem with all of the “Death by Global Warming/CO2? models thus far is that the underlying assumptions tend to vary from a bit sketchy to downright dodgy. Invariably, the math used in the models is to the laws of thermodynamics as Enron’s bookkeeping was to standard accounting practices…

elgeneralisimo on March 22, 2007 at 1:10 AM

elgeneralisimo’s AGW explained for the masses…

You = everyone living on the planet
Cop = IPCC, Al Gore, et alia
Posted speed limit = known laws of the universe

You are driving down a road in your car.
The posted speed limit is 40 mph.
You are traveling at 40 mph.
Cop pulls you over.

You: Is there a problem ?

Cop: You were going 40 in a 35.

You: But the sign says 40.

Cop: Theoretically it could say 35.

You: Uhm… I suppose it could, but in reality the sign says 40 and I was traveling at 40.

Cop: Step out of the vehicle.

You: Why ?

Cop: You are under arrest for denying a theoretic reality.

You: What ?

Cop: Due to your unwarranted skepticism, I will be be required to use force.

You: WTF ???

Cop: Remember, you brought this upon yourself.

You: What are you tal…

You are interrupted mid sentence by 75,000 volts…

elgeneralisimo on November 26, 2009 at 11:33 AM

http://gatewaypundit.firstthings.com/

Holdren connected to ClimateGate….

nondhimmie on November 26, 2009 at 1:06 PM

The “hide the decline” in Phil Jones’ Nov. 16, 1999, e-mail referred to Mann’s 1998 temperature chart that hid the declining reliability of tree-rings after 1960. Mann’s chart apparently shows temperatures from 1900 to 1960, as reflected in tree ring studies, and then tacks on actual thermometer readings for the years 1961 to 1998. Both Mann and McIntyre said this yesterday. According to skeptic McIntyre, the “hiding” was not exactly an exercise in deception, but McIntyre also said Mann has not yet fully explained why tree rings were a good measure until 1960, but a bad measure after 1960.

Meanwhile back at the White House we find out that

Barack Obama’s radical socialist climate czar Carol Browner on Wednesday rejected claims that e-mails stolen from a British university show climate scientists trumped up global warming numbers, saying she considers the science settled.

.. but in one of those “shades of Haliburton” moments, we find out that she …

was a board member of one of the leading carbon offset trading companies, APX.

J_Crater on November 27, 2009 at 11:36 AM

The attempts to marginalize dissenters sounds like a tactic straight out of the Saul Alinsky playbook.

4thQTR on November 27, 2009 at 11:49 AM

Comment pages: 1 2 3