An audit of uncertainties in the HadCRUT4 temperature anomaly dataset plus the investigation of three other contemporary climate issues

McLean, John D. (2017) An audit of uncertainties in the HadCRUT4 temperature anomaly dataset plus the investigation of three other contemporary climate issues. PhD thesis, James Cook University.

PDF (Thesis)
Download (2MB) | Preview
View at Publisher Website:


This thesis is in two parts. The first discusses many uncertainties associated with the widely used HadCRUT4 global temperature dataset. The second part deals with three other issues in climate science, viz (i) the possible relationship between cloud cover and global average temperature; (ii) a better indicator of the El Nino-Southern Oscillation; and (iii) the probability of severe coral bleaching along the Great Barrier Reef prior to 1998.

Part 1

The HadCRUT4 dataset provides information about temperatures over as much of the Earth's surface for which historical temperature recordings are available. It is regularly cited by government authorities and by organizations such as the Intergovernmental Panel on Climate Change (IPCC) and the United Nations Framework Convention on Climate Change (UNFCCC) and yet very little appears in the scientific literature about its errors and uncertainties other than from within the organizations responsible for its collation. Authors of the IPCC's Fifth Climate Assessment Report (2013) admitted during the review process for that report that no audit of the HadCRUT4 dataset or any associated dataset had been undertaken.

Given that governments are deciding energy and climate policies on claims based on the HadCRUT4 dataset, an independent audit of its accuracy and uncertainties was undertaken using data from 1850 to 2015. The audit covers a broad range of issues but leaves the quantifying of the impact of such errors to others, save for some general comments about the direction of changes in error margins.

Bespoke software revealed many areas of concern. Data coverage was found to vary between 12% and 91% of the Earth's surface, which means that any declared "global average" assumes that temperatures over the part of the world for which no data was available were identical to that average. It also assumes that meaningful trends can be calculated even though the coverage varies. Also revealed was that when coverage was low it was found to be concentrated on particular regions, meaning for example that less than 13% of a hemisphere might account for more than 60% of the data coverage for that hemisphere in a given month.

Sample size was another problem. A single observation station in the entire Southern hemisphere reported data in the first three years of the HadCRUT4 record and only nine were reporting by 1859. The data is processed and presented on a monthly basis as values for each grid cell, each of which covers 5° latitude x 5° longitude. More than 30% of the grid cells derived from sea surface temperature do so from 1850 to about 1950 on the basis of from one to five measurements in the entire month.

Outliers were also discovered in the data, even in the 30-year period from 1961 to 1990 over which long-term average temperatures are calculated and sometimes in the longer period over which standard deviations were calculated. Their presence in these periods firstly widens the range of acceptable values (i.e. includes the inclusion of other outliers) and secondly distorts the crucial long-term average temperatures.

Numerous other inconsistencies and uncertainties were identified, including differences between dataset values that should be identical, unexplained differences between land and sea temperatures, inconsistent sources of coastal temperature data, questionable "bulk adjustment" to sea surface temperature data, poor data quality control and possible errors in the processing of land and sea temperature data prior to their submission for inclusion in the dataset.

Over 25 findings are presented in the summary to part 1of the thesis, the most serious being that HadCRUT4 global averages prior to 1950 are of limited value because for almost all of the period from 1850 to 1950 the coverage of the Earth's surface was less than 50%. The summary also proposes how a new historical dataset with fewer uncertainties and inconsistencies might be constructed from some but not all of the existing data.

Part 2

Three topics are covered in this section. The first is a published paper that attributes the pattern in HadCRUT4 global average temperature anomalies since 1950 firstly to a change in the El Nino-Southern Oscillation after 1977 from having few El Nino events to having many, then from 1987 to 1997 to a general reduction in cloud cover. Since 1997 the trend in global average temperature is minimal but low-level cloud cover has decreased while mid to upper level cloud cover have correspondingly increased.

The second topic focuses on an alternative to the commonly used Troup Southern Oscillation Index (SOI), which is used to indicate the state of the El Nino-Southern Oscillation (ENSO) and is based on sea-level pressure (SLP) at just two locations - Darwin (Australia) and Tahiti (French Polynesia). It proposes that an ensemble index be created from data at six other locations, three in the eastern Pacific and three in the western Pacific or further west. The ensemble index is shown to have three significant advantages. Firstly, it provides better coverage of the entire region where mean sea level pressure is directly related to ENSO conditions. Secondly, and somewhat related, is that data noise such as that caused by small-scale weather conditions at one or two locations is less likely to have a major impact on the ensemble SOI than in the Troup-SOI. Thirdly, and less significant, it is refining an existing index rather than introducing a new one.

The third topic addresses the often-repeated claim that severe and extensive coral bleaching on Australia's Great Barrier Reef (GBR) driven by warm water first occurred in 1998. Regular aerial surveys of the GBR did not commence until the late 1990s so to address the question of earlier severe bleaching four different techniques are used.

The first approach uses statistical and probability argument based on known recent bleaching events and Willis Island temperature data to conclude that the probability of bleaching in one or more years of the period from 1922 to 1997 is at least 0.6.

The second approach considers previous instances of El Nino events during summer months, such events being related to bleaching overseas and along the GBR in recent years. El Nino events during eighteen summer months over the period 1939 to 1997 were identified, the strongest occurring in 1983 when extensive coral bleaching was reported in the eastern Pacific near Panama and in waters off Indonesia.

The third considers the number of warm summer days and the maximum length of sequences of warm days according to temperature records from the observation station on Willis Island, about 350 km east of the GBR. Warm summers prior to 1998 include those which ended in 1944 and 1964, the latter coming immediately after a short El Nino event that weakened in January of that year.

The final approach was to analyse sea temperature records in the ICOADS database that applied to summer and the GBR, with the data divided into three bands, each spanning 5° of latitude. An examination of the proportion of warm days revealed that of the 37 observations made in the northern third of the reef during March 1970 33 include reports of temperatures ≥29°C and 25 of temperatures ≥30°C. Sequences of warm days were also examined, this revealing sea temperatures ≥30°C on nine successive days in the middle third of the reef commencing 25 December 1923 and seven successive days in the northern third of the reef in late March 1983.

Contrary to various claims, severe coral bleaching on the GBR prior to 1998 seems likely with a probability of ~0.6 that bleaching occurred in one or more summers from 1922 to 1997, with the other methods suggesting it was most likely in the summers ending in 1983 and 1963.

Item ID: 52041
Item Type: Thesis (PhD)
Keywords: anomalies, climate change, ENSO, HadCRUT4, insolation, ISCCP, temperature shift
Date Deposited: 17 Jan 2018 02:47
FoR Codes: 04 EARTH SCIENCES > 0401 Atmospheric Sciences > 040105 Climatology (excl Climate Change Processes) @ 33%
10 TECHNOLOGY > 1006 Computer Hardware > 100602 Input, Output and Data Devices @ 34%
04 EARTH SCIENCES > 0401 Atmospheric Sciences > 040104 Climate Change Processes @ 33%
SEO Codes: 89 INFORMATION AND COMMUNICATION SERVICES > 8902 Computer Software and Services > 890205 Information Processing Services (incl. Data Entry and Capture) @ 50%
96 ENVIRONMENT > 9603 Climate and Climate Change > 960304 Climate Variability (excl. Social Impacts) @ 50%
Downloads: Total: 5664
Last 12 Months: 110
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page