Creating a Spurious Correlation: How The Most Recent 'Wind Syndrome' Study Ditches The Scientific Method

After The Australian broke news, on their front page, of a ‘ground-breaking’ study into ‘wind turbine syndrome’ based at a Victorian wind farm last week, I spent the whole day firing off tweets into the relentless river of social media, picking apart the story.

It's not a good report. It breaks a raft of basic tenets of scientific inquiry, and by doing so, creates a spurious correlation. It's this fictional relationship, between wind turbine operation and a collection of very real reports of ill health, that's caused harm - the propagation of a medical diagnosis that has no backing by the medical profession.

-----------------------------------------------------------------------------------------------------------------------

The one thing that stood tall above the noise was a statement released almost immediately by the six participants in the study (an unusual thing for research participants to do). It was published on the website of the Waubra Foundation – an anti-wind group (that recently lost its ‘health promotion’ charity status):

“Our diaries and the concurrent full spectrum acoustic measurements inside and outside our homes clearly demonstrate that it is the operation of the wind facility correlating with our symptoms”

The article in The Australian, penned by Graham Lloyd, also states that the study conclusively proves a direct causal link between the operation of wind turbines and the ill health experienced by the research participants:

“People living near wind farms face a greater risk of suffering health complaints caused by the low-frequency noise generated by turbines, a groundbreaking study has found….“We now have a basis on how to start the medical studies,” 

…The relationship between turbine operation and sensation demonstrated a “cause and effect”, something Pacific Hydro was not prepared to concede, he [Cooper] said. 

…Mr Cooper was not engaged to establish whether there was a link between wind turbine operation and health impacts, “but the findings of my work show there is something there,””

If the statements here by the journalist and the author of the report are accurate, it means Australia’s operational wind farms are emitting a toxic acoustic signature into the homes and heads of thousands of rural residents, and that this inaudible pollution has a direct physiological impact on these residents.

Every single scientifically sound investigation into this issue has found no relationship between wind farm operation and measured health impacts - the most recent being a Canadian study using ~1,300 households, 17 different models of turbine and a large range of international acoustic and medical experts.

So - does the report support the statements made by residents, the author and the journalist?

The (Extremely Small) Sample Size

The report featured noise diaries taken from six individuals at three residences near the Cape Bridgewater wind farm, over the course of eight weeks. The six participants weren't randomly selected – they were chosen based on their status as complainants, dissatisfied with a number of previous noise studies done by acoustical consultants, some at Cape Bridgewater itself.

This is what’s known as ‘Selection Bias’ – something that peer-reviewed, scientific studies control for by selecting a random, large number of individuals.

If you stepped onto a street, and picked six blonde people, and ignored all others - could you conclude that everyone on the street is blonde? It's an extremely poor way to establish causal linkages - it almost guarantees a spurious linkage.

The "Sensation" Metric

A much-heralded feature of the report was the inclusion of ‘sensation’ reporting, in addition to industry-standard methods of reporting audible noise impacts – diary entries that inform residents what the expected ‘symptoms’ of wind turbine exposure are, prior to the start of the research. As explained in Appendix 1:

Before the study begins, the participants are provided with a suggested list of experiences, each presented as symptoms of being 'subject to operational wind farms'.

Each of these sensations shows up in a study published by Petrie, Faase and Crichton in the British Medical Journal. The research determines the most common symptoms reported by an average population of randomised individuals:

So, how do we know that the sensations reported by the residents wouldn't have been reported regardless of location? There's no control group in the study - each participant was chosen for the fact they're already reporting the health effects being measured and reported by the acoustician.

Without a control, we don't know if we're measuring the signal or the noise - this is basically why the scientific method exists - it's a tool for controlling our own tendencies to see patterns in data. This study rejects that philosophy.

Stationary Turbines Cause ‘Wind Turbine Syndrome’

Lloyd claims in The Australian that “The study by acoustics expert Steven Cooper is the first in the world in which a wind turbine ­operator had fully co-operated and turned wind turbines off completely during the testing” (it's worth noting that this is a standard feature of nearly all acoustic tests of wind farms, and it's been done many, many, many times).

Here's the thing: the appendices of the report show an enormous number of reports of noise, vibration and sensation impacts during the times at which the wind farm isn't generating: whether due to low winds or a site-wide shutdown. Only a few examples are shown below – you can see them all in Appendices 2 and 3.

Blue arrows are noise impact reports, green arrows are 'vibration' reports, and red arrows are 'sensations' - the power output of the wind farm is shown at the bottom of each chart.

There are many, many more examples. That so many 'wind farm impacts' occurred during times at which the turbines weren't spinning ought to indicate that the report isn't quantifying what it thinks it’s quantifying.

The author's response to this criticism was published in a follow-up article by Graham Lloyd in The Australian, and it’s incredible (in the 'this totally isn't credible' sense of the word):

“Some sensations and vibration impact had been reported when the turbines were not operating. But Mr Cooper said this was due to vibration of the blades and towers when they were subjected to wind gusts”

This doesn't explain the large number of 'sensation' reports at low wind speeds, and it isn't mentioned that 'noise' impacts are reported during periods of non-operation as well.

Post-hoc rationalisations are not a good way to do science. By creating this hypothesis, the report is unfalsfiable - all possible measurements either confirm his hypothesis or can be explained away by 'vibrating stationary turbines'.

Reversing the Scientific Method (Conclusion First; Evidence Later) 

The author of the report also declares his analytical philosophy in Graham Lloyd's follow up article:

“The study was required to work backwards from the resident’s observations and see what wind or noise levels agreed with the complaint,” Mr Cooper said. “I don’t think you can get any more objective than that.”

He says pretty much the same thing on page 50 of the full report:

"The use of the diary observation is relevant in that the concept from the outset was not to take the noise levels and then provide a correlation of the noise levels with the diary observations. The approach was to start with the diary observations and then look to see if there was any correlation with the measurement data. 

Hence the use of the visual concept of arrows and sensations overlaid on the data, rather than a statistical analysis of the severity ranking and the data"

It may seem like an irritatingly obvious thing to say, but you can’t establish a causal link by ignoring the ‘misses', highlighting the 'hits' and ditching measures of statistical significance.

Digging deeper - as discussed by Tristan Edis in this article in Climate Spectator, page 115 confusingly outlines the extremely large number of'sensation' reports that were 'excluded' from the analysis:

"From the resident’s diaries there are 441 Sensations classified as severity ranking 4, and 81 as severity ranking 5.....Whilst sensations 4 & 5 would normally be grouped for analysis, as sensation 5 is of a level that would make the specific residents in the study want to leave their premises to obtain respite, the following analysis is based on sensation severity 5 and being the absolute worst case scenario. Noting that the degree of time involved in analysing the data for sensation 4 would be significant"

This is hard to decipher, but I've tried to simplify the remarks below:

It doesn't seem right to reject 84.48% of the recorded data because 'the degree of time involved in analysing the data' would be significant. That's the whole point of the scientific method: taking the time to follow the evidence, wherever it leads.

I've seen confirmation bias in scientific studies before; but this is the first time I've seen it openly championed as an analytical philosophy - only 5.94% of the collated 'sensation' reports ended up being used in the analysis.

The 'Bending Over Exercise'

Another feature of this report that seems to pull it out of the realm of standard acoustic studies and place it deep into 'wind turbine syndrome' territory is section 6.9.4 - disconcertingly entitled 'Bending Over Exercise':

"For the purpose of assisting others in terms of the investigation of balance mechanisms for individuals that are sensitised to the wind farm there was a request to undertake a separate exercise in the hotspots, open areas and in the dwelling where the resident would first stand towards the wind farm close their eyes and observe the perception at that position. 

The resident was then requested to turn through 90° repeat the exercise, turn through another 90° (so as to be facing away from the wind farm) repeat the exercise, and then continue another 90° (to be side on, opposite to the second orientation) and repeat the exercise"

The author goes on to request the residents do the same, but lying on the ground and then, sitting up.

It's admitted that this part of the research isn't 'part of the acoustic study' - certainly, requesting research participants orient physically point themselves at a wind farm isn't a standard part of an acoustic study. Again; there are no controls, no justification for the methodology, and no discussion of confounding factors.

------------------------------------------------------------------------------------------------------------------

The residents involved in this study express a profound sense of relief, in their written statements included in the appendices, and in the remarks they supplied to journalists.

"During the testing program Steven Cooper revealed some of his preliminary findings. The fact that he had established correlations between the residents recorded observations and the operations of the wind turbines, was immensely encouraging, particularly in light of previous assertions by Pacific Hydro that there was no cause for complaint"
- House 89; Resident 5, Appendix Part 3 

“It is an absolute relief, like an epiphany to have him (Mr Cooper) say I was not crazy (that) when I am doing the dishes I feel nausea and have to get out of the house.”
- Sonja Crisp, Research Participant, The Australian 

There is no question that the residents consider the report to be definitive proof of a causal link between the symptoms of 'wind turbine syndrome' (labelled euphemistically by the report author as 'sensations') and the operation of a nearby wind farm.

Our own biases confound our attempts to gain knowledge about the world. We're riddled with cognitive shortcuts that make us see patterns in meaningless noise, or ignore information that threatens our worldview.

The scientific method is a tool for wrangling these biases and cramming them into a box, where they're less liable to lead us to mistakes. Without them, we're at the mercy of the shortcomings of the electrified meat between our ears, whether we like it or not. To not only reject these anti-mistake bias-control tools, but to champion their absence, is the hallmark of a study that is truly non-scientific. To then convince human beings that their ills are caused by a nearby technology, based on this, is deeply unethical.

There's much to explore on this, including a series of credulous, unscientific and error-ridden media reports that accompanied the release, and the history of Cape Bridgewater wind farm's development - something that I know for sure will tell us more about the sentiments expressed by nearby residents than this report. More on that later.