Monday, December 14, 2009

Pollytics: Data, fiction and politics.

Possum has written a brilliant post over at his Pollytics blog. I won't spoil it for you but here is a quote.


That linear regression is statistically significant to so many decimal places that I had to go into the guts of my stats software to find a value that wasn’t zero. The p-value of the linear regression is 0.000001 (rounded up), meaning that there is less than a 1 in one million chance that the long term increase in demand we are witnessing could be due to chance alone. It’s a slow grinding increase, but a consistent one regardless of the local, short term variation involved.
If you were a government facing that kind of data, there would be absolutely no hesitation at all, on either side of politics, to plan for a future which contains that increasing demand for this particular service. There might be political differences over how to do it, maybe even on what ought to be done – but there would be universal acceptance of the empirical reality of increasing demand, albeit one that contains significant variation at any given time. Similarly, if you were a business or an industry facing that sort of data on the demand for your own product, there would be universal action to deal with it. In fact – you would struggle to find anyone that would suggest such data doesn’t have an observable and statistically significant linear trend over time strong enough to warrant acting on.
Yet – the data above isn’t actually polling data about demand for a government service...
All content that comes from Wikipedia is used under license. Terms available here