Surveys, statistics and statistically significant economic tremors

Once in a while, an article would pop up in the news and make me go, "oh great, here comes another guy who is talking about statistics, but knows nothing about it". This article on the BBC made me feel just like that, but luckily only in the first half: How one woman can cause economic boom or bust. However, having finished reading it, I came to appreciate his point. He is portraying how the world, especially when in crisis like these days, is reacting to 0.1% changes in unemployment rate or deviation from economic forecasts, without fully understanding the data source the conclusions are drawn from, or the statistical significance level it can be trusted to.

The author goes quite the distance to move his reader's emotions, and raise my suspicion:

She (the lady in the fictitious story who just lost her job and by chance was surveyed by the Labour Force Survey) is just one of those surveyed. But Eve, unknowingly, is about to move mountains. She will make economies tremble with a 30-minute interview and a cross in a box on a laptop questionnaire.

Vast sums of money will lurch round the world's financial system. Politicians will reel and businesses be broken.

But then he comes back across the line and is in my good books again:

Check the ONS (Office for National Statistics - UK) and it states clearly that the figure is accurate only to 0.2 per cent, most of the time. This means that a rise of 0.1 per cent in the unemployment rate could be consistent with an actual fall in unemployment across the whole economy of 0.1 per cent.

I like his final point the best, suggesting how people should treat survey results - more like clues, not knee jerk reactions to trigger panics:

... feverish times make attention twitchy. Every piece of evidence about the state of the economy is interpreted, explanations offered, forecasts recalculated, and much is made out of little, perhaps too much.

The difference between a rise and a fall is judged with solemn faces when the truth is the change we observe may not even be there. Economic data is never a set of facts; it is a set of clues, some of which are the red herrings of unavoidable measurement error.

Forecasting or Palm Reading?

A study says that

forecasting “experts” don’t do much better than novices — or, for that matter, guessing — when it comes to predicting the future.

I've been taking a forecasting class, and find the topic rather fascinating. However, as in the course of my study of different forecasting methods, it seems that in a lot of cases the best forecast is the naive forecast - which is simply guessing using current values.

I suppose what it really comes down to is that most things cannot be predicted with great accuracy - how else would we have surprises and excitements in life?! However, it does pay to spend time to analyze information or data that would greatly influence a person/company's decision. If one can find a good forecasting model, then great - life is easy! If not, then at least you know you cannot forecast it well. Move on and don't waste money finding "experts" for forecasts. This is when one needs to consider contingency plans very carefully to deal with variability!

For more information on the study mentioned above: http://blogs.wsj.com/numbersguy/grading-the-forecasts-of-experts-182/?mod=hpp_us_blogs