Last Summer I read Ben Goldacre’s Bad Science and I’ve been a bit of a disciple ever since – for those of you who haven’t had the pleasure there is a great 20 minute TED talk right here that is a nice introduction to his philosophy.

He has 3 key principles for spotting “Bad Science”

1) Beware articles that quote “authorities” to make their case – a definite sign that they have no real evidence to support it
2) Beware of studies that are run without control groups – they will almost definitely see a positive result due to placebo effects
3) Beware of collections of studies that show a lopsided range of results – someone is hiding the negative results from you. Basically – Make sure you have ALL the information, not just what other interested parties want you to have.

Since then I’ve tried to use some of those principles to help me make decisions about lifestyle and health etc. What I’ve recently found though is that the same principles apply just as well to the way I do business and the opinions that I hold.

The most important principle that Goldacre higlights is “Make sure you’ve got ALL the information”. It seems obvious, but it is all too easy to just take the data that you see first and run with it and you could end up doing exactly the wrong thing if you aren’t careful

Today I found a live example of study bias in the business world and a timely reminder of how easy it is to manipulate statistics to say what you want.

Yesterday the FSA (Financial Services Authority) released their 6 monthly “league table” of complaints in the financial sector. This first came to my attention whilst reading the Independent and then I also found this article on the mail online (I really started to pay attention when those two agreed with each other!)

Both articles left the reader in no doubt that consumers were still very angry with Banks and that banking services were the worst offender in the financial sector.

I would probably have stopped there (and come to the fairly obvious conclusion that something drastic needs to happen with banking services), were it not for the fact that this story is very pertinent to a client that I am working on at the moment and so I wanted to be able to discuss it with them – obviously I wanted to make sure I had my facts straight and so I dug out the actual report from the FSA.

I suppose I shouldn’t have been surprised by what I found, but I was.

The real headline from the study is that things are actually getting a lot better – complaints about banking services have actually dropped by 22% since last year and a whopping 60% since banking’s nadir at the end of 2009. Also banking is no longer the worst offender – Insurance services now have that honour. My conclusion now would be that we must be doing something right and we should find out what and keep doing it (Effectively the opposite conclusion to the one I would have made had I taken either the Independent or the Daily Mail at face value)

The reason I’m so suprised (or maybe even impressed) is that the media didn’t just exaggerate the findings for effect, they actually made out the opposite of the findings to be true because that was the story they wanted to tell (and the story that might keep their readers nice and angry.) “Banks a bit less shit” isn’t much of a headline I guess, although it would definitely be surprising!

Anyway, maybe this is blindingly obvious to all of you, but I thought it was worth sharing. Question EVERYTHING!

• Uncategorized