Time to ignore all surveys

Dear All

We all read the articles and editorials based on these proclaimed HUGE surveys...Richard Smith has some important points to share which we should all consider before reaching a conclusion.

Regards

Farooq

http://blogs.bmj.com/bmj/2009/08/12/time-to-ignore-all-surveys-says-richard-smith/#comment-4643

Time to ignore all surveys, says Richard Smith

12 Aug, 09 | by BMJ Group

Richard Smith Recently in Bangladesh I had breakfast with a Harvard professor of economics who told me: “Economists pay no attention to what people say, only to what they do.” Now I know, as we all do, that there is a big gap between what people say and what they do, and consequently I’ve always been wary of surveys; but we did publish them in the BMJ when I was the editor. The more I think about it, however, the more I think that we should ignore all surveys. Life is too short.

And the gap between what people say and what they do is not the only problem with surveys. There are hundreds of problems, but I’ll pick out two that render meaningless stories that appear almost every day in the media—because silly surveys are important for filling newspapers and even more important for public relations people and editors of small time publications to create stories from nothing.

Firstly, these publications stick a questionnaire into their pages and ask readers to respond. Or you can do the same thing online. It’s cheap and easy. Probably less than 1% respond, but if you have a large circulation that can be a lot of individuals. Needless to say, the 1% are probably wildly different from the 99% who didn’t respond.

This is, I think, what must have happened with the survey by GP magazine that led to this report in The Times: “A survey of 251 family doctors, published today, shows that 90 per cent believe that other diseases risk going undetected because their symptoms are similar to those caused by swine flu.” The circulation of GP Magazine is probably about 40 000—so those 251 doctors constitute a response rate well below 1%. The results are meaningless.

The second trick is, of course, in the questions. For example, here’s one that’s a favourite of the BMA, the British Misery Association: “Have you ever contemplated leaving the NHS?” Well, if you’ve been a doctor for 25 years you’d be odd if you hadn’t.” The result is a headline that says: “Two thirds of doctors ready to quit NHS.” Another common variant is: “Have you ever thought of moving to Australia?”

And sex is of course the best topic to survey in order to get coverage. I’m thinking of surveying BMJ editorial staff asking: “If you woke up to find yourself in bed with two sexually gorgeous people would you get immediately get out?” The resulting headline, perhaps in The Lancet, would read: “50% of BMJ editors ready for three in a bed romp.”
Next time you see the results of a survey, cast a cold eye on it and move on

Posted by: drrathorePosts: 428 :: 14-08-2009 :: | Reply to this Message