A lot of science can be relatively quiet, perhaps even staid, but some topics can cause things to kick off. So-called alternative medicine (acupuncture, homeopathy, etc) is one of them. A couple of papers in the British Journal of General Practice have been the cause of no fewer than 10 (count ’em here) letters to the journal, a response by the editor, and even some fisticuffs in The Guardian. Clearly the journal’s editor Dr Robert Jones and UCL’s Prof David Colquhoun will not be exchanging Christmas cards this December. The paper even received the very rare (for a scientific paper) accolade of being satirised in The Daily Mash. To be honest, I’m jealous, The Daily Mash never pays any attention to my papers.

So I read the paper. Basically what the authors of the original paper Patterson et al did is recruit a relatively small number of people, give acupuncture sessions to half of them (randomly selected) and nothing to the other half (they then reversed this but I will just consider the 1st half of their study, the 2nd half does not change the conclusions).

They then asked the people to fill in a questionnaire to judge whether they had got better. For the half who got acupuncture the result on their scale was 3.3 ± 1.3 and for the half that did not the result was 4.0 ± 1.2. Here the scale is that low is good, and the error bars are standard deviations.

Clearly the error bars for the two groups overlap, i.e., some people who were not given acupuncture considered themselves healthier than some given acupuncture. However, on average the people given acupuncture were healthier. The authors argue this is significant. That is maybe pushing things, but you can say that it is probable that on average the group given acupuncture felt healthier. However, I would not bet my shirt on that being true.

So the authors conclude: “A course of traditional acupuncture consultations is therefore a safe and potentially helpful referral option…”. That is not an unreasonable conclusion from their data.

So, why the annoyed letters? Well, there are a couple of obvious problems with the study. The first is how small the effects are. The group given acupuncture were only slightly healthier, so a cynic (e.g., Colquhoun and some of the other letter writers) could reasonably say, what is the point of a intervention that has such a small effect. If you see your GP, you probably don’t want to be told “Well with a bit of luck I can make you about 10% better”.

The other is that, as the authors to be fair admit, the survey tells us absolutely nothing about whether acupuncture, in the sense of sticking needles into what are apparently called your Qi points does any good. The reason is that Patterson compared a group given acupuncture and a nice chat with the acupuncturis, and maybe a cup of tea as well, with a group given nothing.

If you want to test whether sticking needles into your Qi points does any good, you need to compare two groups, both of whom get the chat and cup of tea and a good stabbing, but in only one group is this into the Qi points – and you need to keep it a secret from the patients who is getting stabbed in the Qi point and who is not. All this is required to eliminate the placebo effect. The placebo effect is a very real and it is the most obvious explanation for any difference between the group given acupuncture and the other group.

So, I think you can argue that the work of Patterson et al. dodged what many would consider the most important question: Will stabbing needles into your Qi points cure you of anything? The authors seemed just to care whether the overall sum of stabbing Qi points plus the placebo effect makes a difference. And it seems likely that it does make a small difference.

This is fine but then the number of things that might make you report on a questionnaire that you are a bit better is kind of long: acupuncture, homeopathy, beer, wine, vodka, a good movie, a victory by your team in the upcoming rugby world cup, a nice holiday,… If the NHS is going to pay for one of these, surely it should pay for all of them. In which case I’d like a Brew Dog IPA, a Marlborough Sauvignon Blanc, a victory over South Africa, and a holiday in Tahiti. Thanks.