Diluting the Truth.

Both followers of this blog (Hello Mum!) may remember THIS post, where I discovered homeopaths on THIS website were citing THIS article in the BMJ (De Lange de Klerk et al, BMJ 1994; 309: 1329–1332) that found that “Homoeopathic medicines produce no clinically relevant improvement” as proof that homeopathy works. Maybe they hoped that people would see the title in the citation (Effects of homoeopathic medicines on daily burden of symptoms in children with recurrent upper respiratory tract infections), and assume that paper does indeed support homeopathy.

Let’s remind ourselves of what the take home messages in De Lange de Klerk et al are:

Conclusion : Individually prescribed homoeopathic medicines seem to add little to careful counselling of children with recurrent upper respiratory tract infection in reducing the daily burden of symptoms, use of antibiotics, and need for adenoidectomy and tonsillectomy.Clinical implications

• Some children suffer more and longer episodes caused by upper respiratory tract infections than their peers
• Homoeopathic doctors claim success in the treatment of such children
• In this study the small difference in symptom score found in favour of the homoeopathic medicines was not significant
• Antibiotic use was reduced greatly in both groups, but slightly more in the treatment group
• Homoeopathic medicines produce no clinically relevant improvement in recurrent upper respiratory tract infection

“The small difference in the mean daily symptom scores in the treatment and placebo groups was not significant (P=0.06 or 0.07 after correction for baseline differences).”

So, not a paper that conclusively shows any proof of efficacy for Homeopathy*.

So I thought it might be fun (think what you will of my definition of ‘fun’) to do a little digging and see in what other contexts this particular reference was cited.
According to Scopus our paper is cited in 48 scholarly articles.There are some rather interesting ones in and amongst those 48.
Let’s take a look at three of them.

1) Priorities and methods for developing the evidence profile of homeopathy Recommendations of the ECH General Assembly and XVIII Symposium of GIRI

M. Van Wassenhoven

Homeopathy
Volume 94, Issue 2, April 2005, Pages 107-124

“Nevertheless RCTs have been performed and meta-analyses of these studies have concluded that there is clear evidence of efficacy in favour of homeopathic treatments that cannot be attributed only to a placebo effect…. In children, using comparison groups, significant results have been obtained for recurrent infections (my bold), postoperative agitation, adenoids, otitis media, and stomatitis in patients with cancer (complication of conventional treatment). There are promising pilot studies on low back pain, premenstrual syndrome and chronic fatigue.^{140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452 and 453}“

Our reference is number 206 in that lot. And our reference was significant all right – if I can just remind you – “Homoeopathic medicines produce no clinically relevant improvement in recurrent upper respiratory tract infections” & “The small difference in the mean daily symptom scores in the treatment and placebo groups was not significant (P=0.06 or 0.07 after correction for baseline differences).” – this would appear mutually exclusive with “clear evidence of efficacy in favour of homeopathic treatments that cannot be attributed only to a placebo effect”. Hmmm.

(and no, I haven’t gone through each and everyone of those references)

2)The research evidence base for homeopathy: a fresh assessment of the literature

RT Mathie

Homeopathy
Volume 92, Issue 2, April 2003, Pages 84-91

In this paper, our reference, (55) is cited in table 1 in a group of 47 papers that “observed a homeopathic effect superior to placebo.” Just remember this is a paper that clearly states “The small difference in the mean daily symptom scores in the treatment and placebo groups was not significant (P=0.06 or 0.07 after correction for baseline differences).” Confused? You should be. 3) Antibiotics and the development of resistant microorganisms. Can homeopathy be an alternative?

Petter Viksveen

Homeopathy
Volume 92, Issue 2, April 2003, Pages 99-107

This paper states that “Results in a double-blind randomised placebo-controlled trial of children suffering from upper respiratory tract infections were in favour of homeopathy (P=0.06).[de Lange de Klerk E, et al. Effects of homoeopathic medicines on daily burden of symptoms in children with recurrent upper respiratory tract infections. BMJ 1994; 309: 1329–1332.68] The conclusion of this study was that individually prescribed homeopathic medicines seem to add little to careful counselling of children with recurrent upper respiratory tract infection in reducing the daily burden of symptoms, use of antibiotics, and need for adenoidectomy and tonsillectomy.”

Contrast this with the original paper: “The small difference in the mean daily symptom scores in the treatment and placebo groups was not significant (P=0.06 or 0.07 after correction for baseline differences).”

Once again – the original paper contends that the differences between placebo and homeopathy were not significant – Viksveen says otherwise, but then in the sentence after that, quotes verbatim the negative results for homeopathy in the study. WTF?

Is “mis-citing” a word?

The vast majority of the 48 papers that cited De Lange de Klerk et al, cited it correctly – to indicate that this particular study has not shown that homeopathy is really effective beyond placebo, or more research is needed in this area. The three I have gone into more detail about have (pretty much) taken the exact opposite conclusions from the paper.

The eagle-eyed amongst you will have spotted that all 3 of these papers come from the Journal “Homeopathy”, the organ of the Faculty of Homeopathy. The members of the editorial board / advisory board of Homeopathy can be found here.

But wait? What’s this? Of our three papers who cite De Lange de Klerk et al as positive proof for the efficacy of Homeopathy above that of placebo effect, 2 are authored by members of the Homeopathy editorial board? Shurely Shome Mishtake?
(Michel van Wassenhoven – on the Editorial Advisory Board, RT Mathie on the Editorial Board).

The author of the 3rd paper, Petter Viksveen is a practising homeopath, working in Norway, and a graduate of the fine MSc Masters Degree in Homeopathy, which was until recently awarded from the University of Central Lancashire. Anyone interested in the course materials for that degree course can of course see them on David Colquhoun’s excellent blog.

The fact De Lange de Klerk et al has been erroneously cited 3 times by the same journal demonstrates a failure of the individual authors concerned to correctly reference and source their work, and a repeated failure of the reviewers and editors of Homeopathy to spot such an obvious error. I leave the reader to make his/her own conclusions about the quality of the peer-review process at Homeopathy.

It is interesting to note that publishing house Elsevier (who publish and distribute Homeopathy) have signed up to the committee for publication ethics, COPE.

Obviously, as a concerned scientist I have contacted COPE about this matter, for their advice on how to get these errors corrected.In Conclusion…

Of all the qualities a good scientist must have, intellectual honesty and rigour are paramount.
Any scientist worth his Natrum muriaticum would have at the very least, skimmed though any paper he is citing – certainly he will be aware of the major conclusions that a paper draws. To do otherwise is neither intellectually honest nor intellectually rigorous.
Given the format of De Lange De Klerk et al, even when viewed alone the abstract gives a really big hint as to the conclusion of the paper:  (hint – look at the words after “Conclusions”) – it is really exceptionally difficult to avoid the conclusions this paper makes.

I cannot conceive of a situation in which a professional scienist could cite a paper to support a hypothesis which is the opposite of the conclusions of the cited paper, without being either woefully negligent or intentionally duplicitous.
Given the credibility problems that homeopathy has in the scientific community, if homeopaths are to obtain evidence that is irrefutable and universally accepted, their actions must stand up to close scrutiny – after all “remarkable claims require remarkable evidence”.
The examples above fail to stand up to such scrutiny. I might add that it did not take me very long to find this out – I have looked at 1 paper, cited 48 times and found 3 egregious errors. Have I been lucky enough to stumble upon the one paper that has been so erroneously cited? Or are there more examples out there?

—-
*I am not commenting on the accuracy of De Lange de Klerk et al (there are a couple of letters to the BMJ from Peter Fisher amongst others complaining about the stats/methodology in this paper) – that is NOT the point I am making here. The fact is that this paper clearly and unequivocally concludes that “Individually prescribed homoeopathic medicines seem to add little to careful counselling of children with recurrent upper respiratory tract infection in reducing the daily burden of symptoms, use of antibiotics, and need for adenoidectomy and tonsillectomy.” The paper has NOT been retracted, neither has anyone published a re-analysis the data, as far as I can see. If either were the case, one would cite the re-analysis or the retraction – not the original paper.