The incredible story of the PACE trial for ME/CFS | Part 2
Chronically ill citizen-scientists and an intrepid journalist expose the PACE trial, a court battle, angry disputes between scientists, and top UK medical institutions fight turn against each-other.
The story continues from part 1.
2015: The Institute of Medicine report and Trial by Error
In 2015 two publications changed the way the medical world thought about ME/CFS. One was a scientific report produced by the US Institute of Medicine (now the National Academy of Medicine) at the request of several US government departments, and the other was a blog post.
The Institute of Medicine report, ‘Beyond Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: Redefining an Illness’ stated bluntly that ME/CFS is not a psychiatric or psychological illness, and is in fact a very serious and debilitating chronic medical condition. The report set out new diagnostic criteria, which require the presence of post-exertional malaise for diagnosis.
The report also suggested a new name for the illness: Systemic Exertion Intolerance Disease (SEID). That name didn’t catch on, but the picture of ME/CFS as a very serious chronic illness did, and sent ripples through the medical community in the US and worldwide.
Trial by Error
Also in 2015, US journalist and public health expert David Tuller, who has a friend with ME/CFS, did some in-depth reading about the PACE trial. He was astonished to discover the large and detailed body of scientific criticism of it which had been created by the ME patient community. Speaking in 2018, he said:
I kind of at first was reading, people are recovered at baseline on measures? The patients really have to be wrong, that can’t be possible, no-one would publish such a paper. Well, then you look; oh my god, they did.
Tuller believed that the patients were right: the PACE trial was bad science. But before publicly criticising such an important scientific work, he wanted to double check that it was really as bad as he thought it was. He brought his concerns to several of his colleagues, who were experts in the biomedical sciences. All of the people he consulted, after learning the details of how the PACE trial had been conducted, agreed that it fell short of the standards required for a phase three clinical trial. One of these colleagues was the biostatistician Bruce Levin, an expert in the design and interpretation of clinical trials, who later gave a lecture about the PACE trial titled ‘How NOT to Conduct a Randomized Clinical Trial’.
Bolstered by his colleagues, Tuller wrote a three-part article titled ‘Trial by Error’, in which he explored the methodological problems with the PACE trial. He struggled to find a place to publish it – the article was too long for a newspaper or magazine, too short to be a book, and since it took a wide-ranging look at the social and political situation surrounding the PACE trial, it wasn’t suitable for a scientific journal.
Eventually ‘Trial by Error’ was published in three parts on Virology Blog, a site hosted by Tuller’s friend, the virologist Vincent Racaniello. Virology Blog is a long-running, well-regarded and widely-read science blog – it was probably the perfect home for Tuller’s in-depth articles.
The original version of ‘Trial by Error’ contained typographical errors; a cleaned-up version can be found at: Trial by Error: The Troubling Case of the PACE Chronic Fatigue Syndrome Study.
‘Trial by Error’ became the most-read post ever on the popular Virology Blog, and sparked a flurry of discussion on social media, on other biomedical blogs, and eventually in formal scientific publications.
Further reading:
• PACE study and the Lancet: Journal reputation is a two-way street, Statistical Modeling, Causal Inference, and Social Science
• Hope for Chronic Fatigue Syndrome, Slate
• Worse Than the Disease: Curing Chronic Fatigue, Undark
• Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study, Mind the Brain, archived by WaybackMachine
• My thoughts about the PACE trial, Lucibee’s blog
• The PACE Trial Invalidates the Use of Cognitive Behavioral and Graded Exercise Therapy in Myalgic Encephalomyelitis/ Chronic Fatigue Syndrome: A Review, Journal of Neurology and Neurobiology
• To keep science honest, study data must be shared, STAT
• The PACE trial missteps on pacing and patient selection, Journal of Health PsychologyNot everyone was a fan of ‘Trial by Error’, though. Shortly after its publication, Vincent Racaniello, the owner of Virology Blog where ‘Trial by Error’ was published, received several emails from a public relations manager at Queen Mary University of London, where chief PACE trial investigator Peter White worked. The PR manager asserted that ‘Trial by Error’ contained misinformation and inaccuracies which had led the trial’s authors to be targetted for online abuse, and which could cause a considerable amount of reputational damage. Because of this, the PR manager argued that the PACE trial authors should be allowed to post a rebuttal on Virology Blog. The rebuttal was duly posted.
Scientists ask The Lancet for an independent review of The PACE trial
Out of the flurry of discussion on the PACE trial that occurred in the wake of ‘Trial by Error’, a group of six scientists wrote a letter to Lancet editor Richard Horton urging him to order an independent review of the PACE trial.
The scientists received no response, apart from an email from Richard Horton’s office saying that he was away and would reply once he got back. Three months later the same letter was sent to The Lancet again, this time signed by three dozen scientists. Again there was no response. Two years later, a similar letter was sent to Richard Horton once again, this time signed by ninety-four scientists, medical doctors, and other experts. A couple of months after that, the letter was sent once again, this time with ten UK members of parliament and thirty ME patient support and advocacy organisations from around the world added to the list of signatories. All of these letters were ignored by The Lancet.
Petition
In March 2016, the campaign group ME Action sent a petition, signed by 11,000 people, to The Lancet, calling for a retraction of the PACE trial.
The data
The PACE trial authors did not include the data from the trial in their scientific reports. I was shocked when I first learned this, but apparently this is a somewhat normal thing for scientists to do: they performed calculations on their data and published the results of those calculations, without revealing precisely what calculations had been performed.
In the UK, as in the US, government-funded institutions are legally required to share information they hold with anyone who makes a freedom of information (FOI) request for it. The data was held at Queen Mary University of London (QMUL), which receives funding from the UK government, so FOI law applied, and QMUL received a large number of FOI requests for data from the PACE trial. However the university refused all of these requests, either on grounds of concern for the privacy of the trial participants, or because the requests were deemed to be ‘vexatious’. ‘Vexatious’ is a legal term meaning that those who requested the data didn’t have a legitimate reason for doing so.
Following ‘Trial by Error’, ME/CFS advocacy groups all over the world issued calls for the data to be released.
Alem Matthees’ legal battle over the PACE trial data
One of the people whose FOI request had been refused by QMUL was Alem Matthees, a young Australian man with ME/CFS. Matthees did not let the matter rest; he wrote to QMUL and requested an internal review of the decision. QMUL duly carried out an internal review, and upheld the decision not to release the data. Undeterred, Matthees studied the relevant UK legal procedures and wrote to the Information Commissioner to complain about the decision. The Information Commissioner agreed with Matthees’s complaint, and ordered QMUL to release the data. QMUL appealed, taking the matter to an Information Tribunal. Around this time, some of the PACE trial investigators began lobbying MPs to change the law so that researchers would not have to release their research data to the public.
Matthees studied and prepared intensely for the three-day tribunal hearing, which was held in April 2016. At the tribunal, QMUL made the extraordinary argument that releasing the data could lead members of the ME/CFS community to commit violence against the PACE researchers, or against people who had been participants in the PACE trial. While Matthees had only requested anonymised data, QMUL argued that hostile activists might get access to people’s hospital records with the help of an NHS insider, allowing them to determine the identities of people who had participated in the PACE trial for nefarious purposes. This was, as the tribunal judges noted in their decision, completely made-up and lacking any basis in reality whatsoever. Matthees won the appeal. QMUL had spent £246,000 of UK tax payers’ money on the legal battle to avoid releasing the PACE trial data.
There is a somber epilogue to this story: in the months following his victory, Alem Matthees’s ME/CFS became much more severe. He became bedbound, and remains so to this day. He is cared for by his mother.
Further reading:
• Tribunal was right to order release of chronic fatigue trial data, The Conversation
• Major breakthrough on PACE trial, Citizen Network
• A mother's story: Alem Matthees, ME Australia
• Information tribunal, MEpedia
• The tribunal decision, official court documentReanalysis
Once the data was released, a group of science-savvy ME advocates who had been examining and criticising the PACE trial’s methodology for years by this point; Alem Matthees, Robert Courtney and Tom Kindlon; teamed up with ‘Trial by Error’ author David Tuller, psychology researcher Caroline Wiltshire, and biostatistician Bruce Levin to reanalyse the data. They found that, when the data was analysed according to the PACE trial’s original protocol, CBT and GET did not, in fact, lead to any significant improvement. The reanalysis was published in the journal BMC Psychology.
Further reading:
• The PACE Trial: How a Debate Over Science Empowered a Whole Community, Carolyn E. Wiltshire, ME AssociationJournal of Health Psychology special issue on the PACE trial
In August 2017 the Journal of Health Psychology published a special issue which focused entirely on the PACE trial, and was strongly critical of it. The press release for the special issue included this quote:
The PACE trial debate illustrates what can happen when researchers become entrenched in a particular point of view, and fail to engage in constructive exchange with critics and stakeholders.
It reveals an unwillingness of the Co-Principal Investigators of the trial to engage in authentic discussion and debate. It leads one to question the wisdom of such a large investment from the public purse (£5million) on what is a textbook example of a poorly done trial.
Three board members at the Journal of Health Psychology felt that the special issue was unfair to the PACE trial’s authors, and resigned in protest.
Further reading:
• Special issue on the PACE Trial, Journal of Health Psychology
• 'You’re a disgusting old fart neoliberal hypocrite' – scientists in furious row over ME study, Inews, 2018 Parliamentary debate on the PACE trial
In February 2018 a debate on the PACE trial was held in the UK House of Commons. It was an unusual ‘debate’ in that there was no disagreement; everyone who spoke agreed that the PACE trial was bad science which harmed people with ME/CFS. One after another, MPs reported on the various ways in which the PACE trial had affected the lives of their constituents. Several described how people with ME/CFS had been dismissed by doctors who did not believe they were seriously ill, or who believed that they could get better if they really wanted to. One constituent who had been a participant in the PACE trial had reported to her MP that GET had in fact made her illness more severe, yet to her shock the people running the trial had failed to record and report her deterioration. There was a heart-rending story of a twelve-year-old child whose ME/CFS had become more severe after a course of GET. The PACE trial was described as “a miscarriage of justice for patients” and as “the greatest scientific scandal of the twenty-first century”.
Medical institutions cease recommending CBT and GET for ME/CFS
From 2017 onwards, various medical institutions, including the CDC, Kaiser Permanente, Mayo Clinic, NICE, and UpToDate, removed their recommendations for CBT and GET as treatments for ME/CFS. Instead they recommended pacing, an illness management strategy whereby the patient carefully spaces out activity with periods of rest in order to avoid triggering post-exertional malaise and relapse.
Further reading:
• CDC Removes Reference to Disputed ME/CFS Therapies From Website, Undark
• Kaiser Permanente Changes Course, Trial by Error
• Re-envisioning treatment plans for ME/CFS, Permanente Medicine
• Mayo clinic removes GET and CBT from CFS webpage, ME ActionSilencing critics
Disturbingly, it seems that there have been attempts to intimidate and silence those who wished to speak out against the PACE trial.
A medical researcher, Emma Reinhold, posted criticism of the PACE trial on Twitter. Shortly after, the Royal College of General Practitioners contacted her to say that they had received complaints about her and were considering cancelling a project on Ehlers Danlos Syndrome which she had recently been appointed to run; Reinhold believes this happened because of her criticism of the PACE trial.
Psychology researcher and journal editor James Coyne was forbidden by the journal PLOS One to criticise the PACE trial on his blog, which was hosted by the journal.
A powerful person attempted to pull strings behind the scenes to get the special edition of the Journal of Health Psychology, which focused on criticism of the PACE trial, cancelled at the last minute, according to James Coyne.
The PACE trial’s principal investigators contacted the Journal of Health Psychology to demand that parts of the article ‘PACE-Gate': When clinical trial evidence meets open data access be retracted. They also demanded that the article’s author, Keith Geraghty, disclose that he had ME/CFS, which they saw as a conflict of interest. Both demands were refused.
Caroline Struthers, a manager of an organisation which works to improve the quality of healthcare research, posted a link to an article which was critical of the PACE trial on her personal Facebook page. In response, chronic fatigue syndrome researcher Lillebeth Larun contacted Struthers's boss and made a complaint about her.
PACE trial principal investigator Michael Sharpe wrote to the British member of parliament Carol Monaghan, who is an advocate for people with ME/CFS and who is very critical of the PACE trial, and told her that her behaviour was ‘unbecoming of an MP’.
PACE trial principal investigator Michael Sharpe wrote to STAT to ask that the article ‘Bad science misled millions with chronic fatigue syndrome. Here’s how we fought back’ be either retracted or corrected with an erratum. He sent a similar request to The Conversation regarding the article ‘How a study about Chronic Fatigue Syndrome was doctored, adding to pain and stigma’. Both requests were refused.
PACE principal investigators Peter White and Michael Sharpe contacted psychology researcher Keith Geraghty’s university to make formal complaints about him because he had published articles that were critical of the PACE trial.
Retired UK doctor and researcher Jonathan Edwards wrote in a forum post that: "If you criticise PACE in the UK there is a quiet phone call to your employer and the next morning you are asked to a meeting to reconsider your contract. I get emails from people to whom this has happened. The only reason why I am the only UK academic to call out on PACE is that I am retired so do not have an employer."
Edwards also implied that “the UK paper that usually exposes corruption and champions the ordinary person” would fire any journalist who spoke out against the PACE trial. The newspaper he referred to is most likely The Guardian.
The NICE guidelines on ME/CFS
The UK’s National Institute of Health and Care Excellence (NICE) publishes guidelines on how medical conditions should be treated by healthcare professionals. NICE is highly respected, and its guidelines are influential in Europe and around the world.
NICE’s 2007 guideline on ME/CFS recommended CBT and GET as treatments. In 2017 this guideline came up for review, and NICE provisionally decided that the guideline was fine and did not need to be updated. However the ME Association, an organisation which supports people living with ME, got wind of this and alerted the various other ME advocacy organisations. NICE received a vehement response from the ME community, which prompted them to reverse their previous decision. NICE would conduct a full review, essentially starting the process of developing the guideline from scratch.
Further reading:
• Breaking news! ME Association has heard back from NICE about the guideline review… and it’s not good news, ME Association
• NICE Rejects Current Guidance, Plans "Full Update", Trial by ErrorNICE spent four years conducting a thorough and publicly available review of the evidence for treatments for ME/CFS. They concluded that the evidence for CBT and GET as treatments for ME/CFS was of poor quality, and that GET could in some cases be harmful. The new guideline, finalised in October 2021, makes it clear that GET should not be offered to people with ME/CFS, and that these patients should receive medical care from a team of medical specialists. This new guideline was welcomed by ME patient support and advocacy organisations.
NICE is a deeply respected institution in the UK medical world. However, astonishingly, leaders of the UK medical establishment spoke out against the new, evidence-based guideline. Several professional associations for doctors, including the Royal College of Physicians and the Royal College of General Practitioners, published a joint statement in which they argued, without evidence, that NICE was mistaken, and that CBT and GET could in fact improve symptoms of ME/CFS.
Furthermore, a number of editorials and opinion pieces have appeared in medical journals arguing that the new NICE guideline is a step in the wrong direction.
Further reading:
• The expert testimony to NICE that took apart the case for CBT and graded exercise for ME/CFS, ME/CFS Research Review
• ME exercise therapy guidance scrapped by health watchdog Nice, The Guardian
• The Updated NICE Guidance Exposed the Serious Flaws in CBT and Graded Exercise Therapy Trials for ME/CFS, Healthcare (Basel)
• New NICE guideline on chronic fatigue syndrome: more ideology than science? The Lancet
• ME/CFS: Researchers question credibility of NICE guidance, The British Medical JournalTwo years on from the publication of the new NICE guideline, CBT and GET are no longer offered to patients with ME/CFS who attend NHS-run specialist clinics. However, apart from this, little has changed for patients. It seems that the debate over science has been won, yet the new scientific consensus has not yet had much of an effect on the way that doctors and nurses treat people with ME/CFS in their day-to-day practice. The PACE trial has been thoroughly discredited, yet somehow it still guides the (mis-) treatment of patients.
This article is licensed Creative Commons Attribution 4.0. You can re-post it anywhere you like - just credit the author, Katie Johnstone, and link back to this page.
Thank you so much for this informative series. It’s the best precis of the PACE trial fraud I’ve ever read. It seems the authors of the trial were backed by very powerful interests, indeed. I hope someone(s) looks into that.