Off the PACE - scientific malpractice and chronic fatigue syndrome

Screenshot of Daily Telegraph article claiming that the PACE trial, a piece of CFS/ME research has found that ME is not actually a chronic illness. The headline is Chronic Fatigue Syndrome sufferers “can overcome symptoms of ME with positive thinking and exercise"

‘Chronic Fatigue Syndrome sufferers “can overcome symptoms of ME with positive thinking and exercise.”’

I’m sure we have all seen these headlines. A remarkable discovery: yuppie flu is exactly that; it’s not a real chronic illness – it’s just in people’s heads. Despite the fact that almost every case of CFS/ME (Chronic fatigue syndrome/myalgic encephalomyelitis) starts with a viral infection, the $8 million dollar, large-scale study, known as the PACE trial (‘Pacing, Graded Activity, and Cognitive Behaviour Therapy; a Randomised Evaluation’), managed to prove the psychosomatic basis of CFS/ME and – even better – that the cure was very simply receiving cognitive behavioural therapy (‘CBT’) and a program of graded exercise therapy (‘GET’). But did it really prove that? As you can perhaps tell from my slightly ironic tone, the short answer is no, and I have, among many others, suffered intensely, as a result of that. So, what better time than CFS/ME awareness month to expose bad science and its effects on CFS/ME patients.

‘Think positively and exercise.’

When this conclusion was first published, many CFS/ME patients were very surprised as it goes contrary to the experiences of the ME/CFS community at large. The key symptom that all CFS/ME sufferers share is that their symptoms get worse after exercise or cognitive exertion. We have cognitive and exercise intolerance. So, a programme suggesting that the very thing which makes us sicker will cure us, seemed very unlikely to be true. It would be like discovering that the way to cure cancer is keep growing the cancerous tumour until it magically goes away. Totally bonkers!

Bad science?

And we were right. It turns out that this large-scale and very expensive research, published by ‘The Lancet’ journal is, in fact, a perfect case study in how not to carry out scientific research. In other words, this research is filled with scientific malpractice and flaws. The flaws that I will explore in this post are:

changes made to the criteria of effectiveness and recovery halfway through the study;
main outcomes tested subjectively, and a persuasive newsletter sent out halfway through the study;
lack of informed consent and information about conflicting interests in the study.

Despite widespread critique (some is linked below) and long legal battles over this study, it has yet to be retracted and is still used in treatment of CFS/ME patients worldwide.

Changes made in the criteria of effectiveness and recovery

In order to qualify for this study, patients had to be diagnosed with CFS/ME using ‘the Oxford criteria’. These criteria are somewhat vaguer than most other CFS/ME diagnostic guidelines and many experts have suggested should no longer be used in diagnosing CFS/ME. So, to begin with, it is possible that some of the subjects of the study were not suffering from CFS/ME, but potentially from other conditions that have similar symptoms.

But that withstanding, subjects had to score below 65 on the SF-36 scale (a scale where 100 means perfectly health and 0 is the maximum possible disability) in order to qualify. At the beginning of the study, it was decided that subjects would be considered ‘recovered’ if they achieved a score of 85. Midway through the study, this recovery score was lowered from 85 to 60. For context, a score of 85 means that people are able to work but they are not fully health; this is, for many, recovery. A score of 60 is equivalent to the average score of patients suffering with Class II coronary heart failure. It is difficult to see how this, in any way, qualifies as recovery.

Perhaps more shockingly, this actually means that some of the people who started the study with a score of 65 or lower were, by the new recovery criteria, deemed to be recovered with a score of 60. This meant that people could get sicker than they were at the beginning and would still be counted as having recovered. This change occurred halfway through the study, with very little explanation. Very conveniently, they would have been seeing the effects of the treatment by this point in the study and could change the recovery score to match up with the outcomes they wanted. If that doesn’t scream bad science, I don’t know what else does!

Main outcomes tested subjectively

The key method used to get data about a patient’s recovery, in this study, was the patient’s personal feedback. This meant that the data was very subjective. This is the perfect set-up for a study demonstrating the placebo effect. However, this was not a study about the placebo effect. Added to this, about halfway through the study, all participants received a newsletter claiming the high success of CBT and GET. Again, this will have added to the placebo effect and, scientists suggest, could have had a strong effect on the outcome of the study (although it turns out their results were void anyway).

There were some objectively measurable statistics taken, such as a walking test, data on employment, etc., but this either did not prove the success of the treatment or was not fully followed up on, as it was deemed irrelevant. Or perhaps, going out on a limb, was not collected so that it could not undermine the supposed success of the treatment.

Lack of informed consent and a conflict of interests

The participants of the study were not informed of the possible effects, or even existence of the conflicting interests of the researchers. This means they did not give fully informed consent. Alongside that, the researchers did not include any discussion of the conflicting interests they might have had in the research.

And these conflicts were quite substantial. For starters, many of these researchers were already using and promoting the use of CBT and GET treatments before the trial began. This means that confirmation bias is highly likely to have played a role in the outcomes of this research.

Additionally, trial leader Peter White was an advisor to the DWP (Department of Work and Pensions) in the UK – the department that deals with disability benefits. He and another of the researchers have a history of working as paid disability insurance consultants, advising that CBT and GET could help get CFS/ME patients off benefits and back to work. In other words, the researchers’ connections meant that they were personally highly likely to benefit from the success of the study, even if that success came at the expensive of those with CFS/ME.

Worse still, this study was part funded by the DWP. In a freedom of information request to the DWP by D Benyon it was discovered that this is the only medical research of this type to have been funded by the DWP. As Carol Monaghan (MP for Glasgow North West), has remarked:

“One wonders why the DWP would fund such a trial, unless it was seen as a way of removing people from long-term benefits and reducing the welfare bill.’
Speaking after the debate she said ‘discovering that the PACE trial was funded by the DWP, no doubt with the intention to reduce the amount of people on benefits, should cause great concern.”

This conflict of interests, alongside the obvious scientific malpractice, makes the results of the PACE trial difficult to believe. More than this it seems like a targtetted attack at the thousands of people with ME worldwide.

A critical response

Thankfully, after a very long legal procedure, using the freedom of information act (which merits an article of its own!) and a lot of work from other researchers and patient lobby groups, the research data was released so that other scientists could review it. This re-review of the data brought about the dramatic reduction in the initial success rates such that the improvement rate – initially calculated at 65% – dropped to 21% (the average CFS/ME improvement rate without GET and CBT is 10%). The full recovery rate of the study dropped from 21% to 7%.

Furthermore, there have been many studies that have not only proved the biochemical basis for the illness, meaning it is not psychosomatic, but they have also demonstrated how harmful CBT and GET are to people with CFS/ME. In fact, in a survey conducted by the ME Association, 74% of participants said that their symptoms had worsened after a course of GET and CBT.

The PACE results are still used

Despite the fact that there is more than adequate proof of the harm caused by GET and CBT, the PACE study continues to have a huge influence on the way ME/CFS is treated. This is because it is by far the largest and most expensive piece of research about ME/CFS. It allows medicine to give an explanation of sorts to something that is not yet very thoroughly researched or well understood. Furthermore, it places the responsibility of health onto the sick person: if something is proved to be ‘all in their head’, they must take responsibility that they are still sick.

As someone who has been to the chronic fatigue specialists both in the Netherlands and the UK, it is clear that, at least in public healthcare, the only treatment option for CFS has been CBT and GET. And would you believe it, when I tried it, I did not have an overly positive experience. At that point in time I had managed to rest enough and was in a place where I was able to work a four-hour shift a week in a little shop. The CFS specialist told me to sort out my sleep pattern and attempt the ‘couch to 5 kilometre’ challenge (where, in the space of a couple of months, you end up being able to run 5 kilometres). I started it and was very excited to be doing exercise again. However, every single time I went for a run I would come back with an intense migraine and would be unable to leave bed for the next day or two. I continued to follow the programme until I was far too sick to leave the house anymore. I eventually stopped but, as a result of it, had lost all the progress I had made before I began ‘treatment’. Not only did I have to quit the job that I was so proud of being well enough to work, I ended up having to move back in with my parents for a while because I was too sick to look after myself and live independently. So clearly not such a success story.

Besides the direct harm that ME/CFS patients will suffer if they follow a programme of GET and CBT, there is a larger impact on the ME/CFS community (I will go into this in more detail, on another blog post). The idea that sufferers of ME/CFS just need to think positively has been widely broadcast to the public, and the idea that the ME/CFS community will send death threats to researchers has been widely broadcast to scientists (even though this never happened). This discredits and invalidates an estimated 15-30 million sufferers of CFS/ME, tells them that they are not sick, but they need to work harder. It persuades their families, friends, employers, etc., that they are faking being ill, and that nothing is really wrong, This means that CFS/ME patients often do not get the support they need, and lose family and friends in a way that is much less common with other illnesses, where there is less stigma.

More dangerously, the conclusion that CFS/ME doesn’t exist, except in the heads of their patients, encourages doctors who know little about CFS/ME to assume that their patients are hypochondriacs, and to dismiss any illness they come in with as an ME symptom – even when it almost certainly is not. I have been in the emergency department and been told I was having a panic attack when they found out that I had CFS/ME. I was not having a panic attack. I will never know what happened and I have not bothered getting it checked out because I know that doctors will not take me seriously. In fact, what it does is to render even more invisible, a very severe illness, just by pretending that it does not exist. So people who, when measured on a fitness scale are as ill, or more ill, than those with coronary heart failure are still unable to access disability services, disability benefits and proper medical treatment. Instead they are sent away and told they are crazy.

In some cases, this goes even further. In both the UK and Denmark there are cases of CFS/ME patients who have either been threatened with, or actually sectioned. They have been forced into psychiatric institutions because it has been claimed that their parents are enabling their sickness by not forcing them to exercise. They are removed from their house and sent to a psychiatric hospital where they are forced to do things that are not possible and will make already severely ill people far worse. I have left a link below to one such case in the UK.

Fortunately, due to an impressive patient movement, the treatment guidelines in the UK and other parts of the world are currently being reviewed. NICE, the organisation who create treatment guidelines in the UK began their review in 2018. However, there have been many questions raised about the impartiality of the board of reviewers, so we will just have to wait and see whether this results in any changes regarding guidelines about ME. Until then we know that there is clearly a very long way to go before we can fully remove the harmful consequences and stigma attached to CFS/ME, as a result of the PACE study.

As a community of people sick with CFS/ME, and their friends, families and allies, we need to continue to put pressure on the right people, spread awareness, write blog posts, facebook posts, have conversations, spend years studying to become researchers, score a grant and research CFS/ME. I feel like I am lucky enough to be sick in a time when things are changing, and that change has been driven by a dedicated group of people and a whole lot of education. We can be part of removing the stigma of CFS/ME.

If you would like to do any further reading about this, I have linked below a mixture of resources, some of them providing a far more scientifically in-depth explanation of the study and its flaws. If you are, or know any medical professionals or doctors, please encourage them to read and share this information with their friends and colleagues. And if you happen to be a science teacher or lecturer, feel free to use this research as a case study into how not to conduct research!

Chronically Ill Kat