FDA Should Not Approve a Procedure with Unreliable Results, Watchdog Writes in Medical Journal
WASHINGTON, D.C. – In a letter published today in the Journal of the American Medical Association (JAMA), Public Citizen criticized the authors of a study in the January 19, 2011 issue of the journal who tested the effectiveness of brain scans using an experimental dye for detecting brain abnormalities found in patients with Alzheimer’s disease. The authors withheld critically important data from the Journal when documenting the extent to which interpretation of these scans varies from one physician to another, Public Citizen said.
“The public health implications of these unreliable tests would be two-fold: First, substantial numbers of people could be incorrectly diagnosed as having such brain abnormalities when they really do not, and by the same token, others could be falsely reassured that they do not have such brain abnormalities when they actually do,” said Dr. Michael Carome, deputy director of Public Citizen’s Health Research Group. “By withholding important information, the authors misled the journal and its readers to believe that this test for identifying brain abnormalities was more reliable than it actually is.”
Alzheimer’s disease is the leading cause of irreversible dementia in the United States. Currently, there is no definitive test for diagnosing Alzheimer’s disease while a patient is alive, and so the diagnosis typically is made by doing tests to exclude other causes of dementia, such as strokes, certain nutritional deficiencies and hypothyroidism. Prior autopsy studies looking at the brains of Alzheimer’s disease patients have revealed plaques (deposits) of an abnormal protein called beta-amyloid, but not all individuals with beta-amyloid plaques in their brains had dementia.
More importantly, there is no treatment that cures or stops the progression of Alzheimer’s disease, and the few drugs approved by the Food and Drug Administration (FDA) for treating such patients have significant risk of adverse events while offering very meager benefits in terms of improving cognitive function.
“Given what is known about Alzheimer’s disease and the lack of effective treatments, any proposed test for diagnosing this disease must be proven to be highly accurate in large, rigorously conducted studies before it is approved by the FDA for marketing,” Carome said.
In the JAMA study criticized by Public Citizen, researchers tested whether injection of an experimental radioactive dye, called florbetapir F 18, can be used to detect beta-amyloid plaques in the brains of living people using an imaging procedure called a positron emission tomography (PET) scan. The researchers proposed that the injected dye would bind to beta-amyloid plaques in the brain and be visible on a brain PET scan.
For the key part of the study, the researchers performed florbetapir-PET brain scans in 35 elderly, terminally ill subjects (only 17 of whom had been diagnosed with Alzheimer’s disease). After their deaths, the researchers compared those scan results to measurements of beta-amyloid plaques in their brains during autopsy. Importantly, three nuclear medicine physicians, who had undergone extensive training, independently read and scored each florbetapir-PET scan for the amount of beta-amyloid seen in the subject’s brain.
In their JAMA article, the researchers presented only the median score of the three physician readers for the amount of beta-amyloid seen on each florbetapir-PET scan, but withheld critical data on the scores for each individual physician reader – data that Avid Radiopharmaceuticals (Avid), a wholly owned subsidiary of Eli Lilly, Inc. and the company that makes florbetapir, had submitted to the FDA on September 17, 2010, four months before the JAMA article was published.
“By only giving the median score, the researchers obscured the variation that actually occurred in the scoring between the three readers,” Carome said. Indeed, even the researchers themselves acknowledged that their procedure of using median score for the three physician readers “is not likely to be replicated in clinical settings.”
FDA analyses of the individual physician reader data showed the substantial variation among the three independent, highly trained physician readers, which led its reviewers to seriously doubt the validity and usefulness of florbetapir-PET scan for detecting beta-amyloid plaques in the brain.
“Given the poor performance of the test at the individual physician reader level in a rigorously controlled setting, the variability and unreliability will only get worse when the test is widely distributed across the country,” Carome said. “As a result, tens of thousands of patients will have either a false-positive test (told they have beta-amyloid deposits when they do not) or a false-negative test (told they don’t have beta-amyloid deposits when they do). This will lead to unnecessary anxiety and treatment in some patients, false reassurance in others and a waste of millions of health-care dollars.”
It is important to note that the majority of the authors of the JAMA study had a direct financial relationship with Avid, including nine who owned stock or stock options in the company. So while the authors of the original JAMA study were highly motivated to portray these results in the best possible light, they also concluded that “Understanding the appropriate use of florbetapir-PET imaging in the clinical diagnosis of [Alzheimer’s disease] or in the prediction of progression to dementia will require additional studies.”
Public Citizen is a national, nonprofit consumer advocacy organization based in Washington, D.C. For more information, please visit www.citizen.org.