Letter Requesting the Halt of Ads for Vagus Nerve Stimulator Device
Timothy A. Ulatowski, Director
Office of Compliance
Center for Devices and Radiological Health
Food and Drug Administration
2098 Gaither Road
Rockville, MD 20850
Dear Mr. Ulatowski:
We are writing to request that you use your authority under § 502 (q) of the Food, Drug, and Cosmetic Act to ensure that Cyberonics immediately withdraws its advertisement for the vagus nerve stimulation (VNS) device currently running on the WebMD website[1]and be forced to run a corrective advertisement. By making a series of claims greatly exaggerating the findings from the VNS clinical studies, the advertisement is “false or misleading in any particular,” the criterion for being considered a misbranded device under the Act. As indicated below, there are at least 10 violations of FDA laws and regulations in this advertisement.
One page of the advertisement is entitled “Frequently Asked Questions About VNS Therapy”[2]and includes a series of questions and answers about VNS, including the following discussion of VNS’ effectiveness:
Q. How well does VNS Therapy work?
A. One of the studies of VNS Therapy that ultimately led to the approval from the Food and Drug Administration involved 235 people with chronic or recurrent depression. At the time of the study, their current episode of depression had lasted an average of 50 months, despite an average of 10 treatments that included a variety of medications, psychotherapy, and, for many, ECT. About 6 out of 10 patients experienced a meaningful clinical benefit from VNS Therapy. Of these people, 1 in 3 who received VNS Therapy for a year experienced significant mood improvement and 1 in 6 had few or no remaining symptoms of depression.
In a similar group of 104 people who received only other antidepressant treatments, without VNS Therapy, 1 in 8 had significant mood improvement and only 1 in 14 people had few or no symptoms of depression. In other words, the people who received VNS Therapy in addition to other treatments were 2 to 4 times more likely to experience significant mood improvements. (emphasis in original)
In the analysis of the advertisement in this letter, we assume that the description in the first paragraph refers to the 12-month data for the long-term phase of Study D02, and that the second paragraph describes the 12-month data from comparative Study D04 (descriptions of these studies follow).[3] The statements in the excerpt are inaccurate or misleading in at least 10 different ways. Our claim is based upon a detailed comparison between the advertising copy and the FDA-approved VNS professional label, [4] which is the standard against which the accuracy and legality of advertising materials is judged. Page numbers in parentheses in this document correspond to those in that label.
1. “One of the studies …”
While the long-term phase of Study D02 certainly was one study submitted to the FDA, it was not the most important one. In fact, it was a nonrandomized extension of a short-term (12-week) study called Study D02. In the short-term phase of Study D02, patients with treatment-resistant depression were implanted with the VNS device and then randomized to have the device turned on or not. The strengths of this study design are obvious, notably assuring that the two study groups were similar at enrollment and thereby controlling for any confounders. But the advertisement makes no mention of these most-important results: at the end of 12 weeks there was no statistically significant difference between the two treatment groups on the primary outcome variable (p=0.238) in the short-term phase of Study D02. In plain English: the primary efficacy study failed to demonstrate efficacy. For the long-term phase of Study D02, patients whose VNS device was not turned on in the short-term phase were allowed to have their devices activated. However, the doctors and the patients were not blinded and concomitant therapies were permitted.
It was only when the original short-term Study D02 was negative that the company decided to create a long-term, non-implanted comparison group (Study D04), yielding the results for the non-randomized, unblinded long-term comparison between Study D02 and Study D04 cited in this advertisement.
The point is that the company has cherry-picked which studies to cite on the website, avoiding the best-designed (and, coincidentally, negative) short-term phase of Study D02. This selective presentation of results in the advertisement (all these studies are described in detail in the device label) is misleading to physicians and patients.
2. “… involved 235 people”
This overstates the number of evaluable patients in the study and thus confers undue credibility upon the long-term phase of Study D02. The number of subjects originally enrolled in Study D02 was 235, of whom only 177 (75%) were available for analysis at 12 months (p. 37).
3. “…with chronic or recurrent depression”
Actually, as the product label makes clear (p. 33), Study D02 included patients “with chronic or recurrent treatment-resistant depression” (emphasis added). Consequently, VNS was approved for “adjunctive long-term treatment of chronic or recurrent depression for patients 18 years of age or older who are experiencing a major depressive episode and have not had an adequate response to four or more adequate antidepressant treatments” (p. 6). The effect of the advertisement’s incorrect description of the study population would be to enlarge the patient population in whom the device could be used.
4. “About 6 out of 10 patients experienced a meaningful clinical benefit from VNS therapy”
This appears to be a reference to the fact that 56% of patients in the long-term phase of Study D-02 had a “meaningful clinical benefit” after 12 months, based on the HRSD scale (p. 42). The obvious problem with this claim that the improvements were due to VNS (“a meaningful clinical benefit from VNS therapy” [emphasis added]) is that it fails to account for the placebo effect or regression to the mean, major factors in any depression study. This, in turn, follows from the fact that the D02/D04 comparison is not based on randomization and the fact that the negative results from Study D02, which was randomized, are ignored. These factors were among those cited by the more than 20 FDA officials who opposed the approval of VNS in the first place.[5] If, for the sake of argument, we accept the validity of the comparison between long-term Study D02 and Study D04, we can see that for the outcome measure IDS-SR, over half of the responders to VNS (22% responded) would have responded to no treatment at all (12% responded) (p. 41), demonstrating the impact of the placebo effect and regression to the mean.
5. “1 in 3 who received VNS Therapy for a year experienced significant mood improvement”
This claim appears to rely upon the data on “responders” according to the HRSD scale, as described in Figure D-6 of the product labeling (p. 41). However, Figure D-6 also contains the data from another scale, the IDS-SR, which shows a lower response rate (22% [one in five] on the IDR-SR vs. 30% with the HRSD). Ironically, the company had used the HRSD as its primary outcome measure in Study D02, but switched to the IDS-SR for the D02/D04 comparison after only the IDS-SR showed a statistical improvement in Study D02.
6. “… 1 in 6 had few or no remaining symptoms of depression”
This assertion evidently refers to the data on “complete responders” on the HRSD, derived from Figure D-6 (p. 41). Again, there is a lower response on the previously favored IDS-SR (15% [one in seven] on the IDS-SR vs. 17% with the HRSD).
7. “In a similar group”
This terminology, in the context of product information that a reasonable person might assume relied upon rigorously collected data, may imply that the comparison group (Study D04) was a randomized control group or that investigators and patients were blinded. Of course, it wasn’t, and thus the basis of any comparison is greatly undermined. Moreover, Table D-13 in the product labeling (p. 35) shows statistically significant baseline differences between the subjects in the long-term phase of Study D02 and those in Study D04 with respect to race, history of electroshock therapy and number of lifetime episodes of depression. It is not surprising that some differences were present because only 12 of the 22 study sites in Study D02 participated in Study D04[6](and one Study D04 site was not included in Study D02 [p. 34]). The authors of a paper based on the D02/D04 comparison concede that Study D04 “had not originally been intended to serve as the [control]; it was intended to describe health care costs.”[7] Patients in both the long-term phase of Study D02 and Study D04 were permitted to undergo concomitant treatment for depression. In an analysis mandated by the FDA, data censored for medication changes (i.e., data on a patient were considered complete if and when that patient changed their antidepressant regimen) and restricted to overlapping sites provided no evidence of a statistically significant impact of VNS upon primary or secondary endpoints.[6]
8. “…1 in 8 had significant mood improvement”
This appears to refer to the HRSD data on responders in Study D04 (p. 41). Note how for the comparison group the website uses the more neutral (and accurate) “had,” while for the treated group, the causation-implying “from” is used (see Point 4, above). Taken together, Points 4 and 8 imply that any change due to VNS is caused by the device, but that any change in the comparison group may be coincidental or a result of the study design. Actually, both arms of the Study D02/D04 have the same limitations.
9. “…only 1 in 14 people had few or no symptoms of depression”
These data appear to be the HRSD data on “complete responders” in Study D04, even though the company had switched to the IDS-SR as the primary outcome for that study.
10. “In other words, people who received VNS … were 2 to 4 times more likely to experience significant mood improvements” (emphasis in original)
At first, this claim was baffling: the relative risks of clinical improvement, based on the HRSD, for “responders” and “complete responders” in the long-term Study D02 and Study D04 are 2.31 and 2.43, respectively (Figure D-6, p. 41),[8] not encompassing the two-to-four-fold range mentioned on the website. But we were fooled; for this analysis, the company appears to have switched back to the IDS-SR, for which the corresponding relative risks are 1.83 and 3.75, which, with a bit of rounding help, yield the claimied benefits. Thus, having relied upon the HRSD data for the vast majority of its efficacy claims in the advertisement, the company has suddenly switched gears and relied upon the IDS-SR (favored in the device application) when that produced results more favourable to its product. (Of course, any of these overstate the benefit of a device that showed no effectiveness in its best-designed study.)
It is no small feat of advertising to cram no fewer than 10 false or misleading statements into just two paragraphs. This device should never have been approved, but, as long as it still is, it is time for the FDA to step in and make sure the device is legally advertised. Please ensure that the company immediately withdraws its advertisement (and any similar ones that may be running in any other medium) and require the company to issue corrective advertising of at least equal prominence.
Sincerely,
Peter Lurie, M.D., M.P.H.
Deputy Director
Nicholas Stine
Research Associate
Sidney M. Wolfe, M.D.
Director
Public Citizen’s Health Research Group
Cc: Daniel Schultz, Director, CDRH
Andrew von Eschenbach, Acting Commissioner, FDA
Senator Charles Grassley
George Lundberg, Editor, MedGenMed
[1] http://www.webmd.com/diseases_and_conditions/depression.htm (downloaded May 18, 2006).
[2] http://www.webmd.com/content/pages/25/113832.htm (downloaded May 18, 2006).
[3] We have reached that conclusion based upon a comparison of study designs and numbers of subjects enrolled provided in the VNS product label.
[4] Cyberonics. Depression Physician’s Manual. VNS Therapy Pulse Model 102 Generator and VNS Therapy Pulse Due Model 102R Generator. December 2005. Available at: http://www.vnstherapy.com/depression/hcp/Manuals/default.aspx (downloaded May 18, 2006)
[5] Committee on Finance, United States Senate. Review of the FDA’s Approval Process for the Vagus Nerve Stimulation Therapy System for Treatment-resistant Depression. February 2006.
[6] Food and Drug Administration. Final Statistical Summary Review for PMA P970003/S50 (Original and Various Amendments), Vagus Nerve Stimulator (VNS) Therapy System for Depression, Cyberonics, Inc. Available at: http://www.fda.gov/ohrms/dockets/ac/04/briefing/4047b1_00_a_FDA%20Statistical%20Review%20Memo.htm (downloaded May 18 2006).
[7] George MS, Rush AJ, Marangell LB, Sackeim HA, Brannan SK, Davis SM et al (2005): A one-year comparison of vagus nerve stimulation with treatment as usual for treatment-resistant depression. Biol Psychiatry 58:364-373.
[8] The relative risk is calculated by dividing the percent of responding long-term Study D02 patients by the percent of responding Study D04 patients. For the HRSD, this is 30%/13% =2.31 for “responders” and 17%/7% = 2.43 for “complete responders.”