Letter Requesting that Medicare Deny Reimbursement for Transcranial Magnetic Stimulation (HRG Publication #1895)
November 11, 2009
To whom it may concern:
We are writing to request that the Center for Medicare & Medicaid Services (CMS) issue a National Coverage Determination (NCD) that would deny Medicare reimbursement for Neuronetics’ NeuroStar Transcranial Magnetic Stimulation (TMS) Therapy System. This product is marketed for the treatment of Major Depressive Disorder in patients who have not benefited from one prior antidepressant medication. In the randomized, controlled trial submitted to the Food and Drug Administration (FDA) by the sponsor, the product did not have a clinically or statistically (p=0.057) significant impact on the depression scale that served as the study’s primary outcome variable. Moreover, the study was marred by unblinding that would likely have favored the product.
Although the device was cleared for this indication by the FDA on October 7, 2008, CMS has stated that such clearance does not automatically guarantee reimbursement under the Medicare program. Indeed, while under the Food, Drug, and Cosmetic Act, a device must demonstrate a “reasonable assurance of … safety and effectiveness” before it can be marketed,[*] the Social Security Act provides for reimbursement under Medicare only if the device is “reasonable and necessary.” As CMS has made clear on multiple occasions, while it typically defers to the FDA’s determinations of safety and efficacy, it determines eligibility for reimbursement under a different, more restrictive standard:
In our view, the device has never met the FDA’s standard for approval, much less CMS’s more rigorous one.
The FDA determines whether the criteria for approval or clearance have been met using one of the two general premarket review mechanisms created in the 1976 Medical Device Amendments to the federal Food, Drug, and Cosmetic Act. The first, Premarket Approval (PMA), is analogous to the New Drug Application for drugs. A sponsor must directly establish the safety and effectiveness of its device through a well-controlled study, generally a randomized controlled trial (RCT). In the second, the 510(k) pathway, a sponsor attempts to infer the safety and effectiveness of its device based upon a comparison to another, currently marketed (“predicate”) device.
With the FDA Modernization Act of 1997, Congress created another premarket review procedure, called the de novo process, for devices rejected in the 510(k) pathway. If a device is rejected under 510(k), it typically has to undergo review under the more stringent PMA process if the sponsor elects to pursue approval. Through the de novo process, the sponsor may instead request that the FDA determine whether the device can be cleared even without identifying a predicate device. A de novo pathway request circumvents filing another 510(k) or a PMA application in order to reach the market. However, if a device is cleared in the de novo process, the safety and effectiveness of the device has not been established through well-controlled trials in the PMA pathway nor through a comparison to a legally-marketed device in the 510(k) pathway. Once cleared in the de novo process, the device may serve as a predicate device for future 510(k) submissions. It is this newer pathway through which the FDA ultimately cleared TMS in October 2008.
In the NCD process, a company (typically) seeks a favorable determination by CMS. CMS then initiates an evidence-based process to determine whether the criteria for reimbursement (“reasonable and necessary”) have been met. This may include an outside technology assessment and/or referral to CMS’ Medicare Coverage Advisory Committee. However, CMS regulations provide for the filing of NCDs by entities other than the manufacturer:
Therefore, we are requesting that CMS initiate an evidence-based evaluation of the NeuroStar TMS Therapy System to determine its suitability for reimbursement under the Medicare program.
Background on Transcranial Magnetic Stimulation
Pivotal Clinical Trial
Expecting to demonstrate the efficacy of TMS over the short-term, the sponsor conducted a randomized, placebo-controlled trial. Three-hundred and one patients were enrolled, randomized to either active treatment or sham treatment (machine mimicked treatment without applying a magnetic field) and were required to complete at least one follow-up assessment. Concomitant antidepressant medications were not permitted during the study. The primary effectiveness endpoint, measured at four weeks, was the Montgomery Asberg Depression Rating Scale (MADRS) and a standard-effect size of 0.40 was specified as the minimal level for clinical significance. In addition, Neuronetics prespecifed 26 clinician- and patient-rated secondary outcome measures. Patients who did not respond to either active or sham TMS on the primary endpoint could withdraw from the study and enroll in a separate nine-week uncontrolled, open-label study. In December 2007, the results of the randomized, controlled trial were published in Biological Psychiatry. We have previously offered critiques of this trial in testimony before the Neurological Devices Panel of the Medical Devices Advisory Committee and in a letter published in Biological Psychiatry.
In that pivotal trial, the difference between patients treated with active and sham TMS was clinically minor (1.7 points on the 60-point MADRS; standardized effect size=0.39) and statistically non-significant (p=0.057);  the difference became statistically significant (p=0.038) only after the post hoc exclusion of six patients who had met a priori inclusion criteria. Of the 26 secondary outcome variables, half met statistical significance favoring the treatment at a p-value of 0.05. However, this ignored the need to correct for multiple comparisons. After a conservative correction for multiple comparisons, none of the 26 secondary endpoints was statistically significant. The sponsor’s multiplicity adjustment did produce some positive findings, but was post-hoc and only included 13 of the 26 secondary endpoints.
Furthermore, during the trial, patients may have been inadequately blinded, potentially creating a bias in favor of finding treatment effectiveness. For instance, application-site pain was nearly ten-fold greater in TMS-treated patients compared to those receiving sham treatment (35.8% vs. 3.8%). And after adjustment for potential unblinding using an inclusive measure of pain, the FDA concluded that “the report of any pain in aggregate could account for some of the observed treatment effect [in the primary endpoint].”
In the subsequent nine-week open-label study, patients from both groups in the pivotal trial who did not respond were followed and administered TMS. The sponsor reported significant decreases in MADRS scores for both groups at six weeks (17.0 in the sham non-responders and 12.5 points in the TMS non-responders)., However, the patients were aware of their treatment status. In addition, the decreases in MADRS scores in both studies can be partially explained by regression to the mean: the selection of patients with relapsing conditions who enroll in studies when they are near their worst clinically and are thus likely to improve, on average, with or without further treatment. Regardless, the importance of the uncontrolled, open-label study findings is overshadowed by the negative outcome in the more scientifically rigorous randomized controlled trial.
Lastly, TMS is not a benign treatment. Compared to patients receiving sham treatment, those receiving active TMS experienced markedly more muscle twitching, application site pain and discomfort (20.3% vs. 3.2%; 35.8% vs. 3.8%; 10.9% vs 1.3%, respectively). However, serious adverse events occurred at similar rates in both treatment arms.
Regulatory Voyage through the Food and Drug Administration
The 510(k) Pathway and the Advisory Committee Meeting
In our view, TMS should have been reviewed as a PMA, but Neuronetics sought clearance for TMS in the 510(k) pathway claiming substantial equivalence to electroconvulsive therapy (ECT). Despite this claim, the company conducted no studies directly comparing the two devices. Instead, the company conducted a placebo-controlled trial and provided only historical data for ECT, mostly more than two decades old and with a different outcome measure (17-item Hamilton Depression Rating Scale or HAMD17 instead of the MADRS). Still, ECT was more effective than TMS (on the 55-point HAMD17, decreases of 9.7 points for ECT and 1.7 points for TMS).
In January 2007, the FDA convened an advisory committee to discuss and make recommendations on the 510(k) submission for TMS. At the meeting, the sponsor contended that, although TMS was less effective than ECT, it could still be deemed substantially equivalent because the lower toxicity of TMS resulted in a similar risk-benefit ratio. The committee’s “consensus was that the efficacy was not established; some [committee members] stated that the device’s effectiveness was ‘small,’ ‘borderline,’ ‘marginal’ and ‘of questionable clinical significance.’”
The committee also addressed the post hoc exclusion of the six patients (which yielded the statistically significant finding on the MADRS): “Considering that the magnitude of the effect size (sic), the Panel thought that post-hoc analysis was problematic.”
Regarding the multiple secondary endpoints, the FDA statistician asserted, “[S]econdary endpoints should only be looked at after the primary endpoint has been met … If we are willing to look past the fact that the primary endpoint was not significant at a strict 0.05 level, in an effort to consider the totality of the evidence, we would still have the significant problem of multiplicity.”
The FDA subsequently determined that TMS was not substantially equivalent to ECT.
The De Novo Pathway
Although the largest randomized controlled trial of TMS ever conducted (it included 2.6 times as many patients as the next-largest trial) failed to demonstrate efficacy on its primary endpoint and, at best, half of its 26 secondary endpoints, Neuronetics persisted. Prior to the advisory committee meeting, the FDA had requested that Neuronetics perform several post hoc evaluations of the trial data, one of which identified a significant treatment effect among patients failing only one prior antidepressant medication (p=0.001). With this knowledge, the company requested review of TMS under the de novo pathway for a more focused (and clinically confusing) treatment population – treatment of depression in patients who had failed to respond to only one trial of antidepressant medications in the current episode (not zero; not two or three; just one).,
Post hoc analyses are problematic for multiple reasons. They may be motivated by prior inspection of the data, a potential source of bias., As the chairperson of the advisory committee that considered TMS stated regarding the stratification of patients according to prior treatment failure, “trying to stratify in this fashion and draw conclusions when the overall p-value is in the range that it is, is done with great peril.” Furthermore, post-hoc explorations can conceal unfavorable findings. In the case of TMS, dichotomizing patients as one versus two to four prior treatment failures obscures the fact that a small group of patients (n=12) with four such failures also appeared to respond to TMS (p=0.022), a finding that undermines the sponsor’s conclusion about the impact of treatment resistance on the response to TMS.
Because of an inflated risk of false-positive results, post hoc observations should always be treated with skepticism, especially when they are associated with negative trials. However, in this instance, the post hoc analysis formed the basis for clearance.
Similarity to a Previous National Coverage Decision
In 2006, CMS faced a NCD for another medical device for depression, vagus nerve stimulation (VNS). Like TMS, the pivotal randomized controlled trial for VNS failed to demonstrate a statistically significant difference in the prespecified primary outcome variable. Also like TMS, the FDA disregarded these findings, instead relying upon less-rigorous evidence to approve the device. As a Senate Finance Committee report subsequently showed, even within the FDA the approval decision for VNS was highly controversial. However, when CMS considered a NCD for VNS, it concluded that the device was neither reasonable nor necessary. Going still further, CMS stated that, “After a thorough review of the current evidence, CMS does not believe there is a treatment benefit directly attributable to VNS therapy for [treatment resistant depression].” We believe that CMS acted appropriately and courageously in the VNS case and urge it to resolve this issue similarly.
The FDA clearance of TMS has not provided assurance that it is an effective treatment for depression, much less that it is a treatment that is “reasonable and necessary” for the Medicare population. Certainly, the evidence mustered for this device does not approach the rigor typically required of anti-depressant drugs. Unless better scientific evidence emerges, CMS would be prudent to deny coverage of TMS, especially because patients are likely to be diverted from clearly effective therapies, such as antidepressant medications to this unproven device.
Peter Lurie, M.D., M.P.H.
Sidney M. Wolfe, M.D.
 21 USC § 360c(a).
 42 USC § 1395y(a)(1)(A).
 68 FR 55636 (September 26, 2003). See 67 FR 66755 (November 1, 2002) for essentially identical language.
 21 CFR § 860.7.
 Food and Drug Administration. Guidance on the CDRH premarket notification review program (k86-3). June 30, 1986. www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm081383.htm. Accessed June 8, 2009.
 21 USC § 360c(f)(2).
 68 FR 55638 (September 26, 2003).
 Simpson KN, Welch MJ, Kozel FA, Demitrack MA, Nahas Z. Cost-effectiveness of transcranial magnetic stimulation in the treatment of major depression: a health economics analysis. Advances in Therapeutics. 2009;26(3):346-68.
 O'Reardon JP, Solvason HB, Janicak PG, et al. Efficacy and safety of transcranial magnetic stimulation in the acute treatment of major depression: a multisite randomized controlled trial. Biological Psychiatry. 2007;62(11):1208-16.
 Lurie P, Park S. Testimony at meeting on repetitive transcranial magnetic stimulation, Duraseal and vagus nerve stimulation. January 26, 2007. http://www.citizen.org/publications/release.cfm?ID=7499. Accessed June 9, 2009.
 Yu E, Lurie P. Transcranial magnetic stimulation not proven effective. Biological Psychiatry. 2009 Sep 29. [Epub ahead of print]
 Food and Drug Administration. Transcript of the Neurological Devices Panel of the Medical Devices Advisory Committee. January 26, 2007. p. 167. www.fda.gov/ohrms/dockets/ac/07/transcripts/2007-4273t1.rtf. Accessed June 8, 2009.
 Food and Drug Administration. FDA executive summary for meeting of the Neurological Devices Panel for Neurostar TMS system. January 26, 2007. pp. 16-7. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_01-FDAExecutiveSummary.pdf. Accessed June 8, 2009.
 Neuronetics. Sponsor executive summary: NeuroStarSystem for the treatment of major depressive disorder. December 19, 2006. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_02-SponsorExecutiveSummary.pdf. Accessed June 8, 2009.
 Avery DH, Isenberg KE, Sampson SM, et al. Transcranial magnetic stimulation in the acute treatment of major depressive disorder: clinical response in an open-label extension trial. Journal of Clinical Psychiatry. 2008;69(3):441-51.
 Food and Drug Administration. FDA executive summary for meeting of the Neurological Devices Panel for Neurostar TMS system. January 26, 2007. p. 16. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_01-FDAExecutiveSummary.pdf. Accessed June 8, 2009.
 Lurie P, Hines J. Testimony before the Subcommittee on Health, Committee on Energy and Commerce, United States House of Representatives on regulation of medical devices, June 18, 2009.
 Food and Drug Administration. FDA executive summary for meeting of the Neurological Devices Panel for Neurostar TMS system. January 26, 2007. pp. 25-7, 52-3. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_01-FDAExecutiveSummary.pdf. Accessed June 8, 2009.
 Food and Drug Administration. Brief Summary From the Neurological Devices Panel Meeting. January 26, 2007. www.fda.gov/AdvisoryCommittees/CommitteesMeetingMaterials/
 Food and Drug Administration. Transcript of the Neurological Devices Panel of the Medical Devices Advisory Committee. January 26, 2007. p. 166. www.fda.gov/ohrms/dockets/ac/07/transcripts/2007-4273t1.rtf. Accessed June 8, 2009.
 Schutter DJ. Antidepressant efficacy of high-frequency transcranial magnetic stimulation over the left dorsolateral prefrontal cortex in double-blind sham-controlled designs: a meta-analysis. Psychol Med. 2009;39(1):65-75.
 Food and Drug Administration. FDA executive summary for meeting of the Neurological Devices Panel for Neurostar TMS system. January 26, 2007. pp. 13-4. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_01-FDAExecutiveSummary.pdf. Accessed June 8, 2009.
 Lisanby SH, Husain MM, Rosenquist PB, et al. Daily left prefrontal repetitive transcranial magnetic stimulation in the acute treatment of major depression: clinical predictors of outcome in a multisite, randomized controlled clinical trial. Neuropsychopharmacology. 2009;34(2):522-34.
 Hines J, Lurie P, Wolfe S. Post hoc analysis does not establish effectiveness of rTMS for depression. Neuropsychopharmacology. 2009;34(8):2053-4.
Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM (2007). Statistics in medicine; Reporting of subgroup analyses in clinical trials. New England Journal of Medicine. 2007;357(21):2189-2194.
 Food and Drug Administration. Transcript of the Neurological Devices Panel of the Medical Devices Advisory Committee. January 26, 2007. p. 396. www.fda.gov/ohrms/dockets/ac/07/transcripts/2007-4273t1.rtf. Accessed June 8, 2009.
 Food and Drug Administration. FDA executive summary for meeting of the Neurological Devices Panel for Neurostar TMS system. January 26, 2007. p. 13 www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4273b1_01-FDAExecutiveSummary.pdf. Accessed June 8, 2009.
 Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005;365(9454):176-86.
 Food and Drug Administration. Transcript of the Oncologic Drugs Advisory Committee. March 4, 2005. p. 72. www.fda.gov/ohrms/dockets/ac/05/transcripts/2005-4095T2.htm. Accessed June 8, 2009.
 Rush AJ, Marangell LB, Sackeim HA, et al. Vagus nerve stimulation for treatment-resistant depression: a randomized, controlled acute phase trial. Biological Psychiatry 2005;58(5):347-54.
 Lurie P, Stine N. Responding to three articles regarding vagus nerve stimulation (VNS) for depression. Biological Psychiatry. 2006;60(12):1382.
 United States Senate, Committee on Finance. Review of the FDA’s approval process for the vagus nerve stimulation therapy system for treatment-resistant depression. February 2006. http://finance.senate.gov/press/Gpress/02_2006%20report.pdf. Accessed June 8, 2009.
 Centers for Medicare and Medicaid Services. Decision memo for vagus nerve stimulation for treatment of resistant depression (TRD). May 4, 2007. www.cms.hhs.gov/mcd/viewdecisionmemo.asp?from2=viewdecisionmemo.asp&id=%20195&. Accessed June 8, 2009.