November 11, 2009

Letter Requesting that Medicare Deny Reimbursement for Transcranial Magnetic Stimulation

Director, Coverage and Analysis Group
Centers for Medicare and Medicaid Services
7500 Security Blvd.
Baltimore, MD 21244

Re: Formal Request for a Negative National Coverage Determination for Transcranial Magnetic Stimulation to Treat Depression

To whom it may concern:

We are writing to request that the Center for Medicare & Medicaid Services (CMS) issue a National Coverage Determination (NCD) that would deny Medicare reimbursement for Neuronetics’ NeuroStar Transcranial Magnetic Stimulation (TMS) Therapy System. This product is marketed for the treatment of Major Depressive Disorder in patients who have not benefited from one prior antidepressant medication. In the randomized, controlled trial submitted to the Food and Drug Administration (FDA) by the sponsor, the product did not have a clinically or statistically (p=0.057) significant impact on the depression scale that served as the study’s primary outcome variable. Moreover, the study was marred by unblinding that would likely have favored the product. 

Although the device was cleared for this indication by the FDA on October 7, 2008,[1] CMS has stated that such clearance does not automatically guarantee reimbursement under the Medicare program. Indeed, while under the Food, Drug, and Cosmetic Act, a device must demonstrate a “reasonable assurance of … safety and effectiveness”[2] before it can be marketed,[*] the Social Security Act provides for reimbursement under Medicare only if the device is “reasonable and necessary.”[3] As CMS has made clear on multiple occasions, while it typically defers to the FDA’s determinations of safety and efficacy, it determines eligibility for reimbursement under a different, more restrictive standard:

Whereas the FDA must determine that a product is safe and effective as a condition of approval, CMS must determine that the product is reasonable and necessary as a condition of coverage under section 1862(a)(1)(A) of the [Social Security] Act. CMS adopts FDA determinations of safety and effectiveness, and CMS evaluates whether or not the product is reasonable and necessary for the Medicare population. Although an FDA-regulated product must receive FDA approval or clearance … for at least one indication to be eligible for Medicare coverage, … FDA approval/clearance alone does not generally entitle that device to coverage.[4]

In our view, the device has never met the FDA’s standard for approval, much less CMS’s more rigorous one.

Legal Background

The FDA determines whether the criteria for approval or clearance have been met using one of the two general premarket review mechanisms created in the 1976 Medical Device Amendments to the federal Food, Drug, and Cosmetic Act. The first, Premarket Approval (PMA), is analogous to the New Drug Application for drugs. A sponsor must directly establish the safety and effectiveness of its device through a well-controlled study, generally a randomized controlled trial (RCT).[5] In the second, the 510(k) pathway, a sponsor attempts to infer the safety and effectiveness of its device based upon a comparison to another, currently marketed (“predicate”) device.[6]  With the FDA Modernization Act of 1997, Congress created another premarket review procedure, called the de novo process, for devices rejected in the 510(k) pathway. If a device is rejected under 510(k), it typically has to undergo review under the more stringent PMA process if the sponsor elects to pursue approval. Through the de novo process, the sponsor may instead request that the FDA determine whether the device can be cleared even without identifying a predicate device. A de novo pathway request circumvents filing another 510(k) or a PMA application in order to reach the market.[7] However, if a device is cleared in the de novo process, the safety and effectiveness of the device has not been established through well-controlled trials in the PMA pathway nor through a comparison to a legally-marketed device in the 510(k) pathway. Once cleared in the de novo process, the device may serve as a predicate device for future 510(k) submissions. It is this newer pathway through which the FDA ultimately cleared TMS in October 2008. 

In the NCD process, a company (typically) seeks a favorable determination by CMS. CMS then initiates an evidence-based process to determine whether the criteria for reimbursement (“reasonable and necessary”) have been met. This may include an outside technology assessment and/or referral to CMS’ Medicare Coverage Advisory Committee. However, CMS regulations provide for the filing of NCDs by entities other than the manufacturer:

A request to make an NCD can be received from an individual or entity who identifies an item or service as a potential benefit (or to prevent potential harm) to the Medicare population; this requestor can be either an aggrieved party as defined by section 522 of [Benefits Improvement and Protection Act], or a nonaggrieved party.[8] (emphasis added) Therefore, we are requesting that CMS initiate an evidence-based evaluation of the NeuroStar TMS Therapy System to determine its suitability for reimbursement under the Medicare program.

Background on Transcranial Magnetic Stimulation

The NeuroStar TMS system is a computerized electromechanical device that directs brief-duration, rapidly-alternating magnetic fields to the neurons of the left dorsolateral prefrontal cortex, a part of the brain associated with mood regulation. The device components include console with controller interface, magnetic coil, treatment chair with head support, and software. TMS is performed by a psychiatrist as an outpatient procedure. A full course of TMS consists of daily treatment sessions lasting approximately 40 minutes each, over four to six weeks (20-30 sessions). At an estimated cost of $300 per session,[9] a complete treatment would cost between $6,000 and $9,000.

Pivotal Clinical Trial

Expecting to demonstrate the efficacy of TMS over the short-term, the sponsor conducted a randomized, placebo-controlled trial. Three-hundred and one patients were enrolled, randomized to either active treatment or sham treatment (machine mimicked treatment without applying a magnetic field) and were required to complete at least one follow-up assessment. Concomitant antidepressant medications were not permitted during the study. The primary effectiveness endpoint, measured at four weeks, was the Montgomery Asberg Depression Rating Scale (MADRS) and a standard-effect size of 0.40 was specified as the minimal level for clinical significance. In addition, Neuronetics prespecifed 26 clinician- and patient-rated secondary outcome measures.  Patients who did not respond to either active or sham TMS on the primary endpoint could withdraw from the study and enroll in a separate nine-week uncontrolled, open-label study. In December 2007, the results of the randomized, controlled trial were published in Biological Psychiatry.[10] We have previously offered critiques of this trial in testimony before the Neurological Devices Panel of the Medical Devices Advisory Committee[11] and in a letter published in Biological Psychiatry.[12]

In that pivotal trial, the difference between patients treated with active and sham TMS was clinically minor (1.7 points on the 60-point MADRS; standardized effect size=0.39) and statistically non-significant (p=0.057);[10] the difference became statistically significant (p=0.038) only after the post hoc exclusion of six patients who had met a priori inclusion criteria.[12] Of the 26 secondary outcome variables, half met statistical significance favoring the treatment at a p-value of 0.05. However, this ignored the need to correct for multiple comparisons. After a conservative correction for multiple comparisons, none of the 26 secondary endpoints was statistically significant.[13] The sponsor’s multiplicity adjustment did produce some positive findings, but was post-hoc and only included 13 of the 26 secondary endpoints.

Furthermore, during the trial, patients may have been inadequately blinded, potentially creating a bias in favor of finding treatment effectiveness. For instance, application-site pain was nearly ten-fold greater in TMS-treated patients compared to those receiving sham treatment (35.8% vs. 3.8%). And after adjustment for potential unblinding using an inclusive measure of pain, the FDA concluded that “the report of any pain in aggregate could account for some of the observed treatment effect [in the primary endpoint].”[14]

In the subsequent nine-week open-label study, patients from both groups in the pivotal trial who did not respond were followed and administered TMS. The sponsor reported significant decreases in MADRS scores for both groups at six weeks (17.0 in the sham non-responders and 12.5 points in the TMS non-responders).[15],[16] However, the patients were aware of their treatment status. In addition, the decreases in MADRS scores in both studies can be partially explained by regression to the mean: the selection of patients with relapsing conditions who enroll in studies when they are near their worst clinically and are thus likely to improve, on average, with or without further treatment. Regardless, the importance of the uncontrolled, open-label study findings is overshadowed by the negative outcome in the more scientifically rigorous randomized controlled trial.

Lastly, TMS is not a benign treatment. Compared to patients receiving sham treatment, those receiving active TMS experienced markedly more muscle twitching, application site pain and discomfort (20.3% vs. 3.2%; 35.8% vs. 3.8%; 10.9% vs 1.3%, respectively).[17] However, serious adverse events occurred at similar rates in both treatment arms.

Regulatory Voyage through the Food and Drug Administration

The 510(k) Pathway and the Advisory Committee Meeting

In our view, TMS should have been reviewed as a PMA,[18] but Neuronetics sought clearance for TMS in the 510(k) pathway claiming substantial equivalence to electroconvulsive therapy (ECT). Despite this claim, the company conducted no studies directly comparing the two devices. Instead, the company conducted a placebo-controlled trial and provided only historical data for ECT, mostly more than two decades old and with a different outcome measure (17-item Hamilton Depression Rating Scale or HAMD₁₇ instead of the MADRS). Still, ECT was more effective than TMS (on the 55-point HAMD₁₇, decreases of 9.7 points for ECT and 1.7 points for TMS).[19]

In January 2007, the FDA convened an advisory committee to discuss and make recommendations on the 510(k) submission for TMS. At the meeting, the sponsor contended that, although TMS was less effective than ECT, it could still be deemed substantially equivalent because the lower toxicity of TMS resulted in a similar risk-benefit ratio. The committee’s “consensus was that the efficacy was not established; some [committee members] stated that the device’s effectiveness was ‘small,’ ‘borderline,’ ‘marginal’ and ‘of questionable clinical significance.’”[20]

The committee also addressed the post hoc exclusion of the six patients (which yielded the statistically significant finding on the MADRS): “Considering that the magnitude of the effect size (sic), the Panel thought that post-hoc analysis was problematic.”[20] 

Regarding the multiple secondary endpoints, the FDA statistician asserted, “[S]econdary endpoints should only be looked at after the primary endpoint has been met … If we are willing to look past the fact that the primary endpoint was not significant at a strict 0.05 level, in an effort to consider the totality of the evidence, we would still have the significant problem of multiplicity.”[21]

The FDA subsequently determined that TMS was not substantially equivalent to ECT.[22]

The De Novo Pathway

Although the largest randomized controlled trial of TMS ever conducted (it included 2.6 times as many patients as the next-largest trial [23]) failed to demonstrate efficacy on its primary endpoint and, at best, half of its 26 secondary endpoints, Neuronetics persisted. Prior to the advisory committee meeting, the FDA had requested that Neuronetics perform several post hoc evaluations of the trial data, one of which identified a significant treatment effect among patients failing only one prior antidepressant medication (p=0.001).[24] With this knowledge, the company requested review of TMS under the de novo pathway for a more focused (and clinically confusing) treatment population – treatment of depression in patients who had failed to respond to only one trial of antidepressant medications in the current episode (not zero; not two or three; just one).[25],[26]

Post hoc analyses are problematic for multiple reasons. They may be motivated by prior inspection of the data, a potential source of bias.[27],[28] As the chairperson of the advisory committee that considered TMS stated regarding the stratification of patients according to prior treatment failure, “trying to stratify in this fashion and draw conclusions when the overall p-value is in the range that it is, is done with great peril.”[29] Furthermore, post-hoc explorations can conceal unfavorable findings. In the case of TMS, dichotomizing patients as one versus two to four prior treatment failures obscures the fact that a small group of patients (n=12) with four such failures also appeared to respond to TMS (p=0.022), a finding that undermines the sponsor’s conclusion about the impact of treatment resistance on the response to TMS.[30]

Because of an inflated risk of false-positive results, post hoc observations should always be treated with skepticism,[31] especially when they are associated with negative trials.[32] However, in this instance, the post hoc analysis formed the basis for clearance.[33]

Similarity to a Previous National Coverage Decision

In 2006, CMS faced a NCD for another medical device for depression, vagus nerve stimulation (VNS). Like TMS, the pivotal randomized controlled trial for VNS failed to demonstrate a statistically significant difference in the prespecified primary outcome variable.[34]  Also like TMS, the FDA disregarded these findings, instead relying upon less-rigorous evidence to approve the device.[35] As a Senate Finance Committee report subsequently showed, even within the FDA the approval decision for VNS was highly controversial.[36] However, when CMS considered a NCD for VNS, it concluded that the device was neither reasonable nor necessary. Going still further, CMS stated that, “After a thorough review of the current evidence, CMS does not believe there is a treatment benefit directly attributable to VNS therapy for [treatment resistant depression].”[37] We believe that CMS acted appropriately and courageously in the VNS case and urge it to resolve this issue similarly.

Summary

The FDA clearance of TMS has not provided assurance that it is an effective treatment for depression, much less that it is a treatment that is “reasonable and necessary” for the Medicare population. Certainly, the evidence mustered for this device does not approach the rigor typically required of anti-depressant drugs. Unless better scientific evidence emerges, CMS would be prudent to deny coverage of TMS, especially because patients are likely to be diverted from clearly effective therapies, such as antidepressant medications to this unproven device. 

Sincerely,

Jonas Hines
Research Associate

Peter Lurie, M.D., M.P.H.
Deputy Director

Sidney M. Wolfe, M.D.
Director
Public Citizen Health Research Group 

---

[*] This is a lower standard than for drugs which must demonstrate “substantial evidence of effectiveness for the claimed indications.” (21 CFR 314.50(d)(5)(v)) 

---