Report critical of U.S. medical device approval process is flawed
Dr. Sidney Wolfe is director of Public Citizen’s Health Research Group. Dr. Michael Carome, deputy director of Public Citizen’s Health Research Group, also contributed.
A new report by Boston Consulting Group is being cited by the medical device industry as evidence that the slower and more deliberate U.S. approach to regulating devices doesn’t boost safety compared to a faster process in Europe. In fact, the study is flawed in a variety of ways, as explained below.
The report was funded by AdvaMed, a major medical device company trade group strongly opposed to many of FDA’s efforts to strengthen the device approval process. Furthermore, the Boston Consulting Group’s clients include many medical device companies. As a result, the individuals who conducted the study and authored the report are highly biased, and the report should be viewed with extreme skepticism.
A clue as to the spin AdvaMed wants to create by this report can be found on their web site:
“The FDA’s longer process may deny patients access to the most up-to-date treatments and cures without a corresponding increase in safety,” said Stephen J. Ubl, president and CEO AdvaMed, which paid for the study.” The study provides no evidence “the FDA’s longer process may deny patients access to the most up-to-date treatments and cures.”
The following are some of the many other flaws in this report:
• The authors of the study relied on several assumptions which likely are not valid:
o They assume that there is the same threshold for issuing a safety alert in Europe and in a recall notice in the US; they present no data to support this assumption.
o They assume there is the same threshold for issuing a safety alert across all countries in Europe; the fact that they report only data for 5 EU countries suggests that this assumption is not valid.
o They acknowledge that there is no comparable public database that captures all medical device safety recalls in the EU as the FDA does in the US; they then assume that publicly posted medical device field safety notices on EU member state websites accurately reflect all serious safety events in the EU; they present no data to justify this assumption except to note conversations they had with EU regulatory officials.
• The description of the methodology used by the authors is difficult to follow and lacks important details; such a report would never successfully pass through a rigorous peer-review process. Based upon the information that is presented, the study has several flaws:
o Because EU system does not categorize safety notices into the US equivalent of class I, II, and III recalls, the conductors of the study classified EU safety notices into these three classes. The authors fail to describe the criteria used for making classification decisions. A good research study would pre-specify such criteria.
o The report authors only include data on the total raw total number of class I and class II recalls in the 5 selected EU countries and the total raw number that matched class I and class II recalls in the US. Such raw numbers are meaningless in the absence of (a) denominator data on the number of devices approved across the EU versus the US; and (b) the number of devices sold and used across the EU versus the US.
• In comparing EU notices matched to US FDA recall notice, the authors note that on average, EU notifications were posted approximately three weeks earlier. This suggests that recalls in the US and in the EU occur at approximately the same time. A much more important parameter to compare would be the time interval between when the FDA approved a device and when it is recalled versus the time interval between when the EU approves a device and a recall occurs. If recalls occur at approximately the same time, but device approval occurs sooner in the EU, this would indicate that patients in the EU are exposed to a harmful device that needs to be recalled for a longer time than patients in the US.
• The authors failed to identify the many limitations of their study, as would be done in a peer-reviewed scientific article.