NEW YORK – The US Food and Drug Administration and at least one state department of health have recently spearheaded projects to make more transparent the strengths and limitations of lab-developed tests used to personalize treatment for patients with cancer and identify those at risk.
The FDA last week announced a pilot program, through which it is hoping to "reduce risks associated with using LDTs to identify cancer biomarkers." Drugmakers can voluntarily submit information about the performance of biomarker tests used to enroll cancer patients into clinical trials that support approval of specific drugs. After assessing the submitted analytical validation information on the clinical trial assays, such as data on test accuracy, limit of detection, and precision, the agency plans to publicly post recommended minimum performance characteristics that labs can use as a benchmark for developing other tests on the market that haven't gone through FDA review, but which can be used to identify best responders to these same drugs.
The FDA is hoping to "facilitate development of better and more consistently performing tests, resulting in better drug selection and improved care for patients with cancer," an agency spokesperson said.
Earlier this month, as part of a separate project, the Utah Department of Health and Human Services (UDHHS) sent letters to 11 labs requesting that they fill out a survey to share validation and other information about their LDTs for assessing hereditary cancer risk. The nonprofit Center for Genomic Interpretation (CGI) received a two-year grant from the UDHHS Cancer Genomics Program to develop the survey with the goal of providing healthcare providers better insight into the similarities and differences between so-called "comprehensive hereditary cancer genetic tests" on the market.
The deadline for completing the survey, which has also been endorsed by the Washington State Department of Health and the Oregon Health Authority, is July 21, at which point CGI will post the test information on the project website TestWisely.org. In awarding the grant, UDHHS indicated it hopes to "spread awareness of hereditary cancer risk and increase uptake of preventive services" and encourage the development of support systems that "ensure equitable access to high-quality genetic services."
Though participation in both the FDA and UDHHS programs are voluntary, they come as the FDA is planning to advance LDT regulations through notice-and-comment rulemaking and hasn't given up on reforming diagnostics regulation through legislation. The emergence of these projects suggests that despite the lab industry's insistence that its tests are of sound quality, there is ongoing concern among state and federal health regulators about the performance of LDTs being offered to patients through CLIA-certified labs but without FDA review.
While under present policies labs can perform LDTs in a CLIA-certified lab without FDA approval, the agency has maintained it has the authority to regulate lab tests and has long felt that the growing technological complexity and broader marketing of these tests have put the public health at risk and warrant more oversight. However, the lab industry and the pathology community have repeatedly stymied the FDA's past efforts to regulate LDTs by challenging the agency's statutory authority and asserting that it has never proven that lab tests are harming the public health.
The FDA has had some success in establishing that it needs to review the safety and efficacy of companion diagnostics, which the agency considers high-risk tests required for the safe and effective use of a drug. However, for every companion test the FDA has approved for identifying patients eligible for a drug, there are often many LDTs on the market that claim to do the same thing.
"The FDA is concerned that LDTs used to make critical healthcare decisions for patients may not all be accurate and reliable," the agency spokesperson said, explaining the rationale for the pilot. "This is especially concerning for situations in which drugs require use of a diagnostic test to select patients for treatment but where there is not an FDA-authorized companion diagnostic."
In these situations, the agency is hoping that labs will adopt the minimum performance characteristics it proposes through the pilot when developing LDTs that can help select patients for treatment.
In a guidance released in tandem with the pilot's launch, the FDA said that clinical trial assays eligible for the program must employ technologies that have underpinned FDA-authorized companion tests, such as next-generation sequencing, immunohistochemistry, and fluorescent in situ hybridization, and they must be comparable to other well-validated tests to determine accuracy. For these types of clinical trial assays, the "FDA believes that, in general, the clinical validity … can be extrapolated to additional tests of the same type with similar analytical performance, when established through properly conducted validation studies," an agency spokesperson said.
As for the genetic tests for hereditary cancer risk that are the focus of the UDHHS program, there are none that have undergone FDA review, CGI CEO Julie Eggington noted. "You often don't know what you're getting from these tests, whether they are LDTs and even with companion diagnostics, unless the company has published a really in-depth paper," Eggington said.
Regardless of where the FDA's LDT regulation lands, by shedding light on the performance of lab tests, the FDA and UDHHS projects could create market pressure to improve performance and align metrics, but industry insiders' initial reactions to these efforts have been mixed. While some see potential in the FDA's pilot to standardize performance of certain oncology tests on the market, others say the pilot runs counter to the agency's message through the years that high-risk LDTs, such as those used to personalize patient treatment, need its oversight. And when it comes to UDHHS's survey on hereditary cancer tests, it remains to be seen whether labs will ultimately participate, though several firms have said they're considering the opportunity.
Parsing the pilot
The scope of the FDA's pilot is narrow, focusing on oncology drugs that the agency is willing to approve without a companion diagnostic. A likely candidate may be a cancer therapy the FDA is reviewing at an accelerated clip for treating patients with rare biomarker-defined tumors who lack other treatment options.
Such scenarios are not uncommon in the precision oncology space. For example, in 2018, the FDA granted accelerated approval to Bayer's TRK inhibitor Vitrakvi (larotrectinib) as a treatment for a refractory tumor with NTRK gene fusions. Less than 1 percent of tumors have these tumor alterations, and in the interest of getting the drug to patients as quickly as possible, the agency didn't require that Vitrakvi be launched with an FDA-approved companion test for identifying treatment-eligible patients.
The initial phase of the FDA's pilot will involve at most nine drug sponsors that choose to submit analytical validation and performance characteristics for clinical trial assays either before enrolling pivotal studies for their drugs, or in certain circumstances, the agency may allow drugmakers to submit data on clinical trial assays if the pivotal study already started. The agency has provided templates that drug sponsors can use to submit performance metrics on NGS, PCR, Sanger sequencing, IHC, and FISH tests. The NGS template, for example, gives sponsors the option of reporting how well assays detect a representative set of genetic variants of different types, such as single-nucleotide variants, rearrangements, and copy number variants.
Elizabeth Mansfield, who in 2017 left her position as head of the personalized medicine group within the FDA's device division to join industry, questioned the extent to which a program like this would spur labs to align LDT performance with the agency's recommended metrics. Mansfield, who since 2020 has been VP of regulatory affairs at Foundation Medicine, one of the few companies that have garnered FDA approval for its tissue and liquid biopsy-based NGS tumor profiling LDTs, participated in calls the agency held to gather stakeholder feedback on the pilot. After reviewing the FDA's guidance on the topic, she characterized it as "quite vague," leaving questions unanswered.
Mansfield's understanding, based on the guidance, is that the FDA will publish minimum performance characteristics that labs can choose to conform to, but the agency will not identify the clinical trial assays on which the recommended metrics are based. And while the agency is providing templates for submitting test validation data, those won't necessarily "define a bar that must be reached" for LDTs on the market, she said. As an example, she pointed out that in the NGS test template, the FDA "appears to allow representative validation as being sufficient rather than validating a test's performance on an actual biomarker."
The FDA hasn't provided any indication that it will enforce that LDTs marketed for indications similar to clinical trial assays are conforming to its recommended minimum performance specifications. Moreover, the guidance doesn't address other parameters the agency considers when ensuring the safety and efficacy of diagnostics, such as post-market modifications, quality systems, or adverse events reporting. "Effectively, it looks like most tests used in therapy selection for cancer patients will remain unregulated LDTs, except for FDA-approved companion diagnostics," Mansfield said.
While this pilot doesn't change the FDA's current guidance that companion diagnostics necessary for the safe and effective use of a drug garner its approval, it does allow the agency insight into the potential impact of future policy changes. "It's … a question for the FDA as to what tests they want to encourage companies to develop," Mansfield said. "Is it a test that just meets a minimum level of standards or is it a high-quality test that patients and physicians can have a high level of confidence in to inform diagnosis, disease progression, and treatment options?"
Richard Bender, who has led medical affairs operations at numerous molecular diagnostics companies, including Agendia, Caris Life Sciences, and Signal Genetics, and consulted with many others, worried whether the FDA getting involved in setting minimum performance criteria for biomarker tests might slow product development timelines. Bender, who is a practicing medical oncologist and an FDA consultant, is not against the FDA reviewing the analytical and clinical validity of new tests, but he believes it might be more efficient to update CLIA regulations so that in addition to considering labs' analytical validity claims through proficiency testing, as is done presently, the clinical validity claims of these tests are also vetted. "That's what's missing from CLIA right now," he said.
Bender recounted how interactions between Agendia and the FDA moved so slowly over the two years it took to get 510(k) clearance for the MammaPrint breast cancer recurrence test that "it was almost discouraging." If the FDA's pilot helps shed more light on the analytical and clinical validity of LDTs used to direct therapy "without encumbering us with too much red tape and tardiness in terms of moving products forward, then there may be some benefit to this," he said. "But it needs to be carefully evaluated and not just dropped in FDA's lap."
In the guidance, the FDA states that in the pilot it will focus on clinical trial assays that use well-understood technologies like NGS, IHC, and FISH and can be compared to well-validated reference methods, so that the clinical validity of the assay can be extrapolated to other similarly designed LDTs. But, as Mansfield pointed out, "the pilot is not conducting clinical validation nor is it clear that any test would be reviewed to assure validity," which raises further questions about how the FDA hopes to mitigate the risk associated with LDTs.
Labs … interested?
While the FDA is leaving it up to drugmakers to express interest in partaking in the pilot, it remains to be seen how enthusiastic labs providing assays within clinical trials will be about test performance information being shared within the program. Although this program is only in pilot phase, some industry observers are already wondering how it would impact the lab industry if the FDA began publishing minimum performance characteristics for biomarker tests more broadly.
"If the FDA weighs in and sets a minimum performance level for say sensitivity and specificity, while this might appear to be a good thing, this will stifle ingenuity and smaller labs will simply fade away because the development costs to meet some arbitrary number that the FDA publishes will drive them out of business," worried Jeffrey Jones, who has previously worked to commercialize molecular diagnostics at Agendia and SkylineDx and now is a managing partner at the consulting firm Deerborne Group.
For the time being, likely due to the pilot's limited scope and less prescriptive nature — for example, while the FDA is collecting performance data on clinical trials assays, it is not requiring they undergo regulatory review — the lab industry doesn't seem opposed to it. A spokesperson for the American Clinical Laboratory Association said the lab industry group is reviewing the FDA's recent guidance on the pilot but noted that the additional information that the agency wants to share about the performance of clinical trial assays used in cancer drug studies stands to "facilitate the development of innovative tests by laboratories" that meet the needs of physicians and patients and make companion diagnostics more accessible.
The spokesperson further noted that the organization expects "there will be interest among ACLA's membership in this program."
Roger Klein, a board-certified molecular pathologist and attorney with expertise in diagnostics regulatory policy, hasn't supported many of the FDA's past attempts to extend its oversight of LDTs through guidance, but he also sees promise in the latest pilot. Klein, who until recently was head of medicine at Labcorp Oncology's OmniSeq division, told the FDA in written comments that its pilot could potentially "encourage optimal care of cancer patients if appropriately implemented."
Diagnostic industry players have long felt that the FDA's companion diagnostic framework — where the agency approves one or a few tests for identifying best responders to specific drugs — doesn't comport with the practice of precision medicine in the real world where doctors have a variety of LDT options through CLIA-certified in-house or commercial labs in addition to FDA-approved companion tests. Because doctors can order a multiplicity of tests to direct treatment, Klein believes that if the present pilot is successful then the agency should expand the program to establish minimum performance characteristics for tests in treatment scenarios where there are FDA-approved companion tests.
Commercial labs, under fierce competitive pressure, will be interested in aligning the performance of their tests to FDA-recommended minimum performance metrics, Klein expects, especially if the metrics make logical sense and oncologists demand it. "There can always be outlier labs, but one suspects FDA-recommended characteristics will become a de facto standard, to which most labs will want to adhere unless there are sound reasons not to do so," he said, adding that some LDTs may even exceed the performance of clinical trial assays.
Labs that obtain a CLIA license through accreditation by a deeming organization like the College of American Pathologists are inspected to see if they comply with very detailed molecular pathology checklists, which probe some of the same test performance characteristics that the FDA might recommend through its pilot, Klein noted. "Given it is a pilot program, we will need to see what FDA will be recommending," he said, further suggesting that in the future CAP may even be willing to add questions to checklists that require labs to meet FDA-recommended performance metrics for certain types of tests or justify why the test metrics differ.
VALID considerations
The announcement of the FDA's pilot immediately brought up the agency's troubled history trying to regulate LDTs. After several failed attempts to regulate LDTs piecemeal through guidance, the agency has backed legislation, called the Verifying Accurate Leading-edge IVCT Development (VALID) Act, which would reform diagnostics regulation for both IVD kits and all LDTs. The bill, which failed to pass through Congress last year but has been reintroduced in the present session, would require high-risk tests like companion diagnostics to undergo FDA premarket review but grandfather most LDTs already on the market and allow lower-risk tests to be commercialized as long as a representative test using the same technology is certified by the agency.
Although some in public policy circles think there's little chance of this bill moving through a divided Congress, to Deerborne Group's Jones, the FDA's pilot "feels eerily like the VALID Act in the sense that the FDA is once again attempting to flex their muscles."
The FDA spokesperson said that as the agency works through the rulemaking process to propose LDT regulations, as it has said it plans to possibly in August, the pilot program can be one way of "reducing the risk of using LDTs for oncology drug treatment decisions." And while this pilot is separate from the agency's efforts to create a new statutory framework for diagnostics via VALID, the spokesperson noted "the goals of each can be complementary."
Foundation Medicine, having garnered FDA approval of its NGS tumor profiling panels, supported VALID in the last congressional session, seeing it as a way to modernize the regulatory framework for all LDTs. "The pilot program, on the other hand, is a step in the opposite direction," Mansfield said. "Under the pilot, it is hard to see how the FDA will address patient safety issues and ensure that all tests used to inform therapy selection meet minimum performance characteristics and are clinically validated."
Proving there's a problem
CGI's Eggington, for one, is "ecstatic" about the FDA's pilot, the aims of which she said are very complementary to the project her nonprofit is leading in Utah. Even though the FDA's program is focused on biomarker tests for directing cancer therapy, and CGI's efforts with UDHHS are centered on hereditary cancer testing, both programs are trying to shed more light on the performance of lab tests.
"The FDA is saying, 'Let's make public the performance metrics of what it takes to be a companion diagnostic,' and what we're doing is having states ask labs what the metrics of their [non-FDA-approved] LDTs are," Eggington said, suggesting that data generated within both efforts can help oncologists and genetic counselors better understand the genetic tests they're ordering for cancer patients.
Hypothetically, if the FDA were to post minimum performance characteristics for germline hereditary cancer tests used to direct PARP inhibitor treatment through its pilot, for example, providers could then check CGI's TestWisely.org project page to see if hereditary cancer tests they order from major labs meet the FDA-recommended benchmark. CGI is seeking funding to run a complementary LDT transparency project for somatic tests used to direct treatment.
The present survey UDHHS has sent to the 11 labs — Ambry Genetics, ARUP Laboratories, Color Health, Exact Sciences, Fulgent Genetics, GeneDx, Invitae, Labcorp, Myriad Genetics, Natera, and Quest Diagnostics — was created by CGI based on feedback it received from around 20 genetic counselors in Utah on what they wanted to know about available hereditary cancer tests. They indicated, for example, that they wanted to understand why hereditary cancer test offerings differed in terms of genes tested and other parameters and what labs' policies were on equity and test access. As such, the survey queries how labs analytically validate their tests and classify variants, what financial aid services they have, and whether they provide multilingual support.
According to the TestWisely.org website, in response to UDHHS's letter about the survey, several labs have said they're considering participating, but others haven't responded. When GenomeWeb reached out to the 11 labs to ask if they would complete the survey by July 21, Myriad, Exact, and Invitae indicated they're still considering the request; and Color said it had not received the survey and does not plan to participate. GeneDx said that it received the letter, but since its business model has evolved to focus on whole-exome and whole-genome sequencing for genetic diseases outside of cancer, the firm is deciding whether or not to participate in the survey. Others either declined comment or didn't reply to a request for comment before press time.
While labs' willingness to partake in the project is unclear, any test information voluntarily provided by labs will be publicly available through the project website. There, doctors, genetic counselors, and patients will also find videos in Spanish and English that educate them about genetic test quality.
CGI's work with the UDHHS is in line with the nonprofit's overarching focus on improving the quality of genetic tests on the market. Although Eggington is in good company with the FDA and many others, who feel that lab standards under CLIA are insufficient to protect the public health, her vocal support for FDA oversight of lab tests hasn't been received well in certain industry circles.
So, a few years ago, when CGI began asking labs if they wanted to participate in its in silico-based quality assessment program, called ELEVATEGenetics Brilliant, to gauge whether their tests could detect and clinically interpret variants they claim to, few wanted to participate in the absence of any regulatory or reimbursement pressure to do so. But more recently, CGI has been working with insurers, like Highmark, that want test validation data beyond what labs have to provide for CLIA certification on cancer risk and drug response tests for which they are paying.
"There's a prevailing belief … that if a genetic test is on the market and insurance companies are paying for it, that it is an accurate test and that it's clinically valid," Eggington said. "And neither of those things are true."
Eggington pointed to an in silico-based analysis conducted by another group that she believes has exposed the variability in LDT quality present in the market today. In the study, researchers used mutagenized variant files and engineered wet samples to evaluate labs' abilities to detect the 56 KRAS and NRAS variants included in Illumina's FDA-approved Praxis test for guiding colorectal cancer-targeted therapy. They found that only 10 out of 19 labs detected the variants in this pilot with an accuracy similar to the Praxis test, and the authors noted a higher rate of false negatives for some single-nucleotide variants and harder-to-detect multi-nucleotide variants.
This analysis was so controversial that the College of American Pathologists, which was initially involved in the study, dropped out. Critics felt that the study set labs up for failure knowing the limitations of tests at the time and by evaluating labs' ability to detect variants that were so rare that they were very unlikely to encounter them in the real world.
To Eggington, though, this analysis captures what CGI has been seeing when it "looks under the hood" at labs — while some LDTs might be as good or better than FDA-approved companion tests, many aren't. But this problem isn't widely appreciated, she said. "One of the many reasons why Congress hasn't passed VALID, which I'm very much in favor of, is that they actually don't perceive that there is a problem," Eggington said. "There's this naive belief that … all the LDTs out there are performing just as well as the FDA-approved companion tests."
As much as Eggington likes the FDA's pilot focused on establishing minimum performance criteria for tests used to personalize cancer treatment, she recognized it will also be important to ensure that labs are incorporating those metrics into their tests. "I agree that without enforcement, it's useless," said Eggington, who suspects that "the FDA is playing the long game" when it comes to regulating LDTs, because "the first thing you have to do is prove there's a problem."