NEW YORK – To analyze oncology slides behind hospital firewalls without compromising on data security, artificial intelligence firm Owkin said it used a federated learning study to identify potential biomarkers of response to neoadjuvant chemotherapy while keeping patient-specific data secured within each hospital.
In an article published last month in Nature Medicine, the authors from Owkin, along with collaborators at four hospitals in France, said their retrospective study applied machine learning software to whole-slide pathology images from patients at two of the hospitals and validated the findings on images from the other two hospitals. Through this method, the authors said they not only developed a more accurate global model for predicting treatment success than they would have by using data from only one of the institutions, but they also discovered potential biomarkers that, with further validation, could improve predictions of which patients would respond well to neoadjuvant therapy.
The study developed the model across institutions using federated learning, a concept that involves applying machine learning to data that remains siloed at each data source and sharing only the summaries or learned model weights to create a global model. Research teams can use such a model to gain access to greater volumes of patient records by agreeing not to copy patient-specific data, which helps them comply with legal and institutional restrictions on handling and storing private data as well as reduce the risk that private information could be stolen through a breach at a central server.
Owkin researcher Jean du Terrail said the federated model is particularly useful for securing sufficient data to build accurate machine learning models in studies of rare diseases, although he said few research teams are applying the method in real-world conditions.
"We really believe that federated learning is the future because it's the only way to unlock data access," he said.
His team hoped that their study results would help physicians identify which patients were likely to benefit from neoadjuvant chemotherapies, the results of which are difficult to predict using current clinical practice.
"The benefit is very important for rare diseases such as triple-negative breast cancer because when you take a look at only one institution, only one center, you might have a biased view of the problem," he said. "And so, because AI and machine learning algorithms are absorbing potentially all the correlations and the patterns that they find in the data, it's really important to have access to multiple centers to understand the full etiology and the full complexity of the underlying problem."
He sees two avenues to pursue based on the results of this proof-of-concept study: His team would like to use the results to help guide clinical trials on the prospective biomarkers identified in the study, and they also would like to establish links between the histological patterns seen in the study to the biological processes behind those patterns within triple-negative breast cancer. Those mechanisms could reveal treatment targets, he said.
Mathieu Galtier, chief data and platform officer at Owkin, said the recent study results grew from a project started in 2018 with the goal of proving that federated learning could help research collaborators jump the hurdles of technical, data security, privacy, legal, and validation concerns on their way to answering a clinical question. The biotech firm is banking on its machine learning models to discover new drugs and treatment targets, and Galtier said the triple-negative breast cancer study results have helped establish federated learning as foundational for that research.
Galtier declined to specify how Owkin plans to apply those results. But, since the article's publication in Nature Medicine, representatives from many more hospitals have contacted the firm about their own data and research projects and he sees a chance to grow its network of collaborators.
The four French hospitals participating in the study — Centre Léon Bérard in Lyon, Institut Curie in Paris, Institut Gustave Roussy in Villejuif, and Institut Universitaire du Cancer Toulouse Oncopole in Toulouse — gave Owkin access to a combined total of 686 slides from 676 patients. Of those slides, 307 came from patients who had complete responses to neoadjuvant chemotherapy and 379 had ongoing cancer burden following treatment.
The largest cohort in the study came from Institut Curie in Paris, which provided access to 427 slides associated with 420 triple-negative breast cancer patients. Using 367 of those slides as a training set and 60 as a testing set, the researchers said they developed a model that predicted which patients would benefit from neoadjuvant chemotherapy with an average area under the curve across all test centers of 0.64. They developed another model with patient slides from Centre Léon Bérard in Lyon by using 82 slides for training and 20 for testing, and that model had an average area under the curve across all test centers of 0.60.
The best federated models created by the research group, however, had an average area under the curve across all test centers of 0.66. The authors also created a machine learning model that they trained on Elston and Ellis histological grade and the percentage of tumor-infiltrating T lymphocytes, a combination that is intended to reflect current clinical practices in predicting triple-negative breast cancer patients' response to neoadjuvant chemotherapy, and they found that their best clinical models had an average area under the curve across all test centers of 0.63.
The authors wrote that the model homed in on biomarkers that have well-known links to response to neoadjuvant chemotherapy such as tumor-infiltrating T lymphocytes, which are linked with higher likelihood of response, and apocrine tumor cells, which are associated with higher risk of poor response. It also identified potential new biomarkers, and additional study could reveal whether the presence of necrosis is linked with higher likelihood of complete treatment response and whether fibrosis is linked with higher risk of poor response.
"A more quantitative study to assess the direct impact of each criterion would be needed to validate those biological insights," the authors wrote.
Galtier noted that the machine learning models developed only on single-institution training sets didn't work as well as the federated model when applied to slide images from the other institutions.
"This is extremely worrying because it means that all of the research which is done on [data from] single hospitals is likely to be very unreliable," he said.
Single-institution machine learning models with seemingly impressive accuracy may, in fact, reflect a model that is overfitted to that institution's data and would deliver poor results with additional testing, Galtier said. Even among two seemingly similar hospitals in France, Galtier said the differences in clinical techniques, equipment used in the hospital, software used to analyze the data, demographics of patients, and criteria for including patients all potentially contribute to difficulties in developing a machine learning-based prognostic that is robust and generalized enough to work across clinical centers.
"What we have reached is the robustness and ability to build models which are going to be applicable across different hospitals," Galtier said. "And this is the key result, from my perspective — from this paper — is the fact that we have built models which generalize much better, which is the most worrying part of machine learning."
A global bias in favor of data from Caucasian patients also exacerbates problems in model development and presents a structural problem for researchers, Galtier said. Owkin's federated learning models can also help with this problem because, by design, they incorporate heterogeneous data and include outliers that would be excluded from other studies, he said.
While privacy concerns and regulation are often cited as hurdles to overcome by federated learning, Galtier thinks competition between researchers is likely a key factor preventing more data centralization. The researchers who collected patient data often want to maintain control of their datasets, and federated learning respects that competition while unlocking the power of those data.
"We have managed to gather 10 pharma companies, working on the same or similar topics, all of them big competitors, and we found a way to make them work together," Galtier said.
Jianyu Rao, of the UCLA Department of Pathology and Laboratory Medicine, said the study's federated model does confer a big advantage in securing access to data and it may help validate data from each cohort by comparing them across institutions. Rao noted that the study authors focused on a specific, unanswered clinical question about how to predict which breast cancer patients will benefit from neoadjuvant therapy, and he sees the results as a good foundation for additional studies that could prove whether the potential biomarkers they identified are useful.
However, Heather Couture, a machine learning consultant who specializes in pathology applications, cautioned that it may be too early to be overly excited about the method. Federated learning remains early in development, she noted, and the Owkin-led study, as well as study results published in April 2022 on the use of AI in cancer histopathology, used small numbers of training cohorts that show the feasibility of federated learning but don't yet make a strong case for its benefits.
That article, also published in Nature Medicine, used swarm learning-based analysis histopathology images from more than 5,000 patients in the US, Germany, and Northern Ireland, and the authors said the results suggest such a model could predict BRAF mutational status and microsatellite instability from stained slides of colorectal cancer. Once researchers can scale up such studies to include larger numbers of cohorts, the results will show whether federated learning can make a difference for patients, Couture said. She said the results from the Owkin-led research and the international histopathology image study are both encouraging, though, and Owkin's results are an important development.
"We're not there yet," she said. "This is a step in that direction."
In another article published in September in Nature Communications, researchers said their international federated learning-based study used data from 6,300 glioblastoma patients at 71 sites to produce a model for detecting tumor subcompartment boundaries, which they said could aid neurosurgical and radiotherapy planning. In the article, the authors said training robust and accurate models requires large amounts of data, yet data centralization can be difficult to scale because of challenges involving privacy, data ownership, intellectual property, hardware limitations, and regulations. By sharing only model parameter updates from decentralized data, a federated learning model can offer increased size and diversity of datasets without sacrificing performance relative to centralized learning models.
BC Platforms has also developed a federated AI learning platform for its global genomic and clinical database network to speed up R&D while preserving patient privacy and intellectual property rights. Lifebit Biotech has been working on federated sharing that could make genomic data more widely available.
Owkin's du Terrail said ramping up federated learning studies could present some challenges because of the regulatory processes to bring many hospitals on board. But he said the results from the study his company and its collaborators performed provide proof of concept that such a model could help unlock access to the data needed to study rare cancers and, more broadly, rare diseases.
In addition to Owkin's studies on predicting response to treatments in cancer patients, Galtier said the company is also working with pharmaceutical companies on drug discovery projects and building a research network that would use data from pharmaceutical companies and hospitals to improve the predictive power of Owkin's studies, although he said non-disclosure agreements prevent him from providing details.
Sanofi said in November 2021 that it was investing $180 million in equity into Owkin's artificial intelligence and federated learning capabilities and the firms would partner on research into discovering and developing treatments for non-small cell lung cancer, triple-negative breast cancer, mesothelioma, and multiple myeloma. Sanofi said it would leverage Owkin's machine learning platform to analyze data from hundreds of thousands of patients, identify biomarkers and treatment targets, build prognostic models, and predict responses to treatment.
"Sanofi's investment will support Owkin's development and goal to grow the world's leading histology and genomic cancer database from top oncology centers," Sanofi said at the time.
Also, in June 2022, Bristol Meyers Squibb agreed to invest at least $80 million into Owkin for the development of more precise and efficient clinical trials for cardiovascular disease therapies. The firms had already collaborated for years on successful projects to identify biomarkers and improve clinical trial outcomes with covariate adjustment, Owkin said.
Galtier said Owkin is also building a network of hospitals with data the firm could scour to identify ways to improve treatment of glioblastoma. That is developing into one of the firm's most important projects and a potential source of intellectual property as it discovers treatment targets, drugs, and subpopulations that would benefit from certain treatments.