CHI Insight Pharma: Tech Update
Clinical Forecasting: A Novel Bayesian Tool for Predicting Phase III Outcomes
By Asher D. Schachter, MD (Children’s Hospital Informatics Program and Division of Nephrology, Boston, MA; Assistant Professor of Pediatrics, Harvard Medical School; CSO, Phorecaster, LLC)
In recent years, there has been an explosion in predictive technologies to help researchers select only the most promising candidates for clinical development. The need for such tools is driven by the disastrous economic consequences of late-stage failures, which account for over 60% of all drug terminations. This report describes a powerful and novel predictive tool called Bayesian network modeling and demonstrates its application in clinical forecasting. Among its many potential benefits, clinical forecasting can:
- Reduce drug development costs
- Increase median cumulative 7-year revenue per Phase III trial
- Redirect capital and human resources to development programs with the greatest likelihood of success
- Expose clinical trial subjects to fewer unsafe or ineffective drugs
- Improve the accuracy and decision-making utility of market forecasts (which currently assume that all drugs in the projection period will achieve NDA approval)
- Increase industry’s and society’s confidence in including pediatric subjects in clinical trials
Moreover, unlike existing predictive technologies such as microdosing, toxicogenomics, or ultra high-throughput screening (HTS), all of which entail significant costs in capital equipment, training, and ongoing maintenance, clinical forecasting based on Bayesian statistics is comparatively inexpensive.
Clinical Forecasting: A Novel Bayesian Tool for Predicting Phase III Outcomes begins by summarizing existing predictive technologies with particular reference to their limitations. Gene expression arrays, while providing useful prognostic information, are limited by the lability of mRNA and inconsistencies across microarray platforms. Microdosing is disadvantaged by limited databases required for the studies, unclear regulatory guidelines, and, in the case of PET studies, short trace half-lives and limited ability to distinguish between the compound and its metabolites.
With complete transparency as to data sources and assumptions, the authors show how the Bayesian network model predicted outcomes (NDA approval or failure) based on an independent dataset of 503 new chemical entities (NCEs) with an optimal accuracy of 78%. The author emphasizes that, with more complete and historical datasets of in vivo and in vitro compound data including therapeutic index ranges, the model’s performance can be even further improved.
Clinical Variables Believed Most Crucial to NCE Clinical Success
Source: Asher D. Schachter, MD
In fact, Bayesian clinical forecasting will supplement other predictive technologies to boost the confidence of decision makers in R&D. The ultimate goal is to incorporate into the model compound-specific data as well as emerging information such as pharmacogenomic and single nucleotide polymorphism data, and new data from novel HTS screens.
The author includes a retrospective case study demonstrating the application of Bayesian clinical forecasting to Eli Lilly’s Xigris (recombinant human activated protein C), a failed drug for sepsis. Based on public in vivo animal data and early Phase II human data, the model predicted that Xigris had a very low probability of clinical success.
The report concludes with an evaluation of the model’s economic impact based on a Monte Carlo simulation. The model was found to significantly reduce median expenditures per successful NCE by 39% below the industry average. Even more impressively, the model significantly increased median cumulative 7-year revenues per Phase III trial by $160 million above pharmaceutical industry revenues ― from $347 million to $507 million.
The real lessons of this fascinating exercise in clinical forecasting are that (1) there is considerable hidden value in the low-hanging fruit of terminating would-be late-stage failures. The other lesson (2) is the urgent need for industrywide sharing of data in order to improve the accuracy of predictive models and thus the ROI of pharmaceutical R&D.
Biostatisticians and decision analysts, portfolio managers, market forecasters, business development managers, and decision makers throughout the R&D organization will benefit from this report.
About the Author
Asher D. Schachter, MD, is a pediatric nephrologist and bioinformatician at Children’s Hospital Boston and the Children’s Hospital Informatics Program. Upon completion of his nephrology fellowship at Children’s Hospital Boston, Dr. Schachter pursued 2 sequential masters degrees at the Harvard-MIT Division of Health Sciences and Technology: a masters of medical science in clinical investigation and drug development and a masters of science in biomedical informatics. Dr. Schachter’s thesis focused on novel approaches for applying Bayesian networks to predictive modeling in drug development. In 2003, Dr. Schachter was appointed as a faculty member of the Children’s Hospital Informatics Program, and as an assistant professor of pediatrics at Harvard Medical School. Dr. Schachter cofounded Phorecaster LLC in 2006, with Marco Ramoni, PhD, and is Phorecaster’s chief scientific officer.