The U.S. Advanced Research Projects Agency for Health (ARPA-H) announced new funding for a “program to help AI-enabled medical tools maintain peak performance.” The new funding is through the Performance and Reliability Evaluation for Continuous Modifications and Usability of Artificial Intelligence (PRECISE-AI) program. The announcement states:

PRECISE-AI will develop techniques that analyze AI-enabled tools and identify root causes for performance deterioration. The root cause analysis will inform self-corrective actions to improve the AI-enabled tool’s performance. The program will also create mechanisms for notifying clinicians, AI tool developers, hospital administrators, and regulators when performance degradation occurs.

ARPA-H Director Renee Wegrzyn, Ph.D. stated “PRECISE-AI is addressing a growing gap in ensuring AI tools used in clinical decision-making are accurate, safe, and robust in real-world settings” and that “in doing so, ARPA-H is creating a foundation of trust between clinicians and these AI tools, which will further expand AI’s potential in improving health outcomes for all Americans.”

PRECISE-AI plans to address five technical areas (“TA”):

TA 1 focuses on the automatic extraction and integration of data across different clinical use cases to establish a ‘ground truth’ about each patient.

TA 2 seeks to continuously monitor model performance, determine root causes of degradation, and suggest or make automatic corrections when needed.

TA 3 aims to quantify uncertainty and improve clinical outcomes by finding novel ways of communicating model uncertainty and complementary measures to clinicians, developers, and other stakeholders.

TA 4 will aggregate and share data across medical institutions and across performers to advance development of TA 1-3.

TA 5 will confirm the progress made by all the TAs by performing independent verification and validation.

More information regarding PRECISE-AI can be found on its program page here. A draft program solicitation to bring together experts to develop the TAs, having a response date of January 15, 2025, can be found here. The full announcement by ARPA-H, which is part of the U.S. Department of Health and Human Services (HHS), can be found here.