Objective and transparent evaluation of AI models in healthcare has never been more important. getty

The application of AI technology in healthcare is likely one of the most important and substantial contributions of human kind in the 21 st century. The work in this arena has been monumental, with large language models that are now competitive with (and often can out-compete) human physicians in reasoning, aptitude and breadth of knowledge. For example, Med-Gemini was found to be 91% accurate in early benchmark tests for the United States Medical Licensing Exam (USMLE). Early versions of ChatGPT were found to achieve the passing threshold for the USMLE as well.

Nevertheless, the technology has since evolved far beyond just passing simple written exam questions; now, healthcare an

See Full Page