Researchers from Flinders’s College of Medicine and Public Health found that a vision-enabled AI scribe, employing a combination of Google’s Gemini model and Ray-Ban Meta smart glasses, substantially improved the documentation accuracy of pharmacist-patient consultations and reduced omissions and errors in clinical notes.
“AI scribes are already helping clinicians by listening to consultations, but health care involves far more than spoken words,” says research author Bradley Menz, an academic pharmacist in Flinders’s College of Medicine and Public Health.
“A lot of clinically important information is visual. Important visual cues during consultations include patients’ medicine containers, prescriptions and devices, as well as their body language. When an AI system can use both what it hears and what it sees in these consultations, it captures more of the details that matter for patient care.”
