New research from Klick Applied Sciences found that people using interactive voice technology for information on medications still get the most accurate results from Google Assistant. Klick conducted the research as a follow-up to its June 2019 first-of-its-kind study assessing Google Assistant, Apple’s Siri, and Alexa’s comprehension of the 50 most dispensed brand and generic medication names in the U.S.
According to the team behind the just-released, “Medication name comprehension of Intelligent Virtual Assistants: A comparison of Amazon Alexa, Google Assistant, and Apple Siri between 2019 and 2021” study in Frontiers in Digital Health, while Google continued to provide the most accurate search results for drug names, Alexa and Siri dramatically improved their comprehension accuracy. The findings signal an improvement in their AI software algorithms to better recognize the complex speech characteristics of most medication names.
“A voice assistant’s ability to accurately recognize proper medication names can have life-altering consequences for patients who use technology to help manage their health,” explained Yan Fossat, vice president of Applied Sciences, Klick Labs and principal investigator of the ongoing research. “We continue to be encouraged by the progress of these findings and expect AI to grow more ‘intelligent’ as it gathers more data from users to update software algorithms. Currently, however, our research suggests that patients and consumers should not solely rely on voice platforms for accessing important medical information and advice.”
Key results of the study
- Siri delivered the most improved performance with 78.4% brand-name accuracy and 75% generic accuracy (up from 58.5% and 51.2% respectively).
- Alexa again ranked third with 64.2% accuracy on brand names (up from 54.6% in 2019), and 66.7% for generics (up from 45.5%).
- Google Assistant’s accuracy rates largely remained the same, with 86% for brand names and 84.3% for generics, indicating a possible “ceiling effect” in performance to researchers as the software has already achieved peak accuracy, and the small percentage of issues it encountered were likely due to human error.
Interestingly, there were no significant differences in accuracy based on people’s accents, unlike the 2019 study, which revealed Siri and Alexa were 8-to-11% less accurate for people with foreign accents.
“Not only does this show that the AI systems have made tremendous progress in better detecting and understanding accents, but it tells us that the latest voice assistant technology is more usable for consumers from a wider range of demographic groups, which is extremely important and good news given the diverse population,” said Adam Palanica, PhD, a Klick Scientist and co-author of both studies.
