Our friend Megan, part of the team behind the HelpHer chatbot, pinged us with a link to a very interesting medical paper.
We read through it and were happy to have a study to back our believe: Conversations are a great interface for healthcare.
The medical paper is called “Improving Access to Online Health Information With Conversational Agents: A Randomised Controlled Experiment“. It was written by a group of scientists who studied a group of people who were asked to look for medical trials about specific diseases using a traditional keyword-based search engine and a conversational interface. They were then asked to do the same, making it a bit harder by adding criteria the trial had to match.
The sample for their study was of 89 people of an average of 60 years old (very interesting!) with a very good 50/50 gender balance and 23 people with low health literacy. A fifth of them had previous experience looking for clinical trials.
The main finding was that participants were definitely more satisfied using the conversational interface.
“Results indicated that all participants were more satisfied with the conversational interface […] compared to the conventional Web form-based interface.”
Must be noted that looking at the description of the conversational interface it was an advanced one and probably a pleasure to use, with features like read-out-loud, bookmarking and different levels of detail.
The paper also mentions a previous study in which the success rate was also higher in the conversational interface for those users with low health literacy, and gives a figure that may surprise some: 36% of USA adults are included in that definition.
“[Another study] demonstrated that individuals with low health literacy had lower success rates when using these interfaces to search for general health information on the Web. Usability by people with low health literacy is important because this population comprises 36% of US adults.”
Interestingly enough none of these low health literacy users managed to find the clinical trial with the constrains using the keywords search, but a third did using the conversational interface.
“In our standardised task (task 2), it is notable that none of the low health literacy participants were able to find a correct clinical trial using the conventional search engine interface, whereas 36% (5/14) were able to do so with the conversational agent.”
What I read here is: maybe because you understand what your problem is you are able to find the best solution on google. For a person with not so good of a health literacy, that may be a way more challenging task. As in non of the low health literacy passed.
On the down side the time it takes is a bit longer in the conversational interface (around 30%) but participants were not bother and the time difference was actually subjectively perceived as shorter.
The final bits are encouraging:
Apparently several studies have shown that traditional keyword search does simply not work for kids, the elder, or people who speak a different native language.
Another study worked with conversation interfaces and proved their success making health easier to understand for those not familiar with it in health areas such as physical activity promotion, hospital discharge instruction, explanation of medical documents, and family health history-taking.
“Our findings suggest that conversational agent-based search engine interfaces could be a good alternative to conventional Web form-based interfaces for many kinds of applications, but especially for those intended for low health literacy users or those with limited computer experience or skills.”
We are thrilled to hear this and looking forward to a lot more serious research papers. If you stumble upon any please send them our way at firstname.lastname@example.org.