Blog

The real meaning of chatbot stats

In preparation for out third blog post with #sincerestats out later this week we thought we’d share with you our sarcastic glossary of Chatbot Analytics.

Users

Vanity metric used by chatbot developers when talking with investors or other chatbot developers.

Subscribed users

Number of users who like your chatbot OR who don’t but could not find the Unsubscribe button.

Active users

Users (including paid users) who interact with your chatbot even if it is just to read and ignore one of your broadcasted messages.

Usually measured on Monthly Active Users (MAU) or Daily Active Users (DAU).

Engaged users

Mythical, magical creatures who sit on the end of the rainbow quietly following navigation buttons and chatting without asking “asdkahkahs?” or “are you a human?” to have conversations that fit in the use case for which the chatbot was designed instead. They were usually acquired organically, are lactose and gluten free and their tears can cure all diseases.

Our numbers for the best day this month are 1111 > 435 > 183 > 66.
What are yours?

Event: A journey through conversational interfaces with healthetia

We’re happy to announce that next January 10th our founder Cristina Santamarina will join the experts of the swiss healthcare network healthetia to talk about our journey using conversational interfaces in healthcare.

2017-01-04_1446.png

The talk will discuss the current challenges with health data collection, the advantages of the conversational interface, our experiences with Eva, our women health chatbot prototype, and the potential of this family of technologies in the future.

Learn more about the event on the healthetia website or stay tuned to hear the recordings.

#SincereStats December 2016

Once again we’re here with our #Sincerestats. November was great, we completed a full translation to english, had conversations with our first 500 users and learned a lot about how we could make Eva better.

December stats

When we started this journey our goal for end of the year was to reach the first 500 users.

With our November figures (500 users) we changed the end of the year goal to 1200 users. Wanting to not fall short in our aspirations it looks like we were too confident, and we missed our target for 200 users. Every one of the thousand people we interacted with, however, counted.

We’ve looked at your questions and at the bugs we forced you to face and we keep learning and improving with you every day. We send you a thousand thank you’s.

Some users unsubscribed, too. Journaling is not for everybody and a lot of people that checked Eva out of curiosity dropped as well. 450 users stayed with us, and we have set as a goal not only keeping them but getting back to 1000 active subscriptions in the next few weeks.

b4h 201612 Total users.png

Is the drop of growth speed and the opt-out wave a bots4health thing, or did your chatbots also see lower figures for December?

b4h 201612 daily new and unsubscribed.png

Confession: I have been terrible at promoting Eva in the last half of the month. The reason has been an international move just before the holidays, and a lack of stable internet to participate on discussion boards, tweet and post on facebook. One of my goals for December is to keep those peaks of new users through the month. Maybe it’s time to spend my first euro in marketing?

That’s an area to explore, but absolute user numbers, everybody knows this, are the wrong metric. Lets look at active users.

b4h 201612 DAU.png

We had over 100 daily active users on average, with peaks after broadcasting campaigns. You can see the peaks in active users in blue and the broadcasting activity in red corresponding to the broadcasting of the meals feature announcement, the medical research we published to justify conversational interfaces in health or the holiday greetings.

Very noticeable is the drop in the number of users reached through our broadcasts – this has to do with better message targeting and respecting the new facebook messaging rules. Notice the number of users interacting with the messages (in green) is not affected by the change of denominator. That’s good news for us.

Plans for January

Our goal for January is to focus on increasing the number of interactions. We think we can achieve this by generating more frequent, more relevant interactions, using not only journaling reminders but also providing short tips to make our users’ days better: quick healthy snacks, 10 repetitions of a cramp relieving exercise or a mood raising  youtube video.

In January we will drop the shop. With only 31 clicks on product referral links we don´t think this is a function worth exploring at the moment and we decided to leave the quest for monetisation at the moment to focus on the generation of value through content and experience of use.

We trust providing a better experience will keep you coming back. Increasing the number of users who regularly journal will give us the data we need to define our mood algorithm. But that’s another story.

Enter January.
Make every day count.

 

 

#Doctorsays: Conversational interfaces work better than keyword search for people with low health literacy.

Our friend Megan, part of the team behind the HelpHer chatbot, pinged us with a link to a very interesting medical paper.

We read through it and were happy to have a study to back our believe: Conversations are a great interface for healthcare.

The medical paper is called “Improving Access to Online Health Information With Conversational Agents: A Randomised Controlled Experiment“. It was written by a group of scientists who studied a group of people who were asked to look for medical trials about specific diseases using a traditional keyword-based search engine and a conversational interface. They were then asked to do the same, making it a bit harder by adding criteria the trial had to match.

The sample for their study was of 89 people of an average of 60 years old (very interesting!) with a very good 50/50 gender balance and 23 people with low health literacy. A fifth of them had previous experience looking for clinical trials.

The main finding was that participants were definitely more satisfied using the conversational interface.

“Results indicated that all participants were more satisfied with the conversational interface […] compared to the conventional Web form-based interface.”

Must be noted that looking at the description of the conversational interface it was an advanced one and probably a pleasure to use, with features like read-out-loud, bookmarking and different levels of detail.

The paper also mentions a previous study in which the success rate was also higher in the conversational interface for those users with low health literacy, and gives a figure that may surprise some: 36% of USA adults are included in that definition.

“[Another study] demonstrated that individuals with low health literacy had lower success rates when using these interfaces to search for general health information on the Web. Usability by people with low health literacy is important because this population comprises 36% of US adults.”

Interestingly enough none of these low health literacy users managed to find the clinical trial with the constrains using the keywords search, but a third did using the conversational interface.

“In our standardised task (task 2), it is notable that none of the low health literacy participants were able to find a correct clinical trial using the conventional search engine interface, whereas 36% (5/14) were able to do so with the conversational agent.”

What I read here is: maybe because you understand what your problem is you are able to find the best solution on google. For a person with not so good of a health literacy, that may be a way more challenging task. As in non of the low health literacy passed.

On the down side the time it takes is a bit longer in the conversational interface (around 30%) but participants were not bother and the time difference was actually subjectively perceived as shorter.

The final bits are encouraging:

Apparently several studies have shown that traditional keyword search does simply not work for kids, the elder, or people who speak a different native language.

Another study worked with conversation interfaces and proved their success making health easier to understand for those not familiar with it in health areas such as physical activity promotion, hospital discharge instruction, explanation of medical documents, and family health history-taking.

“Our findings suggest that conversational agent-based search engine interfaces could be a good alternative to conventional Web form-based interfaces for many kinds of applications, but especially for those intended for low health literacy users or those with limited computer experience or skills.”

We are thrilled to hear this and looking forward to a lot more serious research papers. If you stumble upon any please send them our way at cristina@bots4health.com.