Design Process

Using a youth-centered design approach, we conducted interviews in 3 counties and tested low and high fidelity interactive prototypes with youth (n = 99; adolescents aged 15-24, influencers and guardians) in three selected counties in Kenya during the research and design phase from January to April 2021. I created the design research and testing guides for youth and guardians to understand key questions like access and desirability, and I iterated on the prototypes during our three week-long sprints.

The key hypothesis tested during this phase was:

If Kenyan adolescents access sex ed through an interactive platform, it will make them feel more at ease and engaged as they gain sexual reproductive health (SRH) knowledge and thus increase their demand of SRH services.

This was further broken down into multiple assumptions that we tested:

"Advise us, what hospital to go. We do not know and we cannot ask."

- Girl in Mombasa, 16

Key findings

In this phase, we interviewed youth about their general device access, and tested the various delivery methods concepts, tone and voice, and visual styles to evaluate the desirability and feasibility. Here is what we learned about access, gender inequality, and platform preferences:

• Access to data bundle and free Wifi, and slow network are huge barriers. Many youth know that they can use WhatsApp text and audio for free as long as they purchase an Internet data bundle, which is one of the most common ways for youth to access the Internet.

• As a platform, WhatsApp is the most popular and trusted, and while Facebook is popular, youth do not perceive it to be a safe SRH platform. In both counties, youth’s most preferred prototypes are Faya Digital (44%) and Faya WhatsApp (49%).

• Most boys and urban girls generally have their own smartphones and access to the Internet, and are familiar with using them. Girls in peri-urban and rural areas (Homa Bay) typically only has access to a shared phone, and may not have access to WhatsApp or Facebook. Boys in urban and rural areas seem to have similar needs and challenges.

Co-designing with youth helped us develop our key personas, design principles, and informed the final AskDoki chatbot content strategy and design. 

Design Principles

We derived the following principles based on user feedback and the final solution is developed based on them:  

1. Privacy is key

2. Elicit trust through authority figures 

3. Don’t judge and don’t ask unnecessary questions

4. Don’t beat around the bush and don't leave them hanging

5. Option to be frugal with data usage

From the beginning, we realized the importance of creating a private, trustworthy, judgement-free, and safe space for youth so that they can explore questions that they are curious about.

• Don’t beating around the bush means we should communicate in direct and condensed info since youth may already in crisis mode. Youth really appreciated the FAQ formats and the interactive chat styles from our rough prototypes, and so combined the best of those two interactions. 

• Don’t leave me hanging means providing quick responses and a comprehensive answer instead of snippets of info. In particular, when youth want to self-diagnose, they want to get information on 1) description of symptoms, 2) possible causes, 3) actionable ways & places to get treated, and 4) preventative measures.

• No matter how great a service is, Internet and data access is a key barrier for many youth in Kenya. Most boys and urban girls generally have their own smartphones and access to the Internet, and are familiar with using them. However, girls in peri-urban and rural areas (Homa Bay) typically only has access to a shared phone, and may not have access to WhatsApp or Facebook.

Service Design

I also developed an initial user journey and theory of change based on the knowledge we gained from interview and testing with youth. This helped us align the project scope with our partner, and initiated the conversation to integrate with existing healthcare services in Kenya.

Prototyping & Testing

Leveraging the rich data that we collected, I supported my team to work on content creation, translation, and I led the chatbot conversation design, interactive prototyping, and branding and rich media creative directions. To begin designing a chatbot, I first conducted a competitor analysis to learn about the existing solutions, in particularly the healthcare space. WhatsApp is a rather well-known platform, but WhatsApp chatbot is a new technological solution that few NGO’s and healthcare organizations have leveraged.

Then, I created a rough prototype in Botsociety to test the initial content style and interactions e.g. videos, infographics, and quizzes. Shortly after, we created an interactive WhatsApp prototype that youth can test on their phone. The testing process involved doing a three way WhatsApp call, with two team members and one participant. On the call, the participant would interact with AskDoki while we followed through their progress via Rasa, the chatbot backend gateway.

Challenges

Our initial scope was to adapt an existing sex ed curriculum to WhatsApp for youth in Kenya. We created with a content creative brief, a chatbot conversation design guide, and started with content adaptation. However, as soon as we started, we realized the existing content from the client was incomplete, didn't include key indicators of success, and there were red flags regarding messaging. I made the difficult decision to hit pause before moving forward with content that our research found would not be meaningful or measurable.

Working with the content strategist, I created an outcome mapping tool to analyze the gaps between the existing content and the key outcomes the program aimed to achieve. Our team then revised the creative brief and created a content index to align on our objectives with the client. In the end, we made a lot of content revisions with clearer key messages, best practices in CSE curriculum, and voice and tone guidelines for a sustainable project.

Finally, we developed all of our content — content strategy, videos, photos, illustrations, and infographics — based on the following program level key outcome indicators:

• Increased use of/intention to use modern contraception

• Increased health-seeking behaviors for /intention to use Adolescent Sexual and Reproductive Health and Rights (ASRHR) services

• Increased testing and treatment for STIs

• Increased egalitarian behaviors in intimate relationships

Content Creation

From both rough prototyping and user testing, we know that interactive media like photos and videos are highly desirable by youth. I wrote a creative brief to guide the directions for videos, photos, and illustrations. We then produced a series of eight short videos and a series of photos that map to the four key intermediate outcomes to supplement the text curriculum. Due to a shortage in content support, I stepped in and helped write several video scripts, including the one below on relationship and consent that is a topic that is near and dear to my heart.

Technical Decisions

Our lead developer and I made the decision to build AskDoki as a natural language processing (NLP) chatbot on the Rasa platform despite the additional design, development, and training effort because a NLP chatbot can learn from real-life experience and adapt. We want AskDoki to be shaped by the questions and responses by our young people beyond our co-design process, so that the chatbot becomes truly relevant and helpful to them.

Our experienced developer built the classification system, and I coordinated with our Kenyan team to create and test sample data in order for the chatbot to understand intents (defined as the goals the chatbot users have in mind when typing in a question or a comment) so that AskDoki was able to generate human-friendly, medically-accurate responses.

Currently, our most pressing challenge is linkage to child protection services. Despite having incorporated sample training data so that the chatbot is designed to provide appropriate resources for users in distress, AskDoki is far from perfect. Our AskDoki reviewers monitored the anonymous conversation logs frequently, and they flagged the cases when AskDoki is unable to correctly respond to users' questions and needs. If the content already exists, but AskDoki misinterpreted the users’ intent, our development team corrected it. As we tracked new requests or unanswered questions from users in the current version of AskDoki, these become a great source of user-driven content! With the help of a technical writer, we created additional content for our final pilot.

Pilot Launch

To prepare for the pilot launch, I created an evaluation strategy based on the overall program level key outcome indicators in order to align on what success looks like, and how to measure them. Once I developed the key questions to evaluate in the pilot, I revised the plan with our implementation partner and our technical advisor, then I set up the qualitative and quantitative data collection methods to measure the metrics.

During the first pilot phase in May - July 2021, we collected qualitative and quantitative data using a mix of chatbot analytics, conversation log data, and remote qualitative interviews with youth (n = 142; adolescents aged 15-24, and influencers) in four counties with poor sexual and reproductive health outcomes. The goal was to gather data on the chatbot experience and usage in a natural setting in order to improve the intervention design and implementation.

Some of the key learning during this phase include:

• Almost all interviewees expressed positive impression when interacting with the chatbot. Key reasons include privacy, straightforward and interesting content, and quick responses. In addition, 78% of the chatbot users responded positively when asked if the chat was helpful. Youth responded well to the chatbot prompts designed to engage them - both simple check-in’s e.g. “does this make sense” & mini reflection prompts e.g. “tell me an example”. The most popular topic is Healthy relationship among interviewees and general chatbot users, followed by HIV/AIDS.

• The complimentary videos and images were well received by youth. However, they wanted more visuals. They also wanted additional referral services like rehabilitation centers, mental health and GBV resources.

• Interviewees reported that they used AskDoki anywhere from 15 minutes to 6 hours over the span of several days. However, youth are busy, constantly multi-tasking, and expect Doki to answer their questions; if not, they get frustrated and leave. Biggest barriers of interacting with AskDoki are a lack of data bundle, a lack of time, and AskDoki’s comprehension ability.

Some of the key design decisions we made for the final iteration include:

• Create beautiful illustrations that would enhance youth’s visual learning experience

• Continue to train the AI chatbot in order to improve its comprehension ability

• Gather and expand our referral services resources offering

Final product

So, this is the current version of AskDoki! Young people can use AskDoki the WhatsApp AI chatbot to:

1. Learn life skills including sex ed. They can either go through the preset topics that are popular among youth, or ask their own questions.

2. Find nearby healthcare facilities, and access mental health and gender based violence (GBV) hotlines

Note: At the time of this project, WhatsApp chatbot doesn’t support any quick replies button like Facebook Messenger. We recognize that this might create additional barrier for youth, but our initial field data shows that people are able to navigate AskDoki easily.

“[AskDoki] is helpful, I am thankful, She has helped me with my [self-esteem] because she helped me be proud of the little things I have been and by teaching me how to improve my esteem.”

Girl in Homabay, 19

Next Steps

AskDoki is currently live in Kenya and more usage data is being collected. Our local partner, Amref, is currently conducting a Randomized Control Trial (RCT) to assess the impact and cost-effectiveness of this digital platform and the other three delivery channels in 2021 - 2022. The chatbot currently has over 4000 users, and AskDoki will be advertised and scaled nationally once the RCT is completed. Stay tuned!

P.S. Want to read more about my reflections on designing chatbots for adolescent sexual and reproductive health? Check out this article that I wrote!

Curious about my other projects?

Back to Projects