Discussion with leaders from Quizlet, Facebook Messenger, IBM Watson, LifeSpark, and Skype

Last Tuesday, Dashbot and Quizlet co-hosted a panel of edtech leaders who discussed how bots are changing the way people learn and the educational space as a whole. The speakers included:

Justina Nguyen — Developer Evangelist, Dashbot (Moderator)
Alex Quintana — Product Manager, Quizlet
Angelique Kamara — Facebook Messenger Partnerships, Facebook Messenger
Anamita Guha — Product Manager, IBM Watson
Rachel Sklar — Co-Founder, LifeSpark
Steven Abrahams — Group Product Manager, Skype

They shared insights and strategies for bot builders creating educational bots, as well as their excitement for the future of bots.

Here are the highlights from the discussion.

1. In what ways are bots changing the way people learn?

AQ — Quizlet: Bots bring a unique superpower because they are conversational. You can ask bots questions as customers and they can listen to what users say. Right now, I don’t know if bots are truly changing the way people learn yet, but there are a lot of different use cases that we see at Quizlet. For instance, you can think about group quizzing and how a bot might facilitate outside-of-school interaction as a trivia master to help people learn and study.

In the far future, you can imagine a bot that is everyone’s nerd that sits in your pocket and understand your strengths and weaknesses because it’s been quizzing you. — Alex, Quintana, Quizlet

AK — Facebook Messenger: I’m really excited about this idea because if you can build it so that it learns the way I learn, the experience becomes much more personalized. I think it’ll be fascinating if you fast forward a few years when A.I. can learn the way you best learn content.

If you look at some of the top education bots like math learning and language learning, you can really gamify the experience and provide daily digests of content.

You can use some of the mechanics that game developers have used like leaderboards and social challenges to bring your social circle into the bot experience.

AG — IBM Watson: I think bots are already changing education. On the most basic level, bots are really good at aiding habit formation. You can feed it vocabulary words if you’re studying for the SAT. You can prime yourself to learn all of these things and form a habit.

Bots are a really good gateway to get young students into tech. — Anamita Guha, IBM Watson

One of the first things we do at IBM Watson in our K-12 cognitive curriculum is have students build a chatbot. It’s quick and it gets them excited because they created something. Bots are also moving us towards personalized learning and one of the best examples from Watson is Jill Watson, a teaching assistant at Georgia Tech. It’s a TA that answers tedious questions like, “When is this due?”, “How much is this assignment worth?” It learns really well based on how students are responding in different forums and it can mimic how the student is speaking.

RS — LifeSpark: As a mom, I’m so excited about what bots can bring to the classroom. Teachers can’t track every student’s progress, strengths, or weaknesses. If a bot eventually could track these things, kids could take charge of their education in a completely different way. Students could have customized learning plans and teachers wouldn’t have to do the administrative work to develop these plans.

Bots are leveraging the way people learn best. — Rachel Sklar, LifeSpark

Research shows that people learn best when they learn in smaller chunks at a time with a certain amount of spacing in between in each lesson. People learn best when they’re interacting with their content rather than passive consumption of content. At LifeSpark, we think about adult learning theory and what does it say about how adults learn best and how can we incorporate that into our bot.

SA — Skype: Something that makes me really excited about education and bots, beyond messaging, is the fact that bots can traverse so much data at once.

Bots have the ability to be best assistant an educator can have. — Steven Abrahams, Skype

At Skype we have a program called Skype in the Classroom where we connect 800,000 classrooms around the world with subject matter experts in specific units. If you were in the midwest and you wanted to run a unit on the Galapagos Islands, you can bring in a marine biologist over Skype and they would do a lesson for you. You extend that to what bots can do and all of a sudden, bots are essentially helping coordinate these opportunities for students to have classroom-to-classroom learning experiences.

2. What types of capabilities and functionality should developers consider including when building ed-tech bots to increase engagement?

AQ — Quizlet: If you’re a new bot builder, you need to consider the main problems the users have and how can you leverage a bot’s superpower to answer those unique problems.

Recreating functionality on web or mobile won’t be a winning strategy for bots because people already have adopted those platforms. — Alex Quintana, Quizlet

One thing I’d emphasize is thinking deeply about the user problem and the superpowers that bots offer around conversation. Then, applying the ability to create a unique and personalized relationship with the user via bot.

AK — Facebook Messenger: I would consider starting from a user’s problem or a business problem. If you’re building an experience that delivers something daily that creates a habit then this is a way to increase engagement. Also, I would think about how to encourage users’ friends to get onto the platform. For example, with Quizlet it’d be really cool if you were able to chat with someone in your friend group who knows the answer. At Facebook, there are a lot of tools available for developers to help increase engagement — such as best practices, design reviews, and platform feature adoption.

One of the best ways to drive engagement is to humanize the bot. — Anamita Guha, IBM Watson

AG — IBM Watson: I’ve seen a lot of success when developers throw in typos or different broken structures so it doesn’t come off as a robotic interaction so that users are more likely to share how they feel. The other thing I would add is to think about what medium your chatbot will manifest itself in, whether it’s an IOT device or a whiteboard interaction. I’ve also seen a lot of cool applications with voice interfaces and VR and I think that will be extremely important when you think of autistic students and students with disabilities because they’re using VR headsets to drive their learning experience.

When you’re fully immersed in this meta-verse, you’re in a different state and the way you type versus the way you talk.

RS — LifeSpark: We’re constantly striking a balance between letting people binge on learning or giving them content in small chunks. We think about whether they people perform better with a funny and relatable character or a dry, professor-type character. We also think about the balance between giving them content and providing them with help. Driving engagement for us is figuring out what’s going to retain people and what’s going to help people learn. Re-engagement is highly important to us as well because people leave during a learning session and getting them to come back needs to be contextualized.

SA — Skype: We saw that with a bot that Alex worked on where he was able to re-engage users within the first 24 hours if they stopped in the middle of an activity.

That was one of the more novel things that a bot developer did that impressed me because I like to think of things in terms of ‘product truth’ — does what the bot says it does payoff for the person using it?

If you’re saying you have a product that helps people learn faster or more thoroughly, the real metrics is how successful they are at that. Their ability to be successful with your product will drive engagement, drive word-of-mouth, easier adoption for the next user. You need to think about how successful you were at completing the features you say the bot can do and measure that against your actual user success with the bot.

AK — Facebook Messenger: And this is what made Quizlet successful — their ability to deliver on what their feature promises (help people practice for tests), which then got kids to tell other kids about it.

AQ — Quizlet: I think if you start with something that you know you can build extremely well then you can proof that out and showcase your value to the user to bring them back.

3. What recommendations do you have for increasing bot discoverability?

AQ — Quizlet: There are tools that platforms provide that enable developers to contextually ping people who are using bots. From Quizlet’s perspective, if you’re part of a class and studying a set, there could be a contextual callout that leads you to a messaging platform with, “Do you want to study this set with a group?”

AK — Facebook Messenger: On Messenger, you have various ways to find new experiences, but what works best for developers and partners is the contextual discovery. If you’re doing events in classrooms, you can show people the Messenger code to the bot and they can connect directly. We think about discovery in a broader context, in and outside of the platform, because it’s incredibly important talking to customers where they are and in the right context will drive the discovery.

The more nurturing we are of all bot users, the more people will use bots. — Rachel Sklar, LifeSpark

RS — LifeSpark: Even though our bot is only relevant to a specific subset of users, we want everyone to have a good experience and feel welcomed when they talk with our bot. If they decide to leave, we want them to leave with a really good experience with bots and not think that bots aren’t relevant to them. Strategic partnerships are a great way to get discovered because there are many markets that you can serve that have similar audiences as you.

SA — Skype: I’d echo Rachel’s point about making a great bot because it will make it easier for the platforms to help users find your bot. I spend a lot of time working with bot developers and I know all of the bots in our store. We have very few of the them in the store because we only want our users to have good bot experience. We have an editorial team that will work very closely with you to make sure that you have feedback, best practices, and ways to improve welcome messages. A lot of people don’t know what bots are so removing barriers for people will get them to use bots more. You want to be there at the point when people are looking for something or have a question or intent. The bot can be at the top of your funnel so that the earlier that you start the conversation with a user, the better chances you have at introducing the value you can bring to them.

AG — IBM Watson: Right now people say, “Oh, there’s an app for that!” I’d love to get to the point where people say, “There’s a bot for that,” so all you have to do is find a use case and start building!

4. What KPIs are important to you ?

AQ — Quizlet: The most important thing to Quizlet is that people learn and master subjects. It’s really important that we track the following: did they study the material we presented, how much progress did they make, and which study modes did they use.

AK — Facebook Messenger: On Messenger, we look at our partners’ metrics and what we see is people building bots for entire consumer funnels — from awareness to customer acquisitions, enabling transactions and customer care. We have metrics for each of these use cases so it depends on the experiences people are building.

RS — LifeSpark: Do you have different metrics for different verticals?

AK — Facebook Messenger: When looking at platform health, we have benchmarks that we look at across the board. We want partners to be successful so we look at each partner’s business outcome and individual verticals.

AG — IBM Watson: We define success based on how many developers we get to build bots on Watson. I also think about the successful conversations that happen so user feedback is important to the building process.

RS — LifeSpark: I look at conversation transcripts and read the free-form text. Also, I look at specific key words and we use customer feedback to iterate the product.

SA — Skype: It’s really important to be transparent with the people you depend on to understand the metrics you care about.

Writing is so important for bots that I can see a model in a world where writers are incentivized based on the number of messages the bot receives. — Steven Abrahams, Skype

I continually harp on the fact that people shouldn’t take shortcuts when it comes to writing. The voice of your bot is the thing that fundamentally differentiates it and if it has a bland or boring voice, people will not engage with it.

We spend a lot of time looking at what people say back to the bot and we think deeply about making sure that the response back to the user elicits continuing engagement.

No bot is capable of answering everything so you have to have clever fallbacks and think about re-engagement. Re-engaging users at the most critical time is important and that time is when they’re about to leave your bot. You need to catch them on the way out.

We have to be different than apps or websites because it’s a conversation that people can leave at anytime so we need to be proactive about how we engage with users.

Want to know when our next MeetUp is? Join our group and follow us on Twitter to get updates.