IBM Watson Assistant:
Using Custom Extensions
The ask: Improve the experience of a non-technical content owner adding custom extensions to an action (tasks that an assistant is able to accomplish). Specifically, gather information from people interacting with the assistant (chatbot) and using that information to interact with an external application in a variety of ways (open support ticket, get account information, see what time the sun rises, etc).
The team: Design lead (me), Junior Designer, Product Manager, Engineering Lead, 2 Engineers
The timeline: 2 months
The process: Stakeholder Management, Design Thinking Facilitation, Evaluative Research (Task Analysis), Competitive Research, Iterative Prototyping.
The outcome: Inspire the direction of Watson Assistant, impact not measured before I left IBM.
I led the research effort to evaluate the current experience for our persona Tanya (non-technical content owner) to add custom extensions to their assistant. I planned the research, screened, and recruited users that matched our bob-technical content owner persona (Tanya). I conducted the task analysis sessions with 10 participants.
In parallel with the task analysis sessions, our team evaluated other applications for building chatbots that had ways to add extensions to their assistant.
Using this research, I led the effort to build a conceptual prototype for our team to align on the direction for Watson Assistant.
Evaluative Testing Plan
I advocated for evaluative research to test this assumption and to give us more points of data on how to best improve the experience. Below is the task analysis test I created and conducted with 10 participants. I worked closely with my team to determine what we wanted to learn and tailored the test accordingly.
What did we learn about our current experience?
It is difficult for Tanya to understand the structure of her assistant’s conversational capabilities.
It is difficult for Tanya to find the right operation for her custom extensions and set it up correctly.
It is difficult for Tanya to use variables effectively when building her actions.
Competitive Experience Research
Our team looked at the experience that other chatbot builders had for adding extensions to their chatbots. This was a valuable exercise that supported our assumptions that we needed to make the structure of the conversation easier to understand and separate types of content visually.
Our solution
We came up with 3 principles / goals to achieve as a team for the new experience we wanted to build. We went through a few iterations of prototyping (sketches, lo-fi, mid-fi to align as a team on what we think we should build. The goal for this prototype was to test again with users and share the results with leadership/stakeholders to get buy-in to build it.
