Cameras and chatbots: imagine a contextual UX
“The true content of a photograph is invisible, for it derives from a play, not with form, but with time… a photograph bears witness to a human choice being exercised.” — John Berger
Camera-ready conversational commerce
Snapchat, Instagram, Virtual Reality, Augmented Reality, cloud image recognition services, and more. Designers and developers of chatbots are surrounded by the atomic unit of images, their combined storytelling power, and the innovative possibilities imaging technology presents for conversational interface solutions at work and play.
A smartphone sans camera today is the exception to the norm, and the means to take pictures with your device is more powerful than ever in terms of image quality and options. Even the Nokia 3310 ‘relaunch’ added an onboard camera, confirming that taking pictures on the go is a must-have part of the mobile user experience for even the most retro-conscious hipster.
And, although Amazon’s Echo Look might take your sense of personal style to a new level of conversation and Sephora’s Virtual Artist can help you get that right shade of red, let’s be clear: Using cameras to take digital pictures enables chatbot designers and developers to work with much more than the look of the photographic subject.
“Image recognition has made a leap of progress in the past few years due to breakthroughs in machine learning. Bots can now, using an image processing service, recognize images, spot emotion in photos, and extract text from images” — Amir Shevat, Designing Bots: Creating Conversational Experiences
Because the chatbot user experience avoids the need for users to procure special devices and apps, relying instead on what’s already available and familiar, means chatbots are well-positioned to leverage cameras for those asks and tasks of conversational interactions.
Indeed, the disruptive impact of chatbots has given the humble QR code a whole new lease of life too. Popularized by China’s WeChat for example, using a QR code and the mobile camera is now the de rigueur messenger and bot onboarding interaction.
The power of the camera in conversational commerce goes beyond the visual quality of the image, going straight to the essence of conversational computing: a deeply contextual “in the moment” experience for the user on their journey to accomplishing a task or ask with a chatbot.
“In China, WeChat’s mobile application has had an embedded QR-code scanner since 2012, so the millions of WeChat users don’t need to install any special app to have access to this functionality — they can simply take advantage of the preinstalled WeChat.” — NN Group
Context is king
User experience pros will already be familiar with the capture power of photography from their world of contextual inquiry and research-in-the-wild approach of ethnography. That word is key: context.
Digital images captured on a smartphone carry built-in context around with them: the date, time, location of the image, its relationship to other images, what’s going on around it, and so on.
Whether at the heart of an initial chatbot intent or enhancing the flow of a conversation, the underlying agency of the image is based on captured context, and nowhere is context more powerful and manageable than in the enterprise.
The HomeHero Messenger chatbot, for example, relies on a mobile device camera to capture a user’s energy bill details for a conversation about switching energy providers. HomeHero leverages the camera further for engagement by inviting the user to take a picture of an appliance and get energy-saving information about it in a gamified conversational commerce interaction.
Now, figuring out what the image actually is, or could be, is a function of the image recognition capability and AI of the chatbot service and conversational discovery, but the use case possibilities presented by our fascination with using mobile phones to take pictures seem endless.
Take, for example, the Google idea of figuring out the calorie counts of all that Instagrammed ‘food porn’. Surely, that is a use case for the fitness and wellness industry just waiting to be integrated into a chatbot solution for those who want to talk about healthy options and lifestyles!
For enterprise chatbots, built-in image context offers even more powerful options when it is combined with context such as user role, goals, past behaviour, schedules, check-ins, network IDs, what other team members and customers are doing, and so on.
Using the mobile camera in chatbots goes to the essence of conversational interfaces: it makes for a deeply contextual “in the moment” experience for the user’s ask or task for a chatbot.
As a simple use case example, a mobile photograph of a business card might be added to the SaaS cloud and imaging technology then automatically adds the details of the card for use in a chatbot conversation.
This can trigger a deeper discovery process around how the user came to get the card, why they captured it as an image, a process that leverages existing enterprise context. For example, that card details might represent a new business opportunity or an existing one, and the chatbot platform intelligence and learning can prompt the user for more information or respond with how the card details relate to other enterprise objects in the cloud (people, places, things, and relationships) and even advise the user what to do next.
Picture this
There is a huge opportunity for using cameras and imaging technology in enterprise chatbots to not only solve problems but to do so in a natural, conversational way and to disrupt business processes.
More broadly, this notion of contextual capture also includes other mobile device features such as microphones and sensors to augment the human-computer dialog of text and voice interactions giving chatbot designers and developers a modality of solutions to explore.
You can read more about the role of the camera and chatbots in the Chatbots Magazine article by HomeHero founder Colm Moriarty: “Show Me, Don’t Tell Me”.
Also, check out the RTÉ Radio 1 “The Business” podcast about conversational commerce.
As ever, your views on cameras and chatbots are welcome. Find the Responses section!
Ultan Ó Broin (@ultan) is an independent user experience professional and also blogs about UX and technology.
All screen images and photographs are by the author
Multilingual chatbot designer? This story can also be read in Spanish: Cámaras y chatbots: Construyendo una experiencia de usuario contextual
