A Mixed Modality Approach To Chatbots

• Read original article here

“The Medium Is The Message” ~Marshall McLuhan

A modality is a particular way of doing or experiencing something. That somethingcan be a chatbot, voicebot or conversational interface.

The modality can be voice or text…the modality can also be graphic, visual and structured.

However, this is highly dependent on device you are using. Think of an Echo Dot opposed to an Echo Show, for instance. So there is this tendency of an ideal environment for a user: highly unstructured input, very conversational like. Hence the user input is pure text or voice.

The output again is very structured, graphic and leverage all the possible affordances of the modality.

This is the fastest and most convenient input for the user, free speech which is compounded. But the output is structured, not ephemeral but lasting; and easy to digest visually and audibly.

This approach is heavily reliant on the device you are using. And also reliant on the medium you are making use of. Examples of mediums are WhatsApp, Messenger, Web Chat etc.

A medium like Messenger has more option to present and structure your output compared to WhatsApp for instance.

This brings us to the idea of Conversational Components.

Firstly we need to think of any conversation, be it in voice or text, of being presented to the user by means of conversational components…

Conversational Components are the different elements (or affordances if you like) available within a specific medium.

For instance, within Facebook Messenger there are buttons, carousels, menus, quick replies and the like.

These allow the developer to leverage these conversational components to the maximum hence negating any notion of a pure natural language conversational interface.

It is merely a graphic interface living within a conversational medium. Or, if you like, a graphic user interface made available within a conversational container.

It is taking the interface to the user’s medium of choice. Hence making it more accessible and removing the friction apps present.

The problem with such an approach is that you are heavily dependent on the affordances of the medium.

Should you want to roll your chatbot out on WhatsApp, the same conversational components will obviously not be available, and you will have to lean more on NLU, keyword spotting or a menu driven solution. With a even more basic medium like SMS are are really dependent on NLU or a number/key word driven menu.

Where the menu is constituted by keywords or numbers, the menu will have to be a key word or number the user need to input to navigate the UI.

Technically, yes; as it lives in a multi-threaded asynchronous conversational medium.

But is it conversational in nature? One would have to argue no.

So in the second example a rudimentary approach needs to be followed for one single reason, the medium does not afford the rich functionality and is primarily text based.

With Voice, these components disappear completely. With only the natural language component available within the Voice User Interface (VUI) environment there are no components to leverage with the affordances are invisible.

One trend showing people’s propensity for a mixed modality approach is voice search. The trend where we speak into our browser and have our usual search results return.

Somewhere we migrated from searching with key words and cryptic phrases to a point where we search in natural language. Structuring our query to Google in a natural language sentence.

There are various statistics reporting on this, with most stating that 80% of searches are in natural language. It was widely predicted that 50% of all searches in 2020 will be via voice.

There are messages which simply consist of plain text and there are also richer message content such as cards.

Through this the chatbot’s design can be improved and the data presented can be easier digested.