A key concept in business today is “digital transformation,” which centers on how companies can tap new technologies to re-think how they do business. Often, there’s a real excitement that results from seeing how, with some strategic shifts, familiar day-to-day processes can become more effective. But reshaping internal processes is only one part of the equation: the other vital element is how well it serves your customers.

It’s said that to really get a good idea of the customer experience, you need to think of it as a journey, from the time the customer first approaches the company. In other words, those methodologies you use with digital transformation to develop and streamline your workflow? You can use them to create and improve your customer experience as well. Listen now to hear our interview with Adaobi Kanu, Director of Customer Experience at Ollie, a pet wellness brand.

 


Podcast Transcript

[0:00:00]

Heather Taylor

Hi. I’m Heather Taylor from Simplicity 2.0’s podcast. Today we’re going to talk about chatbots, but so much the simple ones many of us know today which ask you a yes or no question—May I help you today—and follow a predetermined script. Today we’re going to look at bots that learn language and as of the summer even invented itself, apparently.

This June, Facebook announced that it shut down an experiment that succeeded in showing bots could develop their own linguistic system for officially negotiating transactions. The problem? Well, the researchers never incentivized the bots to ensure that the language was one that humans could understand too.

Today this kind of AI is still a bit of an outlier, but data suggests broader adoption may lie on the horizon. McKinsey forecasts that 29 percent of service sector jobs could eventually be done by bots. If there’s a lesson we can draw from Facebook’s experiment, it’s that if businesses want to deploy this technology, they’ll need to think carefully about the marching orders they give to bots. And to do that they need to see language through the eyes of a bot.

[0:01:00]

So let’s dive in. Simplicity 2.0 is brought to you by Laserfiche, the elite provider of global enterprise content management software, which manages and controls information so you can empower employees to work smarter, faster and better.

We’re joined today by Zach Lipton who is finishing up his PhD in the artificial intelligence group at the University of California in San Diego, and will be joining the faculty of Carnegie Melon University this January.

I want to start with the Facebook AI story, which the news has been kind of spinning as a Terminator, rise of the Skynet-type narrative when the reality is a little drier. What does it actually tell us about how advanced AI both interprets directions and learns on its own?

Zach Lipton

You know I don’t really think that the Facebook story tells us a whole lot about AI. It doesn’t really reflect any kind of substantial departure from what we already know about machine learning. I think what it really tells us is something about the interaction between –

[0:02:00]

— the research community and the news media. The Facebook chatbot story kind of originated as just a paper that came out of Facebook AI research labs. They’re looking into chat bots. They’re doing interesting research, but I don’t think it’s any more groundbreaking necessarily than any of the probably 20 other, you know, interesting papers on dialogue with machine learning that happened this year. And the story, the preprint was posted.

The sort of usual suspects that might be interested in this sort of paid attention like MIT Technology Review. I’m not a – I don’t remember exactly if they in particular wrote about it, but you know, stories like that – I remember I interviewed even someone was curious about it and interviewed me asking questions about the paper. That was actually why I read it for the first time.

And then, you know, it was basically they had some agents and they were trained and they could communicate and they’re given some goal.
[0:03:00]

And communication literally just means like choose among a set of tokens, you know, tokens in this case being words to—this will be what the message that’s passed to the next agent and the next agent has to, you know, choose, you know, one word at a time for just to send a small message. And ultimately these agents are supposed to be deciding upon some transaction in the setting that they are passing it.

Heather Taylor

Uh-huh.

Zach Lipton

You know it sort of makes this assumption, this sort of ridiculous idea like something remarkable is happening, but really that’s what happens in every research every day in every lab is somebody runs an experiment and then it’s done.
You know like a sort of cute toy problem where they’re trying to see if different agents have different values can they find a way to send information to each other so they can make a trade. And the story kind of people talked about the paper because it was posted by a prominent AI research lab and then the story kind of died. And then a few months later someone came up with this story of sort of someone has to – Facebook had to shut down an AI. It was just sort of a preposterous statement. It’s almost like every time you turn off your phone you’re shutting off ten or 20 AIs.

[0:04:00]

You know it sort of makes this assumption, this sort of ridiculous idea like something remarkable is happening, but really that’s what happens in every research every day in every lab is somebody runs an experiment and then it’s done.

Heather Taylor

So you know, as we’re – so if we look at this situation we’re saying this is kind of something that is pretty of the norm, but how do you think that this, the ability of these chat bots, the ability of this machine learning, how do you think it will evolve in the short to medium terms? So we’re talking about that you were saying this is quite simple, a kind of cute problem, but kind of what’s next in the next few years?

Zach Lipton

Well, I’ll tell you what the great hope is. Typically chatbots are broken down into a number of components. So the first component is usually have what we call – well if you assume you’re interacting with someone in text versus speech, you know, if it’s speech obviously you need some kind of speech to text, you know, automatic speech recognition component.

[0:05:00]

But even assuming that those are there, so you’re just getting free text and you’re going to communicate back in text you get some input from the user and the first thing you do is what’s called natural language understanding. So this is where you take some kind of raw utterance and you need to convert it into some kind of structured form.

The most dialogue systems in the world, things like Siri or Google Now or they kind of, they tend to work with just what we call slot filling. Like you might say hey, Siri, I’m looking for a movie in San Diego tonight. What time is Transformers playing or something?

Heather Taylor

Yeah

Zach Lipton

Assuming you hava a bad taste in movies.

[Laughter]

Zach Lipton

And then what it’s going to do is turn this into something that’s like there’s an act and then a number of slot value pairs.

[0:06:00]

And the act in this case might be a request and the requested field might be time or location or something and then you might say movie name equals Transformers, city equals San Diego. So it turns it into this kind of thing of you recognize.

You know this is obviously very kind of like rigid system and it only works in your very kind of specific domain like movie booking or travel booking or something like this where you kind of know what – you have to at least know what are the fields and you have some kind of sense of what are the possible values they might take, but it turns out you could do pretty useful things within this.

Like you could say send a message to someone and you know, you recognize what’s the request, you know.

Heather Taylor

Yeah

Zach Lipton

It’s like we’ll send messages in your vocabulary. The recipient is whoever. So when you work in these narrow domains you actually can get pretty far just developing a language understanding unit. And to give you like a bird’s eye view is the language understanding then once you’ve done that language understanding –

[0:07:00]

— you need what’s called a policy. And the policy is basically, you know, what is the behavior of the dialogue system? Like what is it going to do next? Is it going to trigger some kind of app? Is it going to respond in some way? Is it going to retrieve some information for you? So you need some way of mapping from inputs, you know, input basically being like what your observations which include what the human has told you onto like what should your next action be.

We call this like in control theory in reinforcement learning we call this a policy is formally a mapping from states to actions. So the way – and then once you choose an action you have to maybe generate a reply. And we call that step natural language generation. And that’s the one you can kind of cheat on very often pretty effectively just by using some sort of templates.

So that might have sounded like a mouthful, but basically, there’s two like main parts that you can’t cheat on.

[0:08:00]

One is doing some amount of natural language understanding and the next thing is doing some amount of deriving some kind of policy that says what should the behavior of this system be. Nearly all of the dialogue systems that I know of that are actually in production in the world being used –

Heather Taylor

Uh-huh.

Zach Lipton

—have used machine learning to great effect for natural language understanding. So going from speech to text is a task that has been revolutionized by deep learning which is really good at this kind of thing of, you know, doing sort of like recognition and sort of like raw signal data like you’ve seen the vision systems that Facebook uses to recognize Facebook faces in photos. This is the kind of thing machine learning does really well —

Heather Taylor

Right. Right.

Zach Lipton

— because it’s sort of this a very sort of well a crisp task, right. Here’s the raw audio. Here’s the text. Here’s raw audio. Here’s the text. You don’t have to do any kind of sophisticated planning or reasoning about what might happen in the future.

[0:09:00]

The trickier task is the policy part. So you know we’ve used machine learning really effectively for speech to text and for text to natural language understanding meaning slot filling usually. But the policy part is an area where machine learning hasn’t really broken into like the real commercial applications.

And people like the group that I worked with on Microsoft Research Labs for the last year or these folks at Facebook, they’re trying to address the policy part of the puzzle. Can you make systems that are capable of deriving their own policy and deciding what to say?

Heather Taylor: All right. So if we’re living in this narrow domain, you know, you have this narrow domain that you’re developing the inputs, the action, creating the policy, these are humans doing this. So what are researchers discovering about how biases from human language or just human thought patterns seep into those artificial intelligence?

Zach Lipton

I think generally it’s a really important question that sometimes practitioners –

[0:10:00]

— don’t stop and think about. You usually have a problem statement, right, like you have something you want to predict and you have some data set like you want to predict will people default on loans. And so you have a bunch of data. For each loan applicant you have a number of what we call features, some attributes. If you think of a spreadsheet these would be the columns in the spreadsheet and like there would be a row corresponding to each applicant.

And then you have something you want to predict like are they going to default. And there’s a lot of places where if you basically train a system to make this kind of prediction and then you take it and you take these predictions and you strap on some kind of decision rule to make decisions based on this like if they’re more than a certain amount likely to make a loan then denied – to default on their loan then deny it, right.

If you take this kind of predictive model and attach some decision theory there’s a lot of ways that sort of human biases or like –

[0:11:00]

— prejudice attitudes can then seep into the model.

Heather Taylor

Uh-huh.

Zach Lipton

One of them is if, one of the classical is just if the thing you’re trying to predict is sort of annotation that comes from a human and the human has a bias then your model is basically trying to imitate the biases of the humans, right. There’s a number of other crazy things that can happen like people saying what’s called unsupervised models.

So this is not where you have a well-formed prediction task but it’s where you try to say I want to come up with basically some vector, which means like some list of numbers that represent some like real-world like symbolic object. And people have done this recently for words and language. They say I want to come up with a vector that represents each word.

And then once you come up with these vectors you can say like which words are closest to each other. And they find that when they train this on corpora of natural language taken from humans — I mean I guess that’s our only source of natural language, right – but when we train this on things like saying all the books or all of Wikipedia, we come up with weird patterns like –

[0:12:00]

— for example, like traditionally black names tend to be closer to words associated with crime.

Heather Taylor

Uh-huh.

Zach Lipton

Or things like you can compute analogies in those spaces. You can say like roughly what is the distance in this vector space between man and woman and it tends to be like correspond to the direction between say like professor and assistant professor, something like that, which is sort of reflecting a bias that we know we have in society but we hope we would not pass on to models that we build especially when we anticipate using these models to make impactful decisions.

Heather Taylor

So – okay. With all of this, you know, the idea that there’s this, there’s bias, there could be bias, there’s this idea of like how is it advancing but it can be very useful on the other side of things. Let’s say you’re an executive, which I think this is going to be happening a lot more and also it’s going to be in bigger considerations that – and you have your eye on chat bots – and so what steps can you take to ensure that the bots speak with your brand voice –

[0:13:01]

— and serve its objectives and maybe eliminate some of this bias that could potentially happen that could be a potential downfall of utilizing it?

Zach Lipton

I guess I’d say, you know, machine learning systems you have learning. There are a number of different paradigms for, you know, how you can train an algorithm to do something, but they all involve you telling it explicitly I’m being able to sort of explicitly give it an objective at least all the systems that we actually use. And so some things are really to say what’s the objective, right, like –

Heather Taylor

Uh-huh.

Zach Lipton

— we have some raw audio text and we have the transcription for it and you could get 20 people in the room and they’re all going to agree about it. And we can collect large amounts of this data we have. But we can only get the algorithm to do things that we know how to communicate to it whether or not it’s done it correctly. It’s very hard for some of these –

[0:14:00]

— things like does the chatbot speak with your brand voice. Like these aren’t the most crisply formed machine-learning problems. And my guess is that, you know, at least in the one to five year time horizon of where we are in chat bot research that people are doing these really more advanced things that are categorically different than say what, you know, Siri has done. They’re not going to be, you know, they’re trying to address more basic questions not like is it speaking with our brand voice. You know, they’re trying to get can we get it to perform some functional job at all.

Heather Taylor

Right.

Zach Lipton

And so my guess is that for that class of user they’re not going to be looking at the bleeding edge, the machine learning and trying to use what, you know, I was describing before, reinforcement learning to sort of learn the policy part of the picture. They’re probably going to try to get their hands on really good speech recognition, really good –

[0:15:00]

— natural language understanding. And then they’re probably going to have domain experts who are very meticulously handcrafting the policies that they use, that their chatbots execute. And I think in the very short run if you really, really want that kind of control then that’s probably you’re limited to that sphere.

Heather Taylor

Perfect. And so I want to ask you one last question just kind of going on the same kind of tangent of a company’s and executives bringing this into their companies. So as they seriously start to integrate bots into their business strategy, you know, how do you think it changes that day-to-day work for both I’d say like the CEO at the top and then the employees that will have to, you know, either work with these bots or utilize them in their day-to-day work?

I’m wondering if CEOs need to think differently when setting direction for both humans and machines and whether employees will need to learn new skills. What do you think about that in terms of the future of work and bots?

Zach Lipton

I don’t know if you remember like ten years ago –

[0:16:00]

— when like Twitter was kind of a new thing.

Heather Taylor

Uh-huh. I do in fact remember that.

Zach Lipton

Yeah and we were younger people. So do you remember this like weird kind of feeling that like companies were aggressively hiring like directors of social media strategy? Like that was my – I was just getting out of college I guess ten years ago I guess so it was my first taste of the work force. And I remember seeing this that people were aggressively people were carving out careers for the first time in like social media strategy and these kinds of things.

But there was this weird feeling of like there was a mismatch like the frothiness of excitement about social media didn’t necessarily correspond to people actually having any coherent ideas about what they were going to do with it.

Heather Taylor

Yes. I really remember that. There were people like wait; how do you have that job? What? You just tweet a lot?

Zach Lipton

Yeah. And I remember, you know, there were crazy things that would happen like you could – companies were so – they had a sense that it was important but they didn’t know precisely how.

[0:17:00]

Heather Taylor

Uh-huh.

Zach Lipton

So I remember I used to do things. I used to tweet at companies and some companies were just because they were aggressive on social media they were insanely responsive, you know. Like I once purchased like a tea at Whole Foods and then I just left it at the counter and forgot about it.

And I went out in town in New York. And so I tweeted Whole Foods just like on the lark and said hey, I left my tea at the counter at the Whole Foods in Union Square. How are you going to make me whole? And a human actually like responded to me and like took personal care to make sure that I got my tea.

Heather Taylor

Wow.

Zach Lipton

And I think maybe some of that turned out to be a good strategy for customer service but I think to some extent like they were people were just flailing for a while until they figured out like what was actually a reasonable way that this could or couldn’t be useful in their business. And so I don’t mean to like totally, you know, just –

[0:18:00]

— throw cold water all over like the entire enterprise. Like I’m actually quite excited about what we could do with dialogue agents. But at the same time seeing sort of where the research is and I mean I even would go as far as saying that I think it’s one of the next like really interesting research frontiers where we can make an impact, but seeing just where the research is and what companies are talking about in their marketing materials, I think that there’s this like pervasive misunderstandings about what’s capable right now.

And I think even that very idea as you brought it up, right, that like the CEO needs a bot strategy, you know, if you’re in a bot company like if you’re the VP at Google like in charge of android or something or you’re at Amazon in charge of Alexa or you’re at Apple in charge of Siri or something you need to be thinking really seriously about bots. But I think in most industries right now probably most CEOs don’t need to think that hard –

[0:19:00]

—about a bot strategy or I don’t see it as so much more pressing than one or two years ago.

Heather Taylor

Uh-huh.

Zach Lipton

and I think you are going to see kind of a huge wave of – I think you’re going to see an initial wave of sort of naïve people building things that don’t work and maybe some other naïve people buying them and getting chastened. And I think ultimately there’s going to be a huge amount of research progress in this direction and we will build real things that people interact with, but I think maybe some of the exuberance of corporate leadership right now is a bit misplaced.

Heather Taylor

Oh, fantastic. This was an interesting and very informative conversation. So I’d like to thank you, Zach, for coming to speak with us today. So don’t forget to add Simplicity 2.0 to your favorite RSS feed or ITunes. Thanks to Laserfiche for sponsoring today’s episode. Learn more about Laserfiche at laserfiche.com/simplicity or follow on Twitter at @laserfiche. Until next time, this is Heather Taylor for Simplicity 2.0.

[0:20:00]

[0:20:06]

[End of Audio]

Related Posts