The Exec Behind Amazon’s Alexa: Full Transcript of Fortune’s Interview

Adam Lashinsky: I thought it would be better to start by asking you to explain Amazon’s device business because I think people probably don’t know the gestalt of it, if you will.

Dave Limp: The device business is less about building hardware for customers and more about building services behind that hardware. So the original vision of Kindle was to deliver any book ever written in less than 60 seconds, and that was all about creating a cloud-based service that had a great catalogue of books, great selection, and great prices.

And as we’ve rolled out devices since then, everything from Fire TV to, as you mentioned, Echo and Alexa and everything in between, it’s about creating that backend service that constantly improves and adds value for customers, and isn’t just a gadget but instead a full end to end service that can benefit what customers want.

And give us a run through the whole lineup.

So with Kindle, we have Fire tablets; we have Fire TV. We have Echo and Amazon Tap, which are in the Alexa family. And then we have Dash, Dash buttons and a little Dash wand that if you press them, it allows you to buy something from anywhere in your house.

The following, I think, is conventional wisdom. Tell me if it’s accurate, which is that Amazon (AMZN) doesn’t think of devices as a profit-making business per se. It drives the rest of Amazon’s business which by the way usually isn’t but lately is profitable. That being the retail business. Is that so far still good?

What we’re trying to do is build a business model where we sell our products—the hardware side of the products—effectively at cost. And we think that aligns ourselves very well with customers so that if they take a product, say, they took an Echo and they just brought it home, didn’t like it and put it in a drawer, we shouldn’t profit from that as far as we’re concerned.

We really believe and the team believes that we should align ourselves with both the business model and the product, so that if customers use it over a period of time, then we’ll take a small amount of profit every time they have a transaction. It might be an Audible book; it might be a Kindle book; it might be shopping as they go through the lifecycle of that product.

Nothing makes me and the team happier to see a first generation Kindle in somebody’s hands. We’re still supporting it. You can still buy books from it and that’s a great win-win for us and the customer.

The point is you’re still making money on that relatively old, relatively basic device.

Yes, but we’ve constantly upgraded it too, right? It’s gotten multiple software upgrades and the store has gotten their—

Of course, but I just meant that old one that you referenced, the very first version.

Yes.

I’d love it if you’d spend a few minutes telling everybody the story of the Echo, from its idea phase to middle of 2014. Is that correct when it was launched?

Yes, it was launched then, in invite only. About a year ago June, it was available for customers. It’s a slightly interesting story. The two inputs that went into the formation of the idea were because Amazon had a little bit of a front row seat to a couple of trends. The first one, and I know you’re talking about a lot at this conference, is the sort of revival of artificial intelligence and machine learning.

We were using machine-learning algorithms internally in Amazon for a long period of time. Mostly in the early days for our recommendations engines. When you go to the site, you see what products we recommend. Sometimes that’s really correct, and the reason that’s correct is some of these modern algorithms.

And we challenged the teams, once we saw the success of our recommendations engines and the machine learning, to go ahead: how could you use those similar techniques in other areas throughout Amazon? So that was kind of the first input.

The second one—and again we’re a little more public about it now—but four or five years ago when we were envisioning Echo, AWS [Amazon Web Services] was a little bit more obfuscated. But we saw internally how fast the cloud was growing and how quickly AWS was taking off, and how efficient a developer could be by using Compute and Storage … points and time in the cloud.

And so the second challenge we gave to our teams was envision a world, in the not-so-distant future, where you can assume you have infinite storage, and you weren’t bounded by some personal computer or device that you had in your hand, or whatever it would be.

And a lot of teams throughout Amazon, including the Echo team, took those inputs and kind of went off and sort of came up using our standard invention process, which we’d write a press release around a product, and we’d review that.

And the soul of Echo was really about creating a very thin client, a small device, that could have all the power in the cloud that could replicate the Star Trek computer.

Show my geeky side here a little bit, but the Star Trek computer, for those of you who remember, anywhere on the Enterprise, you could tap a button or you could just speak to the Bridge and say, “Computer X.” And X could be anything, and the Enterprise—that computer would always answer it.

And so our vision is to create a voice-controlled computer in the cloud—Alexa—that can do exactly what the Star Trek computer did.

It will take us years and years to get there but that’s the goal.

Did you have other names in mind besides Alexa?

We did go through a number of names and the name is important as much for the personality that it creates around the persona than is this computer-based voice computer in the cloud. But there’s computer science behind it, too…[I]f any of you have Echoes, you know that it only wakes up when it hears the word Alexa, and the phonics of that word and how that word is parsed and the fact that it has a hard consonant with the X in it, is important in making sure that it wakes up only when it’s asked for. And so, a combination of those two things allowed us to kind of narrow in on Alexa.

There was a little alliteration from the fact that we were trying to create this Star Trek computer that was a little reminiscent of the Library of Alexander, and that was a source of all knowledge and that came into the sort of story of the name as well.

But also years and years ago, Amazon bought a small Internet search engine called Alexa.

Yes, and that’s still up there today.

No connection?

Not any connection.

So I was wrong about why—

— not any connection to this technology.

Okay. Now I have a cousin named Alexa, and you must have researched this. She can’t own one because Alexa is going to wake up every time somebody says Alexa.

Yes, we have a lot of examples of that, and dogs and cats. So we do give two other choices for wake words. It’s not the primary ones, but the customer can go into settings and change it to either Amazon or Echo, and those work just equally as well.

Now different personal question. I apologize if this feels indulgent but I want to know; this is an artificial intelligence question really. In my family, we are convinced that Alexa is more responsive to me than it is to my wife. Is this valid or are we imagining things?

It might be personal bias for you, I think.

It isn’t just my opinion. It’s her opinion.

The data—Alexa, the service, works pretty much—we do track that, and it works pretty much equally as well for females and males. There’s not a statistical big difference between the two.

But does it learn a primary user more than over a secondary user, or should it work equally well with anybody in the room?

It should work equally as well, but it does get better over time. So if somebody just walked into your house and went to your Echo first, for the first time, compared to you who might have been interacting with it for weeks or months—

Otherwise known as the Master.

Alpha something, yes. I would say that yours would be more responsive. It’s training itself in the cloud constantly, and so the more usage it gets, the better off it’ll work, she will work.

What’s an example of a press release that your team wrote that didn’t get adopted?

Boy, there [are] a lot of them.

I know but I’m just interested in one.

I would tell you that the first couple of Fire TV press releases that we wrote, that were previous versions before we launched it, didn’t make it through the sort of green-lighting process of a new product. Mostly because this was a category that was reasonably well-served.

There was Google (GOOGL) out there and Apple (AAPL) out there and others. And the general idea that were in those first couple of press releases around Fire TV were not differentiated enough.

You really want those press releases to say when a customer opens up that box, they will know that they got oversized value and they really got something that was interesting.

And it wasn’t until the same speech team that does Echo figured out how to get voice search on Fire TV working really well—highly accurate—that we had the epiphany that that was enough differentiation to kind of green light that product.

And now going to a streaming media player that doesn’t have voice search feels like you’re back in the Stone Age.

Jeff Bezos has recently and for quite awhile, spoken very eloquently about failure and why failure is important for entrepreneurs, for inventors, for Amazon. The Fire Phone was a failure and you took a large write off. Could you explain what went wrong and why?

I’d start by saying, and I think Jeff mentioned this not too long ago, which is we want to [incentive] the teams inside of Amazon to take big risks. If you’re not taking big swings for the fences, you are never going to move the needle in what is a busy and fast-moving technology segment.

If you go back in time, the Fire Phone and Echo were started about both the same time.

I’d call them equally big risks. There’s a thousand-plus people working on Alexa right now. That’s a very, very big investment.

I’m sorry, a 1,000 people working on it now?

Yes, now. That’s a large number of engineers. They’re invented very fast. And there was no guarantee that Echo would be a success. It was kind of all out there as well.

But the specific question, I think that the segment that is phones is very well-served, and we didn’t come out with a product at that particular point in time that was differentiated enough for customers, back to that press release, such that it got the momentum that we needed to see.

I assume there’s things you’ve put a lot of effort into, you took big risks, you invested a lot of money, and you didn’t launch. That’s sort of the right way to do it, right? But you took the next step. Can you reflect on that at all? Did you have a sense if it wasn’t going to work, or did you not know?

No, we didn’t know. A lot of us used them..internally. I think at the point we shipped Echo, we had 3,000 Amazonians using Echo before it launched. So, we have a pretty good sense of what’s working, and a lot of people were delighted by that phone. And what it offered.

So, it just didn’t resonate to the next level of kind of the masses. The phone segment is very large and to be successful, you have to get to scale, and we were just unable to get to scale.

Today’s Prime Day.

Thank you.

You’re welcome. I’ll let you tell everybody what Prime Day is. Would you show everybody else the thing in your pocket and tell us how it’s going?

Oh, I have a Dash button. Well, Prime Day is the day for—any of you Prime out there, you can get a good set of deals on the site today, which is great. It’s going to go on for a few more hours, and so far so good.

Please wait till the session’s over. We’d like your full attention please.

You don’t have to. That’s fine. (Laughter) I’ll forgive everyone.

Well said.

But yes.

But tell us about this thing.

Well, this is something we came out with about six, eight months ago in general availability, which is just a little button. It’s got a battery in it. It has a small wi-fi, and today you can go on the site if you wanted to.

Normally, they’re $4.99, and then it’s basically connected to one [Amazon account]…I have a refrigerator in the garage. I never remember to stock it full of Gatorade. I have a little button that’s attached there; it just goes back to some of Amazon’s original premises, which is to make things super-convenient for customers.

This is a path to a future, though. The future is the backend service behind these Dash buttons, which we call Dash replenishment service, which is we envision a world—and again very soon—where devices all around your house will be connected and replenish themselves. The example I like to use a lot is the smoke alarm that goes off in the middle of the night.

Whoever the programmer was for the smoke alarm was Machiavellian. They always go off and —

That’s one word.

Yes it is. They go off at 2 a.m. and something about the tone, you cannot triangulate on it at 2 a.m. It’s impossible.

So you never know where it is. So that is a fail to consumers. Our vision would be 2 weeks before that battery went dead, that battery should reorder itself and you should get a brown box on your front door step in a day or two, and it should have a note in there that says replace the battery and the smoke alarm in your daughter’s room.

And if that works, and that backend service behind these buttons allows us to do that—we already have printers and all sorts of things coming out.

I’ll say that explanation is far more compelling than “I’m just no good at remembering to replenish my Gatorade,” which sounds like a solution in search of a problem. But the smoke detector, that resonates.

I think they’re both big. There’s a lot of people buying and using these buttons. I think we’re onto something.

How many people in the room have an Echo? (Pause) That’s pretty good.

That’s very nice of you. The team will be very happy.

For more on this interview, watch this video:

You’re the head of devices and services, so let’s talk about the service a little bit. You invented a term for how the Echo works and it’s called a “skill.” Explain to people what a skill is, and I’m very curious how you came to that word.

Well, we wanted to open up Alexa as a platform. It is fairly unique. It’s one of the first voice platforms—maybe the only one that we know of that’s truly open, that you can as a developer with a few lines of code put up that code and actually run and add-on to Alexa. Add to her knowledge base, basically.

And the way this is done is you can use any kind of back-end service. On AWS,…you write a small amount of code, connect that to your backend cloud, and then Alexa can be enabled to understand that skill.

So take Uber, for example, or Lyft; they both have skills on Alexa. You just say Alexa—with your voice now—you say, “Alexa, enable Uber.” And now I can—with my voice say, “Alexa, ask Uber to order me a car.”

And that’s all. It’s zero friction, super-easy to do, and it’s taken off incredibly fast. We’re at over 1,500 of these skills, tens of thousands of developers, and it’s just growing; it’s right at the knee of the curve. It’s growing really, really fast.

It’s easy to use, it’s ingenious, and it’s fun—which I’m going to get to in a moment. What are the two or three most popular “skills?”

I think the one that is taking off very, very fast—I should say music first of all. Pandora and Spotify super-popular. And we’re very happy to have those …

It’s funny you mention Pandora and Spotify and not Amazon Prime Music.

Well, we have Prime Music as well. I was trying to highlight the partners there. But then I think a lot of the skills, like I mentioned on Lyft and Uber, a lot of people are using those.

And then we’ve been very surprised at the uptake of the smart home skills. And we’ve extended those through various sets of APIs. We have a lighting API and a lot of people plugging into that.

We have launched about three or four months ago a Thermostat API. Honeywell (HON) and Nest just started shipping their skills around that.

So, it’s really interesting to see that happen, and then it wouldn’t be any good developer ecosystem if a cat-facts skill wasn’t popular.

Cat-fat?

Cat facts. Facts about cats.

You’re a good businessman; you mostly mentioned very practical commercial things. A little bit embarrassed to say this, but I have a burning question that I need to know. In my house, one of the most popular skills with my nine-year-old and at least one person who is older than nine years old is Alexa Ask for a Fart? (Laughter) Where does that rank?

You know you have crossed a chasm when you get a fart skill. You’ve … Yeah, that’s there. Also, Alexa could be the largest repository in the world of corny jokes, so that’s also very popular.

Do you know where our household’s favorite skill ranks?

I do not. I’m not tracking it personally.

(Laughs) Who would like to ask Dave a question, please? I’ll save Mark Mahaney a little time in saying who he is. Go ahead Mark.

Question: Two questions. One, do you have a sense yet of whether your devices feed the retail flywheel. You have a great cross-selling capability. Do you have a sense of whether the Echo could do more for that than the prior devices that you had? And then getting to the Fire Phone, it was undifferentiated at the time. I was surprised you stopped it so soon. Are you thinking about coming out at a much lower price point—it’s a commodity item now—and if you can cross-sell, it’s more opportunity, more loyalty with customers?

Yeah, on the first question, you’re exactly right. The nice thing about the Amazon device business is that when we sell a device, generally people buy more blue jeans. And little black dresses. And shoes. And so that’s good. And it’s probably too early to tell. We’ve just really enabled voice shopping broadly on Echo. Just maybe a week ago.

So it’s probably too early to tell whether or not that will take up.

I can tell you that on digital things that are enabled by Alexa—for example, audible books, we do see lift there. And that’s very positive for customers. Customers love to hear audible books and that works well.

On the Fire and what we might do next, I wouldn’t want to speculate too much on roadmaps. Obviously mobile’s important and is something we pay attention to.

Question: Scott Kirsner with the Boston Globe and Echo user. As you add more skills to Echo, how do you envision the need to be able to navigate through skills that you may have added months ago and you forget are there? And maybe talk about domains like, “Okay, Echo, I want to talk about travel,” to help it know that you want to ask it questions about Aspen and help it understand you a little bit better? I think the question is similar to the iTunes problem in the app stores. You’ve got so much, how do you—

Yes, I think there’s really two questions there. I’ll start with the second one because it’s the more macro one which is the goal long-term is not to have to have you put it into—”it” being Alexa—a specific mode if you will. We want it to be conversational. It should be just like we’re talking to each other right now. We’ve learned this since birth; we’re very good at it. And to do that, it’s going to take us a long time.

We have to get a sense of a short-term memory and a long-term memory, and we have to understand context and we have to understand who you are versus who your wife is. And so there’s a long way to go.

But we have a lot of pieces of that inside of Amazon. We’ve been personalizing the website for many, many years. And I think it’s not an unsolvable computer science problem, and it’s one that the team is doing a great job working on.

On the specific one, we are trying to add to discovery of skills, and there’s sort of two efforts there. We just added a skills store. Before it was just a list and a list doesn’t scale very well. And so, that’s there, and that will take advantage of all the tools that you see on Amazon which is our recommendation engines, our ability to merchandise, and we think we’re okay at that.

And so hopefully that’ll benefit customers. And then we’re also voice-enabling the ability to discover these skills. So I said enablements there, and you’ll see more things coming in the not-so-distant-future about how we’ll be able to give you more information about what might be available.

So, for example, more multiple skills in a single category. How do we help the customer disintermediate among multiple skills, for example. So you’ll see a lot coming on that.

Question: Sutha Kamal from Technicolor. So, the Echo is a great device and it’s a great thing in my kitchen, for example, to talk to Alexa. But it seems like ultimately the race is to have the voice assistant that works everywhere. It works in my car; it works on my phone; it works in my living room; it works in my office. It seems like that puts you in competition with Google and Siri in places where they have an unfair advantage. How do you guys think about making Alexa more pervasive in that world where there’s a pretty aggressive gatekeeper?

Yes, I think the most important thing that we have to do, and we’ve started, is create self-service APIs that allow anybody to use Alexa, for free by the way.

So just like AWS has externalized lots of Web services—Easy 2, S3, Dynamo TV—name your favorite AWS service—so, too, the Alexa team has externalized APIs called Amazon voice service that allow you to point to that and be able to enable any device, or any app, with Alexa’s capabilities.

And so, you’ve already started seeing some of those come out. There’s been a couple of hardware devices; there’s been a couple of apps. The pipeline is very strong. There’s been some announcements by auto manufacturers. Ford (F), for example, has come out in support of it.

And so, our goal is to try to create a kind of open, neutral ecosystem for Alexa that doesn’t cost anything. And make it as pervasive as we possibly can.

Fortune: If I could just paraphrase, his concern is also your goal.

Yeah.

Question: It does sound like you’re describing a new operating system. Is that the way you see it in terms of your grand ambition? And then just a small related question, how do you make it just so much faster than the other competing products in the category?

It just seems like speed is the overwhelming winner in terms of why Alexa is so much better at this point.

Yes, it is a platform for sure. I don’t know if I’d go so far as call it an operating system. That seems a little backward-looking on how operating systems are thought of these days.

But I’d definitely call it a platform. And then why is it so fast? If you ever want to sit down for an hour-long architectural discussion—it wasn’t always so fast. And for many years while it was in the lab, it was 4,000 milliseconds, 5000 milliseconds, 8000 milliseconds, and that’s the equivalent of me pausing and answering your question…So it’s very frustrating and we just didn’t release the product until we got the latency down. And a lot of hard work at every level of the stack went into getting the latency down.

But we’re not done yet. We’ve got to get it even faster.

Super-interesting and congratulations on your success, Dave.

Great, thanks, Adam.