This AI personal assistant took 3 years and millions to build - it completely fooled me
Amy and I exchanged eight emails fixing up a date and then another five when Blomfield had to rearrange. Only then did I spot something odd in Amy's email signature: "Artificial intelligence for scheduling meetings."
It turns out that I had been talking to an algorithm the whole time.
"I'm very happy to hear not that we've fooled you but that we've created a design that felt so natural that you probably said 'thank you' or 'would you be so kind' or something similar," its creator, Dennis Mortensen, told Business Insider. "That, I think, is a testament to the design success of the agent."
Mortensen is the founder of x.ai, the New York startup that has developed Amy and her male counterpart Andrew Ingrams (Their initials are AI.) The virtual assistant does one thing and one thing only: arrange meetings.
"We're not trying to recreate Siri or Google Assistant, we're trying to make this one agent - Amy, and/or Andrew - which upon being asked can make sure that the three of us end up on this phone number at this time," Mortensen told BI (x.ai's PR manager was the third person on the line.)
'It's very easy to imagine but it's very hard to execute on'
Mortensen had the idea for Amy/Andrew after selling his last company. He was reviewing an old diary and realised he had 1,019 meetings in 2012 and had rescheduled 672 times.
Getting a computer to arrange your diary is harder than it sounds. Mortensen says: "We're about 90 people who've been working on it for 3 years. That might come as a surprise but it's one of those things where, similar to a self-driving car, it's very easy to imagine but it's very hard to execute on. We've just invested the time and effort into it."
It is an expensive project too. Since launch in 2013, x.ai has raised over $35 million from investors including Japanese telecoms giant SoftBank and FirstMark, a New York venture capital firm that has backed companies like Pinterest, Netgear, and Shopify. Its most recent round was in April this year when it raised $23 million.
"Not a single person in your office doesn't do meetings," he says. "Even the most hardcore engineer does meetings. Not a single person says they love setting up meetings, they love that ping pong back and forth. They hate it."
Sean Masters, the founder of on-demand project and staffing platform Socialprise, told BI: "Scheduling meetings is just remarkably inefficient. She [Amy] just takes that away and it makes me more productive.
"I'm a bit of an advocate of her. She's actually got a bit of sass. One of my project managers has rescheduled a meeting with me three times. Amy responds saying she wants to reschedule this meeting, again." (Masters and I weren't meeting to discuss x.ai or connected by the company - the praise was spontaneous.)
X.ai offers a basic, free version of its service to individuals, letting them schedule up to 5 meetings a month, and recently launched a professional edition for the self-employed, costing $39 a month. The plan is to offer a business edition for teams next year.
Hundreds of thousands of people have used the system to date, Mortensen says, and millions and millions of emails have gone through the system. Masters estimates that using Amy saves him at least 3 hours of email admin a week.
'There's still ambiguity and we're still training on that'
There was controversy surrounding the company earlier this year when a Bloomberg article in April claimed that much of the work involved in the emails was done by a team of "AI trainers" who were "hiding" behind the service.
Mortensen insists the author got the wrong end of the stick. Humans are not secretly drafting the emails but are guiding the system to help it get smarter.
"If you look at what Uber did when they put about 30 [self-driving] vehicles on the streets of Pittsburgh - in every single one of those self-driving cars, there'd be two Uber employees," he says. "One is an engineer and one to override the cars prediction of what to do next - a.k.a. the trainers. That is how you move from a situation where only humans ride cars to one where only machines drive cars. If you think of nothing else, you should think of that analogy."
Mortensen says: "An AI trainer is sitting at a console saying I agree what the machine labelled as Wednesday is correct. I agree that 4pm is indeed a time reference, correct. Or if someone says 'have a good weekend' you need to say that's not relevant. If the machine thinks it is relevant, you have to say that is incorrect. Some things have been fully automated and no human will look at it."
The writing side of the x.ai system is "100% automated," Mortensen says, and the "vast majority" of the reading side does not involve human supervision. It is only when complex phrasings or unusual circumstances arise that humans step in.
Mortensen gives an example: "Our ability to work out if you want to cancel a meeting is not fully solved. Sometimes we have a very high competence level and sometimes we won't because people are too friendly. You would say I would love to chat today but things have taken a turn, we should get a coffee in the future - that is you being kind about saying the meeting is cancelled. That is tricky for a machine."
Monzo's Blomfield told BI: "It's really great for one-on-one calls and meetings. It's saved me a bunch of time. It's not yet perfect when there are multiple participants or a human assistant involved."
Mortensen says: "There's still ambiguity and we're still training on that."
'We either solve this, or we die trying'
Of x.ai's 93 employees, 39 are AI trainers. All are based at the company's New York offices, on 25 Broadway. A consequence is that when my colleague Lara O'Reilly tried out the service in July 2015 she noticed that often emails from Amy would only be sent in the afternoon in London - i.e. when New York woke up. (She noted at the time that the experience was also "far from seamless.")
Mortensen insists this is no longer the case.
He says: "The system today runs 24/7, 365 and we have a median response time of 9 minutes today. The system is very different today to what it was a year ago or even earlier in 2016. The response time we're running with, it's obvious we do more emails at higher accuracy, fully automated, without any verification."
Masters, who began using the service a few months ago, says there were "a couple of teething issues by they're just learning." He says he would have had the same issues with a person - he would have to get a sense of how they work and they would have to get used to him. Otherwise, he has nothing but praise for the service.
Ultimately, the goal is to do away with the AI trainers altogether and have Amy and Andrew talking directly to each other, arranging meetings for people behind the scenes.
Mortensen says: "We are making a bet on this being something we can turn into a completely machine-driven process. We are in very good shape and the vast majority of the conditions that we do are fully automated."
He adds: "We have no plan B. There is no outcome of this venture where we are a semi-outsourced setting. We either solve this, or we die trying."