Tech Take: Putting an AI-driven virtual assistant to the test

, in
Tech Take: Putting an AI-driven virtual assistant to the test

By Neil Chalk
Product Manager, 15below

Staying one step ahead of the latest tech trends has always been a priority at 15below. We embrace the challenge of developing new systems that make a difference to our customers, whether that is by resolving a long-standing problem, making the working day a little easier or simply helping them to deliver a superior service.

Not only are we committed to supporting our customers through innovative and creative solutions, our products typically benefit their customers too, so the advantages of getting it right can be widespread. We know how important it is for our customers to deliver a quality service that meets all their customers’ needs and how we can help.

Plus, we love nothing more than a chance to tinker with the latest tech!

Perhaps the best way to look at new technology and put it to the test of helping our customers is by road testing it for ourselves – and that’s exactly what we’ve been doing with our latest idea: An AI-driven virtual assistant.

The idea behind how we wanted our virtual assistant to work couldn’t have been simpler. We would follow these steps:

  • Open our calendar
  • Set some defaults like office location and working hours
  • Copy in our virtual assistant to emails and it will arrange meetings for us

The reality was a bit more nuanced. The devil proved to be in the details. I'll run through three examples to illustrate the pitfalls the product currently faces:

Meeting with the CEO

I copied in the virtual assistant, asking it to arrange a meeting in ‘Nick's office’. However, when the invite went out the location had been changed to ‘Neil's office’. As you can imagine that provoked some amusement about taking over the CEO's office! In this case it was harmless but in other situations it could lead to confusion and more email to arrange a meeting.

Meeting with multiple stakeholders

This one was more complex with a couple of times suggested for when to hold a meeting between three other people. The email conversation reached about 20 emails before I stepped in and booked it manually. It got stuck in a bit of a loop in which it was confirming minor details in a round robin way. It might have been useful to have a calendar integration option for those people as well; this is how a real assistant would work for internal meetings.

After work dinner with a colleague

This one was to stretch the virtual assistant a bit further. It started off quite efficiently, but got derailed by real human level conversation and small talk. It also needed to check with me on a lot of details. We're creatures of habit, so I could have provided the virtual assistant with a bit more background in one go. However, it leads to about five clarifying emails. I can see technically why it did that - and it makes total sense to learn that way - but it feels laborious while you are getting up and running.


Overall the experience was hopeful. Once this kind of technology is linked with greater calendar integration and more situational awareness it will be helpful. But at the moment it takes a little while to become useful. This will provide a barrier of entry to most and, because that work falls on the people you interact with, everyone needs to be an early adapter who is going find the technology beneficial in order to make full use of it.

Applying this at a wider scale in B2C communications, rather than the personal ones described here, is going to take great care. You may only have someone's attention at a touch point once before they choose your competitor. You need that first contact to be slick and helpful, so I'm not sure general virtual assistants are quite there yet. Alexa on the other hand... well, you should see our demo!

Get in touch