[Survey] Virtual Assistants on the march towards omniscient AI?

By March 04, 2016
Vers une intelligence artificielle omnisciente ?

Recent progress in the field of artificial intelligence (AI) means that robots are now equipped to assist human beings in many different fields. These advances herald the emergence of a meta-intelligence which will be able to link up all kinds of connected objects.

In Stanley Kubrick’s masterpiece ‘2001: A Space Odyssey’, the astronauts on a mission to Jupiter are assisted by the spaceship’s computer HAL 9000, a robot with advanced artificial intelligence. HAL speaks in a relaxed manner to the crew as if ‘he’ were a human being, and acts as their intermediary in controlling the ship’s various systems and functions. Now the development of virtual assistants (VAs) is bringing this science fiction scenario day by day a bit closer to reality. Based on AI systems designed to support people in a variety of daily tasks, VAs have recently been making very rapid progress.

As regards the range of services on offer, we have in particular seen major advances in voice recognition technologies. There has been more progress over the last three years than during the preceding thirty, to the point where machines can today actually perform better than humans, says Tim Tuttle, founder and CEO of San Francisco-based AI technology development startup MindMeld. Virtual assistants are now able to understand most of the questions put to them by a human being and reply in an appropriate manner. Progress in computer programming has enabled VAs to hold conversations and even crack jokes. Lots of Internet users have had great fun relaying online the sarcastic remarks made by Apple’s virtual assistant Siri , which are almost on a par with the humorous banter that is the hallmark of Tars, the live-wire robot in the film ‘Interstellar’. 

Tars, one of the robots in the film ‘Interstellar’, is never slow to make a joke. (Gizmodo)

VAs have also benefited from improvements in programming interfaces, which allow new functionality to be added fast and efficiently. In the last few weeks Alexa, Amazon Echo’s AI system, has been empowered to control your central heating, launch a Spotify playlist, call you a car from Uber or order a pizza. Meanwhile Siri has developed the ability to book a table at a restaurant and post on Facebook on behalf of ‘her’ owner. On the demand side, users have started getting used to conversing with a robot, and many now use the technology naturally and intuitively.  Gone are those early days of perplexity tinged with irritation.

Another new aspect is that a virtual assistant is no longer merely the voice of your smartphone. Today VAs are at work supporting human beings in every field: in the fintech sector they can decide the best time to transfer your money abroad; in the health field VAs enable doctors to efficiently monitor patients suffering from chronic illnesses; in the auto sector, along the lines of the famous Kitt  (Knight Industries Two Thousand) – an artificially intelligent electronic computer module in the body of a highly advanced, robotic automobile – a VA is fast becoming a must in order to enable a connected car owner to take full advantage of everything the vehicle has to offer without jeopardising his/her safety or that of others; inside companies, AI systems can automatically programme business meetings; and in the ‘smart home’, AI allows you to monitor and control your various smart domestic appliances.


Trend towards market concentration or segmentation?

The advent of highly specialised virtual assistants poses one basic question as to the future of this fledgling market. Are we moving towards a myriad of ultra-specialised AI systems or on the contrary in the direction of having one super-assistant that is capable of accomplishing all imaginable tasks? The answer will no doubt be found somewhere between the two. First of all, it is useful to distinguish, as Dennis Mortensen, founder and CEO of x.ai, does, between ‘horizontal’ and ‘vertical assistants’. ‘Horizontal’ assistants, such as Siri, Cortana and Google Now, can help with any number of precise, limited tasks, but they are not capable of carrying out a complex assignment involving a number of different steps over a longer period of time. ‘Vertical’ assistants, on the other hand, are able to successfully complete an end-to-end task that may take some time to accomplish. Whereas Siri is programmed to reply promptly to a range of very specific questions, Amy, the virtual assistant developed by x.ai, is able to go and talk to a number of different people, check their availability and propose a date and time for a meeting that will suit everyone concerned.

There are signs that we may be moving towards an oligopolistic market for horizontal VAs dominated by the major names in new technology, led by Google, Apple, Facebook, Amazon and Microsoft. These Internet leaders appear to have embarked on a pitched battle to acquire innovative young startups that can help them improve their products. Google spent $500 million to acquire DeepMind, a London-based AI specialist; Siri was an independent startup before the firm was taken over by Apple in 2010; Y Combinator-backed start-up Wit.ai, which transforms words and text into useable data, was barely a year old when Facebook bought it in 2015, a move which has enabled the Menlo Park-based giant to build its assistant ‘M’; and Amazon has teamed up with ecobee Inc. to enable Amazon Echo to control your central heating. “Companies are hearing from Facebooks and Googles as soon as they announce what they are doing,” Matthew Wong, an analyst at venture capital database CB Insights, pointed out in the columns of the Guardian newspaper recently. Oren Etzioni, who heads the Seattle-based Allen Institute for Artificial Intelligence, quoted by the same newspaper on its website, argues that “the future belongs to the company that can make a personal assistant something like a good hotel concierge: someone you can have a sophisticated dialogue with, get high quality recommendations from and who will then take care of every aspect of booking an evening out for you.”

However, it is very unlikely that any of these players is currently able to design a virtual assistant able to accomplish all imaginable specialised tasks. ‟A central artificial intelligence system, which can carry out immediate action and answer questions, is clearly very useful, but this is completely different from assistants focused on one job, which can carry out a task over a number of days, and go back to the user after having settled all the ins and outs,” stresses Dennis Mortensen. Ron DiCarlantonio, founder and CEO of iNAGO, a pioneering firm that has been developing AI systems in Japan for the last fifteen years, makes exactly the same point: ‟If you take a medical assistant for example, the procedures are so complex that this needs to be created by one single company focusing exclusively on this field. The assistant has to be able to respond to medical questions and make out prescriptions, which demands a certain level of expertise in this area.” It is therefore likely that we are going to see basic standard platforms developed, on the basis of which everyone will be able to build their own special virtual assistant. Just as nowadays no-one need start from scratch when building a website, there are now a number of solutions that provide a common AI-VA platform which can then be customised for specific use.

Théodore, hero of the film Her, falls in love with his operating system (photo: Wild Bunch distribution)

Towards humanoid robots?

In the film Her, the character played by Joaquin Phoenix gradually falls in love with his operating system, an AI system – which is fortunately endowed with Scarlett Johansson’s sensual voice – capable of holding a human-type conversation, organising his daily life and even sharing his thoughts on music, love, and the meaning of life. In an episode of the television series Black Mirror, a young woman unable to get over the sudden death of her lover, purchases a robot which draws on online data in order to adopt his manner, speech and personality. Can we envisage that in the near future virtual assistants will become sufficiently intelligent and versatile for this kind of scenario to become reality? After all, VAs are already capable of holding a real conversation with people, going far beyond just answering their questions and actually getting quite close to the user.  It is also interesting to note that while earlier generations of robots, which did not possess very advanced intelligence, were often given a human appearance to make them more acceptable, the AI systems that we are starting to see today can come across as very human even though their bodies are totally different from our own – as is the case with Tars, the Interstellar robot – or even when they have no body at all, as with the ‘Her’ operating system.

In the early 1990s – before anyone else – Ron DiCarlantonio thought of building an artificial intelligence system with whom a human being could interact as with a friend. Today we are still very far away from being able to reconstruct the complexity of the human brain – DiCarlantonio says that even after fifteen years of work his AI system is only at the level of a human baby. However, ‘deep learning’ can enable machines to participate in complex discussions by picking out their responses from a huge database. ‟The main challenge comes from the fact that human beings often pose vague, imprecise questions. In order to reduce the problem, the AI system can ask a series of questions so as to lead the human being to make his/her initial question more precise. This is what we’re already doing in Japan in call centres and after sales service centres,” reveals DiCarlantonio, explaining: “In the longer term, the idea is that a robot will be able to dig into a large database of questions and answers and so remove any ambiguity from the question without having to ask its owner,” which will enable it to reply naturally to complex or evasive questions, just as a human being would.

However, one of the essential components of what it means to be human is evolution, adaptability, the capacity to react to the unknown, a characteristic which for the moment AI systems are entirely lacking. ‟We’re now getting to artificial intelligence systems with excellent memories which are able to browse their databases with ease, but intelligence is more than that. It’s also the ability to find a solution when faced with a new problem,” underlines Ron DiCarlantonio, adding: “Right now a programme can solve a problem because we’ve programmed it to do so. But what if we designed a programme capable of adapting on its own, in order to solve the dilemmas which we haven’t prepared it for, which we haven’t even thought of yet?” This is what is known as unsupervised learning – i.e. giving a programme the ability to learn all by itself, free itself from established rules and think independently.

Says DiCarlantonio: ‟This concept exists today, but it’s difficult to do it well. Giving a programme the freedom to choose its own actions means it can do everything it wants… and in many cases it will not do what we expect! This is why many virtual assistants, such as Siri, have a limited number of ways they can interact. At iNAGO, we’ve gone a bit further by combining predefined elements with other more dynamic ones in order to enable more natural conversations with Mia, our digital assistant. In the future, we’ll certainly have programmes capable of adapting and evolving, and deep learning will be one way of getting there. The programmes will enable the software to understand the huge variety of things that a user might mean, to figure out what the user really wants and do what’s required to ensure satisfaction. Keeping control over artificial intelligence systems that are able to go beyond what we’ve taught them will take time, but we’re optimistic!

One could perhaps draw some inspiration from Isaac Asimov’s ‘Three Laws of Robotics’, drawn up to ensure that robots work for the good of their human creators. However, VAs are today mainly being developed in Silicon Valley and so designed to understand and converse with Western users. In order for interaction to work equally well with people coming from radically different cultures there will no doubt have to be an adjustment phase. And care should be taken to ensure that the technology developed in Silicon Valley does not impose its norms on the rest of the world, forcing other cultures to adapt in order to use it. This is another way of saying that technology should not allowed to dictate to human beings, a fear evoked by French philosopher Jacques Ellul in his book ‘Le système technicien’ (The Technically-Driven System).

This scenario, whether you find it exciting or scary, is perhaps already on our doorsteps. This is at least what the articles published on the site of the highly secretive startup Viv – which was launched by former Siri creators three years ago – would have us believe. According to information reported by the Guardian and Esquire magazine, the firm is, like iNAGO, working on creating a programme capable of writing its own programme, so that it would then be able to carry out a potentially infinite number of actions.

Isaac Asimov

Changing the face of the world?

If Viv’s venture (or that of iNAGo, or of any other firm with similar objectives) proves successful, the consequences could be nothing short of explosive. Firstly for the user, who would be able to make highly complex requests and leave his/her assistant to take care of a number of aspects of his/her daily life. The assistant could combine geographical and meteorological data with other information drawn from the user’s contact list in order to handle a request of the type: ‟If it’s raining this evening, find me a pizzeria close to where my brother lives.” There would no longer be any need to spend hours on flight comparison websites to find the lowest price airline tickets, or to go through online comments with a fine tooth comb to find the best hotel for your holiday. All you would have to do would be to give your virtual assistant a few ideas and the VA would then organise the entire trip.



But the Viv creators are not planning to lock up their VA inside a smartphone or computer. Long term the idea is to design a platform which can be integrated into any connected object whatsoever. In the era of the Internet of Things we are going to need a ‘connector’, an overall brain which enables smart devices to communicate with each other, so as to give the user the best possible experience. In future we will for instance be able to ask our connected refrigerator about its current contents. The fridge will be able to connect with a cookery site such as Yummly to obtain an easy-to-make recipe using the available ingredients, which matches his/her tastes and mood – what s/he’d like to eat right now, any specific dietary requirements, meals eaten recently, etc – suggest a wine to go with the food and call on Google Maps to find the nearest shop where the wine can be bought. Meanwhile your AI-equipped lawn-mower will take care of your garden while you are on holiday, automatically ordering fertiliser from the supplier if need be. The VA will link up with a dating site and then suggest to its user that as he shares his latest match’s passion for old B movies, and that the cinema on the corner is showing one this evening it might be a good idea to book some tickets… And perhaps call for a car from Uber so that he can go and pick up the young lady at her home? Should the assistant also book a table after the film at the excellent gastronomic restaurant just a stone’s throw away from the cinema? And arrange for flowers on the table at the restaurant? The possibilities are infinite.

In such a scenario, suggests the article in Esquire, the entire Internet advertising model will be shaken up. Some commercial sites today pay Google millions to be among the top recommendations when an Internet user searches for one of their products or services. For example, a site selling airline tickets pays to ensure it is high on the list when you type in a search for a flight. If tomorrow everyone has a VA that has perfect knowledge of the user’s personal preferences and knows how to find the best deals, this may obviate the need for Google searches. The whole advertising paradigm will need a rethink. The connected objects market might likewise take on a whole new look: in tomorrow’s world why would you need a smartphone if your fridge can perform all the same tasks? Why invest in the latest computer when your smart wardrobe can order clothes for you on the Internet? Never has the concept of creative destruction, popularised by Austrian-American economist Joseph Schumpeter, loomed so large as in today’s AI-equipped world.

Joseph Aloïs Schumpeter

Incumbent players in all kinds of sectors will have to adapt their business models to the new realities. The automotive sector is an obvious example. It will soon become absolutely essential for car manufacturers – a number of whom have already embarked on this path – to provide an integrated virtual assistant in the vehicle, offering a range of services to the driver, such as warning him/her that the brake pads are worn or the oil too low for a long journey, or suggesting that s/he put on snow chains if there is a risk of snow, and so on. In this case the VA will act as a sort of onboard garage mechanic, and could even recommend nearby garages for that specific car brand to the driver en route. The value of the vehicle therefore looks certain to gravitate slowly away from the hardware towards the embedded software. iNAGO has been working for several years with the leading Japanese auto manufacturers and has recently teamed up with a US automaker. In the food sector, we can easily imagine a group such as Danone forging a partnership with MyFitnessPal, a smartphone app which helps you keep a close watch on what you eat, suggesting products to fill the gaps in your daily intake – milk if you show a lack of calcium, and so on. Adding a connected watch to the package will enable the consumer to check his/her health/biometric indicators in real time and automatically order the products s/he needs. In the finance sector, banks will need to provide customers with virtual assistants that will help invest their funds in the best-performing stocks, in line with each customer’s individual areas of interest.

However, the road ahead is not entirely free of bumps and potholes. Apart from the difficulties highlighted by Ron DiCarlantonio, the VA of the future should be able to store and manage massive amounts of data and win over a large number of users without having a customer database like Google’s or Apple’s on tap – unless of course the VA happens to be acquired by one of these giant players, find software developers who can organise the integration of the assistant into a wide range of platforms, and, not least, persuade users to share their personal data with a robot. After all, in all the best science fiction, whether we are talking about ‘I, Robot’, ‘2001: A Space Odyssey’, or ‘The Matrix’, robots often end up rebelling against their human creators!

Perhaps the greatest danger lies in making everything uniform. If at a given moment Yelp is deemed to be the most reliable recommendation service for a restaurant, all the personalised VAs might well automatically go to Yelp each time their ‘boss’ asks them to book a table, creating a stranglehold which would close the door to new entrants. Amazon’s competitive advantage would thus morph into a crushing monopoly. So we will need to find a system which allows us to preserve personal choice and diversity, for example by introducing a range of criteria other than mere efficiency.  For the moment we are still in the realm of science fiction with such speculations, but for how much longer?

Legal mentions © L’Atelier BNP Paribas