Acoustruments: sound wave-based gesture controls for smartphones

By May 05, 2015

They are low-cost, can be manufactured using a 3D printer, and do not need power or Internet connections. Acoustruments, developed by Disney Research, offer a completely new approach to controlling your smartphone.

The working principle underlying Acoustruments is as straightforward as that of a flute – i.e. sound vibrations passing through a column – point out the research team at Disney Research and the Carnegie Mellon University in Pittsburgh, USA who came up with this control system. In the case of the flute, the vibrations are transformed into audible sound; in the case of the ‘Acoustruments’ developed by the Disney-Carnegie Mellon researchers, the vibrations serve as a means of controlling your smartphone and using its interface for a variety of purposes The way it works is that the phone speaker is connected up with its integral microphone through a sort of plastic soundproof tube which does not allow any audible sound to escape. By changing physical elements in the tube, the user can alter the vibrations so as to send commands to his/her smartphone through a software app which has been ‘taught’ to interpret the changes in sound. Starting out from this concept, the research team have since developed a series of devices with a range of applications.


A host of applications

The main advantage of Acoustruments is that they are easy to manufacture and run. Made of plastic, they can be produced at low cost by a 3D printer. Moreover, Disney Research’s little objects need neither electricity, Wifi, nor Bluetooth connection – which means, claim the researchers, that the number of potential end-user applications is practically infinite. The team have so far created prototypes for an alarm clock, a smart phone case, and a doll that will respond to touch when attached to your smartphone – for example reacting when you tap its tummy or place an object in its hand. Of course the principle can also be applied to purposes other than toys: the Disney-Carnegie Mellon researchers are now planning to use the technology in virtual reality headsets based on smartphone screen displays.


Disney Research’s smartphone-linked doll, which reacts to gestures

Adjusting our existing habits?

The team claims in its published paper that Acoustruments “can achieve 99% accuracy with minimal training”, but the system still lacks a standard: the researchers have in fact called upon a variety of gestures for controlling the smartphone. Requiring users to familiarise themselves with this set of movements seems a rather inept approach as there are many different movements and they change from one Acoustrument to another. This is not unlike the problem that arose with a connected finger ring and new gesture control systems for smartphones. In contrast with iPhone gestures, which are fixed, simple and easily assimilated by users, the commands developed for Acoustruments will take some time before people can use them naturally and automatically.


Legal mentions © L’Atelier BNP Paribas