Mi.Mu transforms hand gestures into music

By April 22, 2014
Mi.Mu gloves

Originally created for the British musician Imogen Heap, Mi.Mu gloves enable you to play music by moving your arms and gesturing with your hands. The project team have now launched a Kickstarter campaign to raise funds to make the technology more widely accessible.


Electronic sounds have become an everyday part of the production of contemporary music in various styles. However, some artists, including the British singer-songwriter and composer Imogen Heap, feel that staying behind a computer pushing buttons puts a damper on the musical performance and detracts from the audience experience. Natural gestures on the other hand can vitalize artistic expression and help make a connection with the audience. This thinking is behind the idea to create high-tech gloves that allow the artist to create and play music with hands and forearms. The Mi.Mu project has just launched a fund-raising campaign on the crowdfunding platform Kickstarter with the aim of raising $336,600 (€243,110) in 16 days. The Mi.Mu development team, together with Heap, want to see their gloves and programming software – which will be available in open source – used by a large number of enthusiasts. While still at the concept stage, the project was unveiled at the TED Global conference in 2011. In fact, the idea of data gloves is not really new, and this concept is actually reminiscent of Nintendo’s ‘Power Glove’, designed in 1989 to control basic functions in video games with hand gestures. However, the Mi.Mu gloves differ from previous variants of the idea in three main respects. First, the material itself is embedded with sensors in order to interact with the music. Secondly, the gloves have been designed to actually make music and enhance the musician’s ability to play a range of instruments at the same time.And lastly, Mi.Mu is different in the way that the data is received through the gloves and transformed to musical control signals. The algorithms that process information from the sensors have been created specifically for this purpose.

Capturing data and transforming it into music

Mi.Mu works by capturing movement and hand gestures with analog bend sensors. This information is then sent wirelessly to a computer via the x-OSC  wireless input-output board on the glove wrists. Remarkably, the technology allows a musician to play music in a very precise way without any time lag. It enables the musician to vary the note, the tone and even the effects. The open source software developed for Mi.Mu enables performers to map the data to musical control signals and to combine different gestures and movements so as to engender more complex controls. Using lights and movement sensors the gloves can link a wide variety of hand gestures to different instruments and sounds, with each pair of gloves having the capacity to store thousands of combinations. The gloves can also be programmed to control third-party music production and editing software such as the Cubase family of music sequencers. The Mi.Mu founders hope their product will find a mass market, but for the moment only a few rather idiosyncratic musicians have shown interest in the technology originally built for Imogen Heap, including an acrobatic DJ and an ice dancer. The first pair of Mi.Mu gloves, made to measure for Heap, had the finger-ends left open to allow her to play the piano, and the palms left bare so that she could clap her hands.

Artistic and virtual reality opportunities

This invention offers some interesting artistic potential as it blurs the boundary between music and dance. A dancer turns into a musician, a musician becomes a dancer. Other potential uses for Mi.Mu include allowing virtual reality hardware makers to insert realistic hand movements into virtual environments. The system can also be used to recognize sign language. An early backer of the project recently used a glove to trigger the Village People song ‘YMCA’ with American Sign Language. One of the challenges that the Mi.Mu designers have to deal with is embedding a large array of sensors and other electronics onto a very small surface, while still ensuring the gloves remain attractive, convenient and easy-to-use. The funding which Mi.Mu is hoping to raise on Kickstarter is intended to bring this experience within reach of a wide public. However, the price of a kit which has all the components to create a complete glove (accelerometer, sensors, x-OSC etc.) is $1,260 (€912), which is likely to limit takeup. Nevertheless, the development team wants to see their technology become accessible to everyone with an interest in this field – including hackers and educational institutions – so that they can then contribute to its development and help to build a community of designers. The team is planning to have the first gloves ready for delivery by the end of the year, and people who pledge to buy a glove will be given access to its software code before it is made open source in mid to late 2015.

Legal mentions © L’Atelier BNP Paribas