About the Project
In Brief
The Gloves are a cutting edge experimental gestural music ware being developed today for the purpose of Imogen Heap’s studio and stage work.
Using a unique gestural vocabulary, motion data-capture systems, and user interfaces to parameter functions developed by Imogen Heap and her team, artists and other users will be able to use their motion to guide computer-based digital creations. The Musical Gloves are both an instrument and a controller in effect, designed to connect the user fluidly with gear performers usually use, such as Ableton - think minority report for musicians brought to you by the DIY/maker revolution.
The Story
As is evident in all of Imogen’s work, the gloves are an extension of her “no smoke and mirrors” approach when interacting with her audience. So often, tech set-ups on stage create soundscapes disconnected from the audience, who areleft without an idea of how it is all happening.
If you’ve seen Imogen play live, you’ll have noticed a shed load of tech and wires, some attached to her, some not , with her, racing about between various pieces of gear with small microphones attached to her wrists and a headset mic, singing, looping, playing percussion, keyboards, and of course applying effects to all as she goes. It’s great fun to watch, but she always felt frustrated, adapting music to the tech, rather than the other way around and making the best she could with off-the-shelf devices available in the rushed time between album and touring cycles.
“Winding back, when I was 12, I discovered an explosion of musical freedom in a small cupboard, spending hours on a neglected Atari computer at my boarding school. It was so liberating to be able to actually physically play back and edit my compositions and so many sounds! I’d played various instruments growing up and cobbled together recordings with them and my little keyboard onto cassettes or writing on manuscript paper but this was a whole new level. I knew I’d found a friend in music with technology and have been hooked ever since.”
Over the years, she has accumulated more and more complex systems to try to free her up on stage using wireless midi and radio mics attached to her wrists, with various wireless keyboards in tow to control her harmonisers, computers, samplers and sequencers. “I feel tethered to a keyboard or other control surface for computer duties and when you’re ‘stretching’ a sound or changing its pitch, a button, wheel or fader just never really cuts it. I always longed for more expressive control of the tech in studio and on stage, something I could wear and create sound fluidly with, more organically, humanly somehow.”
In 2009, when Kelly Snook invited Imogen to visit to MIT’s Media Lab, she realised the tech had now made this possible when she tried on Elly Jessop’s musical glove. “Elly’s glove could record and loop the voice with a few intuitive hand gestures. A million possibilities flooded my imagination! I was quite emotional and as soon as possible I changed my touring and recording life cycle to get to work on creating my dream setup that felt ever closer than before.”
She called up Tom Mitchell, who’d worked with her before in coding her Monome for a Letterman performance and told him the idea. He loved it and got funding from the University of Western England (UWE ) to do a research paper on the work. She asked 5DT for a set of gloves to get going and began experimenting.
“The tech is all out there, which is exciting, the challenge more now is in the language and how to better articulate the hidden or techie side of stage performance, so it looks and feels natural to engage with. There is a long way to go to add to the dialogue; this is a lifelong project but already it’s greatly encouraging where we’ve come as a team. It feels like such a treat to make something physical as opposed to music. From balsa wood buildings and birthday cards to musical gloves, perhaps my parents’ engineering and artistic backgrounds have brought me here?”
Kelly Snook, in 2010, came to work for Imogen full time as her studio as technology manager. As the team grew, Kelly took on the role of co-ordinating the work flow and engaged in setting up the audio side of things for testing.
Imogen says, “The first hurdle was to tame the data coming from the gloves into an adaptable, controllable set of numbers, defining with them the hand postures we’d use. The more we did, the more I began to think in 3D, the more ideas I got and so on and so on! Very soon I needed more data to play with, to be more expressive and have more control.”
Enter Seb Madgwick with his X-IMUs. Now they had the possibility of not just hand postures but speed and positional data. Next was ‘modes’ for different functions like ‘drum triggering” controlling the speed of the music, pitch etc., but Imogen couldn’t move between these by switching a button! It was important for Imogen that every action was represented in an intuitive motion., thus serving both the audience in their understanding of what’s going on onstage and Imogen to be able to effortlessly play the music using gestures that feel like second nature. “As a simple example, switching into drum mode would be ‘picking up a pair of sticks,’ not just a complicated sequence of disconnected movements that I need to remember.” Here the team began to recruit more help, as there just weren’t enough hours in the day for Tom to do all the coding and software building. Adam Stark joined the team, partnering with Kelly Snook to begin creating a system to make it possible for Imogen, via a user interface, to program and connect her music production and performance software to her every move. Fader movements could now become hand waves, the pan pots of the placement of the sound, or an effect become a twist and curl of the wrist.
Essentially the idea was that the Gloves would take on the personality of a musical companion, not trying to do take the place of what a keyboard or computer is good for but finding and celebrating those things that the gloves do better : expressive and immediate control of sounds recording and played through gesture, “Like painting music rather than typing it into a spreadsheet.”
Kelly suggested her friend, Hannah Perner-Wilson, to join in the team as the gloves now had become a mess of separate elements with wires hanging off and lights sticking out. Hannah began to make a custom pair for Imogen. Part of the brief was to celebrate the tech and not hide it in the design. Also, to design them with elements like having an area to ‘clap’ with and fingerless tips so she could still play percussion and feel the acoustic instruments she wanted to be able to play live. Having mocked up a few pairs, they thought it now time to focus on the harness to hold the gear on Imogen’s back. Hannah sent Imogen some online references of boleros that could work. One, in particular, got Imogen writing a note to Rachel Freire, a London-based artist and designer, to ask if she would join in the fun (she said yes).
The Gloves outfit is made up of three essential parts: The Gloves, armbands and a harness. But now, also with a public performance on the horizon and Rachel on the scene, a unique bolero was in the works!
The harness holds all the wireless audio and bluetooth gear to ‘speak’ to Imogen’s setup off stage. The wires from her gloves are threaded through some ‘crin’ tubing and ‘armbands’ to stop them from dangling and up into the harness. The bolero adding to the aesthetic, combined with Imogen wanting to visualise the level of her voice in LED lights for extra data feedback for the audience, a stunning bolero was put together by Rachel and Hannah, where the circuit board in sewn in conductive threads on the exterior of the collar.
The outfit and the gloves were shaping up nicely in Rachel and Hannah’s studios. Back at Imogen’s, the team grew an extra member in Becky Stewart as Imogen started to expand her mapping by dividing the stage into zones that would give her an extra layer of control and playfulness using the Kinect. “My proximity to the audience becomes a part of the music and where I am on stage could change the timbre or pattern of a sound.” Adam and Becky (who make up Codasign) worked together to add this into the user interface for Imogen to map into Ableton.
It turns out, perhaps obviously, that much of the fluidity and shifting through the music comes from hours and hours of programming what the gloves get up to musically, via the user interface. Here is where Imogen interacts with the gloves and with the live music production software many performers use called Ableton.
Music performance software doesn’t come off the shelf made for a pair of gloves and so she often meets sticking points. This either ends up in Imogen calling up the team to create her something within the patch to help her get around that, or some long arduous route with many plug-ins and mappings for it to flow well on stage. Things you’d expect to be simple like ‘muting’ a sound for instance can be surprisingly clunky to solve yet previously tricky manoeuvres like manipulating the timbre, tuning, length of note, velocity, panning, filtering and generally moulding the sound in hand (literally) come second nature when using the gloves.
This is the most exciting part for Imogen. For the first time, she’s literally dancing around the room. Exercising and making music, imagine that! Usually all of this involved moving a mouse around a computer screen or the ‘old way’ of using faders, switches and knobs.
Preset switching (between sets of mappings within Ableton) she’s doing with the Gloves and Kinect. Using the stage area like hot spots or ‘sound worlds’ or grand overall effects like more/less vocal effects (closer/further she is away from the Kinect).
For a long time Imogen had been dreaming of being able to literally shape sound with her hands, like play-doh; a direct connect, from musical inspiration to creation; to move freely (wirelessly) singing, recording, looping, effecting, amplifying and playing instruments, making music with gestures in the studio and on stage. Basically, the dream was to do all the things she already does on and off stage but without having to be chained to a base station to move a fader up and down or flick a switch.
This is now almost a reality as Imogen is improving the Musical Gloves with an increasing band of talented coders and makers she is grateful to have around her, along with her technology manager, Kelly, who has helped coordinate so much of the innovation with her.
“I’d like to thank Ellie Jessop for giving me a point of inspiration on where to begin the quest back in 2009, when I visited the MIT Media Lab, and to Kelly Snook for connecting us that day!”
The Performance
After showcasing the Gloves v1 at TED Global, Wired Future of Music, and TEDx Bristol in the summer of 2011, and then at Music Matters Mumbai later in 2011, it was time for The Gloves to enter a new phase that would reveal not the just the ideas in the making but an actual performance with music composed with and for the Gloves, themselves.
Imogen embarked her team in a new journey taking the prototype of the Musical Gloves to new levels.
On Earthday 2012, April 22, Imogen produced a world premiere concert for the release of ‘Heapsong 6,’ her new song composed with her Musical Gloves called ‘Me The Machine,’ to mark the midway point of her fourth solo studio album comprisings songs released one-by-one every few months.
After a 30-minute making-of video of the Musical Gloves, the song was performed live at an event held on the grounds of Imogen’s elliptical home, and was filmed and broadcast live on Facebook by FANetwork TV.
To tie in with Earth Day, the whole event (including rehearsals) where be powered “off the grid,” with the best available energy mix locally sourceable. This included friends, family and fans on generator bicycles to power the LED lights in the music video they were a part of! So the only reason people were able to “see” anything on the stream was because they were be sweating it out!
You can find out more by visiting Imogen’s website here.
