Professor Bowman reviewed my initial proposal, which I wrote about in my previous entry, and he had some good questions and comments. I've considered what he had to say, and I'm documenting my thoughts and responses here.

In response to my observations about the theremin's challenging learning curve, Professor Bowman asked:

"If the theremin is so difficult to play, then why base your initial design on it? I would not get locked into the idea of position being directly mapped to pitch or volume, since there is nothing in the physical space to tell me where I am or where I want to be."

This is a good point, but I would argue that all musical instruments are difficult to play for beginners. In general, instruments that are too easy to play end up being severely limited in range and expressiveness. My goal is not to design an instrument that is simple to master, but to design an instrument that is simple to start playing. Theremin technique has had 95 years of refinement. I can't think of any other touchless interface with that much history, and I think that is probably worth borrowing something from it before I go completely reinventing the wheel.

Professor Bowman's observation that there aren't enough cues to guide the musician is exactly correct, and that deficiency is one of the areas on which I would like to focus.

Professor Bowman's next comment was:

"LOTS of people have already explored using the Leap for musical performance. You should check out what's been done already, and perhaps try to go in a very different direction. Or at least start with NUI principles instead of just experimenting like most of these other projects."

Absolutely true! Most of the music apps for Leap Motion that I looked at fell into a couple of categories:

  1. Apps for controlling or mixing loops and instruments, designed for DJ's or composers. Some examples are Chordion Conductor, AeroMIDI, and OscilloScoop
  2. Apps that use hand gestures to create touchless equivalents of drums / drum machines, keyboards, or other traditional instruments. Examples include Airbeats, DRUMair, and AirHarp. Musician and Leap enthusiast Hagai Davidoff recently did some wonderful tinkering to control a virtual violin with Leap-controlled "bowing."

That brings me to the issue of "prior art." I'm not the first person to think of adapting the theremin to the Leap Motion interface. AeroMIDI, which I mentioned before, features theremin-like controls in their demo video (the second screen shown). Syntheremin is explicitly designed to be a theremin app, and it features a range of settings for changing the sound. A programmer named Yu Jiang Tham built a utilitarian theremin program for the Leap in JavaScript, and I may examine the source code for ideas when I program my prototype. There are a handful of other, less polished examples to be found on the Web as well.

None of these examples, however, have accomplished my goal of a well-designed interface for a gesture-based instrument that uses scaffolding and "training wheels" to help novices gain skill. AeroMIDI is clearly aimed at professional (or aspiring-to-professional) music producers, and its on-screen interface would be overwhelming to a novice. It's definitely not geared toward rehearsal or performance. Syntheremin, judging from its screen shots, is designed for engineers more than musicians. The JavaScript theremin is a proof-of-concept more than an instrument.

There is definitely a hole for me to fill with my design if I can pull it off.