The most significant assignment of the semester in the Natural User Interfaces class I am taking is the interface design project. This is my initial concept proposal.

The theremin is a musical instrument that is operated without physical contact using proximity sensors. It creates an ethereal sound that was most famously used in eerie science fiction soundtracks. Its operation is similar to a NUI in that it allows a skilled user direct control over pitch and volume by moving their hands toward and away from proximity-detecting antennae. Much like the human voice, there are no frets or keys to demarcate notes -- it requires practice to instinctively glide a hand into the correct position for a desired pitch. The video below gives a nice introduction to how the theremin is played.

Though the theremin is novel and I find it extremely interesting, it does have its shortcomings. It can only play one kind of tone, and that tone is so "cold" that thereminists nearly always use heavy vibrato to make it listenable. It is limited to one note at a time, and it is difficult to determine where to place your "pitch hand" in space to produce a specific note.1

I propose to create a new musical instrument that combines the theremin's greatest strength, its natural interface, with modern technology and knowledge of interaction design. This instrument, which I am tentatively calling a "theremax," will replace the proximity-sensing antennae with a gesture detection device that can measure the positions of two hands in three dimensions.

I plan for the theremax to have the following features:

  • The musician will control pitch and amplitude (volume) by moving her hands in space. I will perform testing2 to help me determine which gestures control which attribute. The theremin controls pitch by the left hand's X-axis (horizontal) position and volume by the right hand's Y-axis (vertical) position. A different arrangement may be more effective for the theremax.
  • The musician can choose to play the theremax with different MIDI instruments and digital effects. For example, they may choose the sound of a flute. Or perhaps they will choose the classic theremin sound, but with vibrato built into the sound so they don't have to constantly vibrate their hand.
  • The theremax's sensor will be attached to a computer with a monitor, on which the musician can see a visual representation of their current "pitch hand" position. This will allow them to accurately place the pitch even before or after they begin playing it audibly.

I am not a computer scientist, so I will skip the hard work of inventing a hand-detecting device and use the commercially available Leap Motion device and SDK.
Leap Motion Controller

I don't expect that the end result of this project will be physically attractive or have a convenient form factor. What I do hope for is an interface that can be quickly picked up by a novice, engaging enough to encourage exploratory play, and sophisticated enough to play interesting music.

1. The latter two limitations apply to a number of musical instruments, and are not specific to the theremin. They are nevertheless limitations, and ones that can be addressed with modern tools.

2. For the purposes of this class project, the testing will not be scientific. If the end result is promising, I may revisit some of my conclusions and test them scientifically.