Interval Player
Interval Player is a Virtual Musical Instrument (VMI) designed for the 3DUI 2015 competition. The requirements stated that the instrument should be able to play notes and chords. It is possible view an instrument as a device that, besides generating the sound, allows the musician to efficiently select notes from a vast array of options. This is a very complex task since it has to be done with high accuracy and speed while allowing multiple selections. Much of this is possible due to the evolution of the instruments over the time and years of practice.
After considering many options we choose to build a instrument that could leverage its virtuality to simplify the selection space. The Interval Player uses the fact that the sequence of notes in a music are never completely uncorrelated. Melodic intervals often follows a Zipfs distribution, with longer intervals will be less frequent than smaller intervals [7]. We also used the fact that popular and many classical music scores are specified in a key that contains the most used notes and progressions for that piece.
The interface shows a sequence of note from the music key. The center note corresponds to the last played note
To play the note, the user tilt the fingers to left or right and then bring them back to the center. The side of the initial tilt specifies whether a higher or lower note will be selected. The number of open fingers indicate the distance, in whole steps, between the last played note (the one in the center) and the next one. The selected note sounds when it is brought back to the center position. To play the same note again, a downward striking gesture is used instead.
Our implementation used the Leap Motion sensor, a NUl device that is designed for bare hand and finger tracking. Due to the way gestures are detected in the Leap, frequent changes in the number of extended fingers generates a lot of discrepancy between the Leap model and the real hand. This would sometimes cause the interval to be higher or lower than the one intended by the user, generating some disorientation. Chords can be played by using the non dominant hand to select from pre-configured options. The user can also opt to synchronize the chords to the melody in the beginning.
With good tracking the design is quite usable as you can see in the video below:
After considering many options we choose to build a instrument that could leverage its virtuality to simplify the selection space. The Interval Player uses the fact that the sequence of notes in a music are never completely uncorrelated. Melodic intervals often follows a Zipfs distribution, with longer intervals will be less frequent than smaller intervals [7]. We also used the fact that popular and many classical music scores are specified in a key that contains the most used notes and progressions for that piece.
The interface shows a sequence of note from the music key. The center note corresponds to the last played note
To play the note, the user tilt the fingers to left or right and then bring them back to the center. The side of the initial tilt specifies whether a higher or lower note will be selected. The number of open fingers indicate the distance, in whole steps, between the last played note (the one in the center) and the next one. The selected note sounds when it is brought back to the center position. To play the same note again, a downward striking gesture is used instead.
Our implementation used the Leap Motion sensor, a NUl device that is designed for bare hand and finger tracking. Due to the way gestures are detected in the Leap, frequent changes in the number of extended fingers generates a lot of discrepancy between the Leap model and the real hand. This would sometimes cause the interval to be higher or lower than the one intended by the user, generating some disorientation. Chords can be played by using the non dominant hand to select from pre-configured options. The user can also opt to synchronize the chords to the melody in the beginning.
With good tracking the design is quite usable as you can see in the video below: