Keyword
multimedia system, Slackline, improvisation, Kinect, generative art
Year
2015
Performance
- Slackline: Son Insoo
- Percussion: Kim Hoin
- Bass: Ahn Wonsuk
Interactive Multimedia System controlled by Slacklining (with focus on Multimedia Music <Bassline>)
This study invented an interactive multimedia system controlled by Slacklining. Slacklining is an act of balance, walk, and jumping on webbing called Slackline tensioned between two anchor points. This system consists of three parts, motion tracking interface, sound effector, and video system.
There have been various motion tracking strategies already, such as color tracking, face tracking, full-body tracking, and so forth. However, these strategies are not suitable for Slacklining. This is because an acrobat keeps changing his or her posture when performing Slacklining. In this situation, a tracking algorithm used for motion tracking could quickly lose its target. Thus, this study has invented a motion tracking which can keep track of an acrobat's position regardless of any posture using Microsoft Kinect.
A sound effector has been made for the instrument used during Slacklining performance. This effector uses a real-time pitch-shift strategy to give a vibrato effect to the tool. When the sound effector receives the sound signal from the device, it first detects the fundamental frequency of the note, then filters out the audio signal which is below a selected overtone's frequency. After that, the effector gives the pitch-shift effect to the audio signal using the data of the motion tracking interface.
The video system operates based on the physics engine to convert physical movements of Slacklining into visual images. The system consists of particles and invisible forces that push or attract particles. The motion tracking interface data is mapped into forces' position, which attracts particles so that the particles follow the acrobat. A rhythm instrument also controls the video effect by pushing particles out using its dynamics. This video system also supports envelope following to detect instrument dynamics.
This study has tested the effects of this interactive multimedia system by using it during a Slacklining performance. The music style of this performance was jazz improvisation between a contrabass and Korean drums. At the performance, the interactive multimedia system created a synergy effect with the music style. Thanks to the system, the acrobat was able to participate in the improvisation between jazz players. Besides, sound effectors and the video images also joined in the interaction so that the performance was able to give a new impression of Slacklining to the audience.
