STIMMMALER is an interactive installation: Visitors paint a picture with their voice. Pitch and volume define color and look of a virtual brush.
It all started as a semester project and was chosen by the german Institute for Media Design for presentation on a regional science fair. Since then it is presented at various art and media festivals.
General concept: Visitors paint onto a projected canvas by speaking into an augmented megaphone and physically moving it.
The general concept is that of leveraging the human voice as input to paint onto a projected, large-scale canvas. The characteristics of the voice are used to control the expressiveness of the paintbrush. An augmented megaphone captures the voice. The visitor can move it in physical space to control the position of the brush. The expressiveness of the brush depends on two features: the pitch and the volume of the voice.
Pitch: The pitch of the voice determines the color of the brush. The actual pitch frequency is hence mapped to the hue scale.
Design of the megaphone: A wireless microphone at the mouthpiece, a Wiimote at the open end. Simple as that.
Volume: The stroke of the brush is modeled as 14 individual pixels. The volume is mapped to the spacing of the pixels respectively and im- pacts the thickness (or density) of the stroke.
The exhibit is designed to provide a seamless and also serendipitous experience: a visitor walks up to the augmented megaphone and grabs it. The brush appears and the visitor can interact. Once the visitor finished a painting, a printout is generated. The print serves as a tangible representation of the painting that transcends the interactive experience. The canvas can then be cleared to start a new painting.
The overall system is implemented using VVVV, an open-source node-based programming tool.
Download paper (PDF, 7.6 MB): stimmmaler-cr-2014-02
STIMMMALER is a cooperation of:
Institut für Mediengestaltung, Hochschule Mainz
Mainz, Germany
Singapore University of Technology and Design
Singapore
MIT Media Lab
USA
Idea and concept: Benjamin Böhm
Technical Realisation: Julian Hermann
Design Consulting: Tim Rizzo
Producer IMG: Manfred Liedtke
Art Direction: Prof. Anja Stöffler, Jochen Huber
STIMMMALER 