The focus of this project is to enable the translation of sound input into visual output, recalling the theories of Goethe and Kandinsky about Synestesia:

“[A] new synthetic experience of art where the material distinctions between word, image and sound would melt away into a kind of spiritual [...] ecstasy that would shake the body and the world.” (McBurney, 2006) Music became paintings and paintings became music. (cf. McBurney, 2006)

“Color is life, for a world without color seems dead.

As a flame produces light, light produces color.

As intonation lends color to the spoken word,

Color lends spiritually realized sound to a form.”  (Itten, 1997, p.8)

Sonic Painting is designed to arouse creativity by using sounds generated directly from the participants. Their body represents a brush and their voice will be the colored bristle on the surface. Coherently with the sound, a colored pattern will be projected onto the floor exactly where the visitor produces the sound. As a result, the sound emission will be perceived visually from the visitors on the ground where they have been standing/passing.

Therefore, the Sonic Painting lets them play with their inner sense of presence in time and space which remarks this constant attendance by reflecting infinitely and constantly on the interaction with “invisible matters”: air, sound and color.

This constant translation from invisible to visible from here into now, and vice versa, triggers an atavic creative engagement.

The audience will be experimenting with their conception and connection of time and space leading themselves through the discovery of action, reaction and creation.

The audience shall walk around on a white carpet/canvas surface emitting sounds by using their voices, singing, playing musical instruments, simply clapping their hands, or any other ways they find to create noises. The voice or the noise of participants, detected by microphones displayed around the room will be converted into a patterned and sized color, which will go through all of the gradient chromatic scale depending on sound pitch and harmonics.

Saturation = Tone-timber

(Average tone high, not full spectrum for reliable outcome)

Clear Tone = Saturation

Volume = Size

System

The system consists of a matrix of directional microphones hung from the ceiling, a motion sensing input device (Kinect), a computer for sound/visual processing, a ceiling mounted projector, and a large white flat surface onto which the processed image is displayed by a projector.

The location of the sound and the source is detected by the Kinect and comparing amplitudes between all microphones; spatial and sound information is first processed through the visual programming software Max msp then the software for integrated development environment Processing. Ultimately, the object is projected onto the floor - more precisely, where the participant is standing.

Pitch and amplitude of sounds from the microphones are translated visually into colors. The pitch range is mapped to the visible color spectrum. For example, higher pitches are allocated to yellow, lower to blue, and mids to the intermediates colors. Amplitude, or the loudness of the sound source, is assigned to the size of the objects. Louder sounds draw larger objects while smaller sounds make smaller ones.

References

(McBurney, 2006) McBurney, Gerard: Wassily Kandinsky: the painter of sound and vision. https://www.theguardian.com/artanddesign/2006/jun/24/art.art (last visited: 2017).

(Itten, 1997) Itten, Johannes: The Art of Color : The Subjective Experience and Objective Rationale of Color.