Compress|Decompress (Processing and Arduino Part)

•

Compress|Decompress is an interactive sound installation that invites a single occupant to experiment with acquiescence to an imposed but invisible virtual prison, while guided by whispers and a subtle narrative.

It is meant to serve as a counterpoint to our expectations on what interactive art can be by impressing the need for a responsive but meditative silence and stillness, while experimenting with the potential gaps and confrontational peaks in the dialogue between user and audience, user and designer, as well as user and machine.

In this project, Bryan Ma worked with the control panel and Kinect; Micheal Tortely handled MaxMSP; and I was in charge of the communication between Processing and Arduinos (LED Screens).

Inspiration

Cockatoo Island

Our inspiration came from the history of Cockatoo Island. It started with the age of convicts, so we want to represent the feeling of isolation and pressure but also allow the participant communicate with other audience in an indirect way. Our initial idea was to do something physical and sculptural, something that has a physical presence and is something to be physically investigated or negotiated.

Initial Storyboard

The initial storyboard is a quick mock-up of a potential manifestation of these physical directions. This emerged sort of spontaneously, but it’s still quite interesting that the object itself kind of evokes an island in and of itself. The reason we want to play with a little room like this is so a user can remove themselves from the public for a moment and experience something apart and separated from the rest of the space. We were also inspired by other works that involve sensory deprivation and isolation in creating a unique atmosphere and experience.

Simultaneously we wanted this separation to invoke something secondary, something of an idea of separated scrutiny – when you crawl into a little space like this, when you’re ‘taking your turn’ in an object with other people around, you’re becoming a performer in a sense – people are scrutinizing you, attempting to discern something from the appearance of your experience, waiting patiently or impatiently for their own turn – and there’s something that can be explored in this relationship between the actor and the viewer, especially when a sense of direct contact is suspended or interrupted.

We’re thinking of a two-way system here – a Kinect in the interior, as well as on the outside. as we’re not really trying to emphasize it as a technological object, we think the presence of the Kinect itself should be downplayed if possible – building it into the surface of the object itself, etc.

The object should have a physical element to its interaction – you have to squeeze inside, crawl into a little space, sit or lie down in a dark room, and feel a bit uncomfortable in a sense. This model is pretty straightforward and involves a stool and curtain.

Semi-storyboard

Interior Interaction:

The Kinect has a clear view of the user, and is able to:

1- Detect when a user is in the space

2- Detect if the user is in position

3- Detect if the user is occupying an ‘idealized’ state (going to be most likely related to movement and exact positioning, though essentially arbitrary)

This information wills the feed out to the digit displays outside the space to generate the ‘score’. It will be somewhat of a mysterious process- in that the exact rates and delays and pauses in the way the score behaves will vary somewhat from user to user, though will still reflect some basic (again rather arbitrary) behaviors captured by the Kinect.

We have some basic prototypes that use both the basic depth data as well as more sophisticated volume and point cloud tracking in 3D. We wanted to incorporate both methods but probably will only use simple depth data and not worry about rendering in 3D in processing per se. That said, this is still a tough problem for our relative inexperience in programming to crack.

Exterior Interaction:

Users are detected in the waiting space, and their positions, movement, and numbers feed into the MaxMSP algorithms, giving us our generative soundscape for the solo participant inside.

Final Settings

Our final settings abandon the idea of curtain and reduced the number of cameras, integrating all the interaction to ONE KINECT. Instead of establishing a visible isolated room, we use hotpoint class to create a virtual box covering the occupant. Each side of the box is defined by a hotpoint, which impacts on the final score and activates different sound effects in MaxMSP. There is some kind of sound story at the background, but if any part of body touches the edge of the box, the alarm noise will came from that direction. When audience enter the interaction area and sit on the chair, the box would become smaller and smaller, so the participant has to stay still or even huddle up to catch up the sound story as well as to reach the highest score.

Control Panel

LED Screens

Soldering the circuit boards and succeeded in burning the sample program into the chip to display the preset data

What I have learned
1. Reading the circuit map.
Digit Shield showed “6.6.6.6.” when the data was “8.8.8.8.”, so I read the circuit map and the LED pin map, checking the circuit point by point, then found the incomplete circuit pin 15 and fixed it.

Circuit map: http://nootropicdesign.com/digitshield/design.html
LED pin connectors: http://www.kerrywong.com/blog/wp-content/uploads/2010/04/4_7seg.png

Fixed the broken pin on the other board

2. The hardware construction of Arduino.

Before starting to use Arduino, I thought the board use COM cable as ordinary serial port (e.g. 9-pin COM or 25-pin COM) for real-time output and was worried about finding the suitable cable. Fortunately, I learned from the website (http://Arduino.cc/en/Main/ArduinoBoardUno) that Arduino has its special chip, which allows Arduino to use the usb port as the serial port.

3. Writing a program into Arduino.

For the reason that we have two different types of Arduinos, according to http://arduino.cc/en/Guide/MacOSX, I should reselect the board type and the serial port when uploading the program.

4. Using Processing to control 2 Arduinos, sending the information separately.

Serial Library in processing (http://processing.org/reference/libraries/serial/index.html) has list() class to select different serial port.

Realised using Processing to control 2 serial boards in real time

5. Serial communication between Arduino and Processing. Programming simple codes on Arduino, using serial port to receive the data and display on the LED screen.

Every 2 digits need 1 byte data in real time and there are 4 digits for every score, which means every package data of the score needs 2 bytes. Whereas the data is sent byte by byte, I had to use Processing send the data in a certain pattern - high digits byte before low digits one. The first problem occurred - the higher digits and lower digits shifted sometimes when displaying. So I added a check signal before every package of data. Due to the fact that the max value of 2 digits is 99, the check signal was set to be the value of 100. Corresponding to the order of data sending, Arduino checks the value 100 every time. If the received value is 100, then the chip on the board accepts and stores the next byte to high digits, the second to low digits; else the value of the third byte is not 100, then drop this byte. Finally the order of the digits came out right, another problem showed up - the LED flashed when counting too long. It looked like missing the check signal. Eventually, I discovered it was not caused by missing data but by overloading in buffer, so I used the while loop with detecting 3 bytes instead of for loop, which can keep reading and showing the score smoothly.

Able to send and receive 2 bytes data (4 digits) with serial port in real time

6. About the integration.

The program report showed that a serial port to Arduino was occupied when my codes were integrated into the project. At first I thought the problem was caused by different operating system, because I experimented the Arduino part on Win7 but our exhibition computer is MacOSX. Then I found that the MSP is also introduced to our project, which occupies a virtual serial port which is the same one that I use for my second board (http://arduino.cc/en/Guide/MacOSX). So I reset my board to serial port 3 and solve the problem.

7. Learned how to display two output windows at the same time.

One is the control panel, the other is the visual effect on the computer screen. But in final setting, we removed the computer screen from the table.