It is my personal belief that art requires an audience. If art means something to somebody then I believe it has fulfilled its purpose. It may inspire, repel or cause controversy but it will evoke something in someone. Put like that, one could say art may be incomplete without an audience? What if we view that thought literally?
Interactive digital installations seek to make the audience part of the art itself. They unfold like a performance. They create connections. I always imagine interacting with a digital installation as taking part in a dream. Something unreal is taking place and the outcome is magical, whether beautiful or terrifying.
Before I even started this MA I remember watching Shilo Shiv Suleman’s TED talk. She embodied nearly all my aspirations; she started the ‘Fearless Collective’ public art movement and she created the most gorgeous, ethereal digital installations for people to interact with. She believed in the fusion of mythology and technology. My approach isn’t exactly that, but there was so much in her talk to be inspired by!
So you could say, creating an interactive digital installation was one of those dreams that you can’t really envision ever coming to fruition. Then I found this MA and it gave me a bit of courage to try and learn something new. I decided to create a simple interactive installation for “The Untold Edition”.
My idea
Imagine if people walked into a room to see the flickering blue light of a projector screen on the wall. They see a mic facing them and a few instructions. They follow the instructions to talk about their memories of Karachi and as they speak a painting begins to appear on the screen. When they stop speaking the painting pauses as well. The next person then steps forward to speak and continue the painting process, and together a group of people build the story.
How?
I just realized I wrote a short blogpost on this before when my brain gears were jammed with code. Basically, I am using an Elechouse VR3 voice recognition module connected to an arduino (thank you Terry Quin!) to control the voice commands. The only problem was the arduino doesn’t have the capacity to play videos off an SD card and it has no directory to play videos through a computer. It needs to be connected to yet another program, enter: processing!
Last time I wrote on this subject I had basically managed to program my VR3 through my arduino, and write code for both arduino and processing however there were some errors that I was unable to fix. The problem ultimately was that, not being an experienced coder, it was taking me days to figure out my errors and I was making so many syntax errors as well. I tried reaching out to random people I knew through other people but literally NO ONE had heard of arduino let alone processing. Frustration compounded!
Then, one day, I visited an old friend who teaches engineering at King AbdulAziz University. She was frantically preparing for an exam for her students the next day. She was building a giant wooden track of some sort. Apparently, as an end of year project she always asks her students to build robots that are able to navigate a track with hurdles. The tracks or the hurdles themselves are not revealed to the students and they have to make their robot quite capable to be part of the project. Something stirred but didn’t quite click in my brain just then.
The moment I got home from her place I literally face-palmed. I messaged her whether her students use Arduino to program those robots and she said YES! So I asked her to connect me to one of her students. Enter: Ruba bin Jabal.
Ruba had never heard of processing but she was quite experienced in code, so she was able to expertly debug my code. Now the challenge was, she lived over an hour away and was busy with school. Not to mention it is Ramadan and the timings of everyone and everything are completely out of whack. So every time she corrected code she had no way of checking it out! She would need to send it to me so I can reprogram every thing, replace some file extensions and try. It was tedious and literally all done over voice messages on whatsapp. But it was also exhilarating to find ourselves inching closer and closer to our desired outcome.
Here are the final codes we wrote. They still need to elaborated a bit to add some more voice commands but they work!
Arduino
#include <SoftwareSerial.h>
#include “VoiceRecognitionV3.h”VR myVR(2,3);
uint8_t records[7]; // save record
uint8_t buf[64];#define karachi (0)
#define stop (1)
int com = 0; //reply from VRvoid setup() {
myVR.begin(9600);
Serial.begin(115200);
//Serial.println(“Elechouse Voice Recognition”);// if(myVR.load((uint8_t)Play) >= 0){
// Serial.println(“onRecord loaded”);
// }
//
// if(myVR.load((uint8_t)Pause) >= 0){
// Serial.println(“offRecord loaded”);
// }}
void loop() {
int ret;
ret =myVR.recognize(buf, 50);
if (ret>0){
switch(buf[1]){
case karachi:
Serial.write(1);
break;
case stop:
Serial.write(2);
break;
default:
Serial.println(“Record function undefined”);
break;
}
}
}
Processing
import processing.serial.*;
import processing.video.*;
String PATH = “/Users/kehkashankhalid/Desktop/totoro.mp4”;
Movie mov;Serial myPort; // Create object from Serial class
int val; // Data received from the serial portvoid setup()
{
String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port
myPort = new Serial(this, “/dev/cu.usbmodem1421”, 115200);
size(384, 750);
frameRate(30);
mov = new Movie(this, PATH);}
void draw()
{
image(mov, 0, 0, width, height);
if ( myPort.available() > 0)
{ // If data is available,
val = myPort.read(); // read it and store it in val
}
println(val); //print it out in the consoleswitch(val){
case 1:
mov.loop();
case 2:
mov.pause();
}
}void movieEvent(Movie m) {
m.read();
}
I know it’s a really simple interactive installation but for me, novice coder and first attempt-er, it’s a big deal! YAY!
Next steps?
Creating the animation. I’ve been working on it for a while. I tried to use blender to auto-animate but I just wasn’t dexterous enough with that program so I reverted back to photoshop. I’m trying to create a watercolor style collage of Karachi landscapes and the way I’m animating it is by saving .png files repeatedly as I work and I’m going to compile them into a stop motion animation on a simple program like imovie.