The Upside Down V.2

Sketch: https://editor.p5js.org/ryrotella/sketches/8JmpBweFV

Concept

As noted in the previous post about this assignment,  I wanted to make a scary alternate reality of a live video with ml5.js, showing an “Upside Down” world parallel to ours. This is an homage to the Netflix hit series, Stranger Things. I wanted the altered video to feel ominous and surreal.

Breakdown

I also wanted to add wave art (see below) that was generated off a user’s movements.

I accomplished this by first focusing on coding the wave in a separate sketch (link). At first, I managed to create a group of lines that all moved in the same way at the same time. I thought that was boring. I wanted to find a way to get each line to move in a different direction, creating more engaging movement.

Eventually, I used the power of Classes. I made a Waveline class. I defined the Waveline as a very thin rectangle that moved side to side with a defined speed (random for each new object). Then, I set a variable to store the number of lines I wanted to generate for the wave. Then, I used a for loop to draw the number of lines, each with a different horizontal speed, creating the wave effect.

After drawing the wave, I integrated the wave code with the ml5.js code. I picked each wrist (pose.left_wrist.confidence and pose.right_wrist.confidence) as triggering a wave animation in the Upside Down world. Simple enough.

But I wanted to add more. I wanted to make a separate animation when both wrists come together, such as when you clap. I wanted to make a big booming wave effect, like a stone being dropped in water, as if our world was affecting the Upside Down one. After searching online, I found a p5 sketch that already accomplished the effect I wanted. Thank you to Sara Almulla for this wonderful effect, which I incorporated into my sketch. Sara’s code references a tutorial from Happy Coding, where most of the code is originally from.

I integrated this code into the sketch when both wrists are visible to PoseNet. But I wanted to do even more. So I added a sound effect for the right wrist, left wrist, and both wrists. The result satisfies the immersion of dealing with an alternate reality of your own image.