This was a university project for my module Data and Machine Learning. The brief required I use PoseNet creatively and follow the theme "Counter-Action". PoseNet is a deep learning TensorFlow model that allows you to estimate and track human poses by detecting body parts such as elbows, hips, wrists, knees, and ankles. It uses the joints of these body parts to determine body postures. I thought it would be interesting to move the PoseNet skeleton away from the webcam feed to create a mirror effect that might fit thematically. I decided to assign character to the double by inventing a narrative around the sketch that imagined an intergalactic Cultural Exchange Program where humans and aliens might learn to communicate with one another through sign language, as well as drawing ears, eyes, and antennas to specific points on the PoseNet skeleton to add an element of personality to the extraterrestrial creature. I was interested in using sign language, firstly because I’ve always wanted to learn it, and secondly because I am fascinated by the possibilities that this technology might hold for accessibility of this kind and wanted to see how feasible it might be. For the initial submission, I trained my model on three words (‘you’, ‘home’, ‘now’) and designed a very basic interface to visualise the words correctly classified. I also decided to implement sound: I edited some electronic extraterrestrial sounding snippets in Logic Pro X, splitting them into three potential alien words, and imported them in the sketch, assigning them to the specific classified poses. I also took the opportunity to add a sound visualiser responding to the alien sounds at the bottom of the page, to create the bridge of two-way learning that the narrative sought to convey (the alien learning sign language, the human learning alien language).
FollowFollowing
Appreciate
you HOME now - PoseNet
you HOME now - PoseNet
Published:
you HOME now - PoseNet
0
Published:
