Interactive Art Installation Connecting the Physical and Digital Spaces with Sound

digiPop is an interactive game art that seamlessly connects the physical and digital spaces. The concept is to blow virtual bubbles into the digital canvas and is inspired by Mark Weiser's vision of a new paradigm for human-computer interaction [1] where technologies "weave themselves into the fabric of everyday life until they are indistinguishable from it". The bubbles straddle the physical and digital worlds with sound as the mediator between the two spaces.

The gameplay follows the way children play with bubbles. Through a built-in microphone, people can blow differently shaped virtual bubbles into the digital canvas based on the frequency of their voice while also interacting with them through body gestures captured by a camera. The velocity and shape of the bubbles as well as the rate at which they are created depend on the volume (i.e. pressure) and the peak sound frequency at which they are blown. For example, blowing into the microphone creates high pressure, allowing players to create many bubbles quickly. By whistling and singing at different sound frequencies, they can affect the velocities of digital bubbles blown towards one side of the screen. Fast moving bubbles are generated with high frequency sounds while slower bubbles are created with lower frequencies. Once the bubbles are in the virtual space, players can interact with them through gestures. They can combine them into bigger bubbles, pop other players' bubbles, maneuver bubbles around obstacles, all by using any kinesthetic interactions imaginable. Although existing in virtual space, these bubbles loosely follow the laws of conservation of mass and momentum to signify their connection to the physical world. Digital computation is used to allow for augmented interaction that cannot be otherwise accomplished in our physical space.

Each digiPop bubble remembers the frequency used to create it that is then played back as it moves across the screen. A high A note (440 Hz) is played when bubbles collide into each other while a pop sound plays when a bubble bursts. These sounds create a feedback loop into the bubble formation, accompanying the player's voice for a unique concerto.

GSD6338 Introduction to Computational Design | 2014 - 2015
Harvard Graduate School of Design
Instructor: Panagiotis Michalatos

Programming LanguagesC#

Awards + ExhibitionMpls Center for Digital Art, Inter:Connected Exhibit
MIT Museum, Shortlisted

Future developments digiPop's future plans include art installations and a mobile app using APIs developed in NodeJS and MySQL.


Intuitive Interactions: Voice through Microphone + Body Gestures

The gameplay follows the way kids play with bubbles. Computational design augments the experience, allowing for interactions that are only possible in the virtual world

Blowing Virtual Bubbles

Bubble is the mediator between the physical and virtual spaces, traveling the two worlds through sound

Exhibition at Mpls Center for Digital Art

Human Computer Interaction Research

Following the notion of ubiquitous computing, the setup only requires a webcam and a microphone, which are available on most computers, so that anyone can play the game anytime, anywhere. The intent of the project was to make a fun and accessible computer-based game for people of all ages. The goal was to move away from mouse-based interaction and take inputs analogous to natural actions children make when playing with bubbles. The low-res display encourages people of all ages to become more playful and expressive with their movements. Not only is the intuitive interface essential for human-computer interaction but it also invites interaction between friends and strangers alike.

[1] Mark Weiser. "The Computer for the 21st Century". Scientific American, September 1991.