*                                                                                                                                                *
*                        For engineers that moonlight as artists and artists that moonlight as engineers                                         *
*                                                                                                                                                *
*                                                                                                                                                *
*   CCC    OOO    L      L     IIIII    SSS     IIIII    OOO    N    N                   l     l                      t      i                   *
*  C   C  O   O   L      L       I     S   S      I     O   O   NN   N                  ll    ll                      t                          *
*  C      O   O   L      L       I     S          I     O   O   N N  N    ccc     ooo    l     l     eee     ccc    ttttt    i    v   v    eee   *
*  C      O   O   L      L       I       S        I     O   O   N  N N   c   c   o   o   l     l    e   e   c   c     t      i    v   v   e   e  *
*  C      O   O   L      L       I         S      I     O   O   N   NN   c       o   o   l     l    eeeee   c         t      i    v   v   eeeee  *
*  C   C  O   O   L      L       I     S   S      I     O   O   N    N   c   c   o   o   l     l    e       c   c     t      i     v v    e      *
*   CCC    OOO    LLLLL  LLLLL IIIII    SSS     IIIII    OOO    N    N    ccc     ooo   lll   lll    eee     ccc      tt    iii     v      eee   *
*                                                                                                                                                *


Felt, paper, computer with custom software, Kinect interface, video projectors, speakers

DreamDrops is an interactive fiber/video installation where viewers can interact with felt sculptures, brought to life as a colorful, immersive video and audio environment.

Three DreamDrops are constructed with felt and paper. The felt gives them the form, and the paper provides a window into the virtual words. The Drops are suspended by their “tails”, from the ceiling at three heights. The red Drop is hung the most high, the blue Drop is lower, and the green Drop hangs the lowest.

As visitors walk among the DreamDrops, they can see the three animated worlds. The red world looks like clouds at sunset, the blue world looks like the depths of the oceans, and the green world looks like a lush, tropical forest. The three worlds are populated by “boids”, a flock/school of animated critters that dash about in small groups.Visitors are invited to poke their heads up into DreamDrops. They’ll see bursts of color and dynamic sparks triggered by their motion which is detected by a Microsoft Kinect sensor. People can interact with the boids, who will playfully investigate the human visitors, calling out with a futuristic wail. A hand clap gesture will trigger a colorful tunnel effect that will transport the visitor to other worlds.

With their installation, Rob and Kristina are exploring the threshold between personal space and public space in the technology era. The installation allows people to experience a private environment of light and color in a very public arena.

Several open source software projects were used in this installation:

  • OpenFrameworks - an open-source C++ library for creative coding
  • ofxKinectNui - Sadam Fujioka’s addon using the Microsoft Kinect
  • ofxMSAFluid - an addon for solving and drawing 2Dfluid systems by Memo Akten
  • ofxBoids - a flocking motion addon for by Satoshi Okami

Rob and Kristina would like to thank Jennifer Lim for her help with this installation.

See video

JitterBot - A Dialog in Dance

Computer with custom software, video projector, Kinect interface
Rob Gonsalves: JitterBot - A Dialog in Dance


JitterBot is an interactive installation that allows visitors to dance with a virtual robot. This piece explores the threshold of leading / following in dance, allowing the participant to have a dialog with a technological being, using their bodies as the means of communication.

Song Selection

The user chooses a song by raising an arm and pointing to one of the seven original compositions. The computer uses a motion tracking interface and custom software to choose the song.

The Dance

The user is invited to dance along with JitterBot, the red robot. The user's movement is tracked using the tracking interface and displayed on a projection surface as the green robot. Sometimes JitterBot follows the visitor's lead and sometimes JitterBot busts a move on its own, for the visitor to follow.

The Setup

The installation is centered on using the Microsoft Kinect sensor which allows for real-time full-body 3d motion capture. A layer of custom software interprets the 3d skeleton of the viewer and renders a stylized human form, the green robot. The movement of the red robot is driven partially by the viewer's motion, but it is also driven by a library of canned moves to give a sense of autonomy. The two robots are rendered as a series of elongated spheres using OpenGL. The resultant video image is projected on a screen on the wall, and the accompanying music is played through speakers.

More info is available here:

User login
Latest Blob Posts
Syndicate content