415-939-3151 grahamsplumb@gmail.com

The Nemours Children’s Hospital : Living Garden

Creative Director and Interactive Exhibit Designer
Snibbe Interactive

2012 – 2014

Installed in a new children’s hospital, this 60’ long interactive wall is the largest Kinect camera installation in North America. Designed to entertain children of all ages, the fantasy landscape responds to gesture-based interaction, encouraging kids to move their bodies in an immersive social experience.

Presented with the challenge of designing an experience that always felt different, we developed the idea of a game in which children could genetically modify their own plants, so every interaction enhanced and evolved the visual display. Children are able to nurture their plants with virtual water and interactive sunshine, bringing a sense of life and vitality to the scene. Later on in the game, participants are transformed into flying avatars, releasing fruits from the newly grown plants, feeding hungry creatures that emerge from the undergrowth. The wall changes appearance throughout the day, with shifting hues caused by the movement of a virtual sun and realistic weather effects. Then at dusk, all the plants change color, twinkling bioluminescent hues as people’s silhouettes become sources of light, illuminating the magical scene.

The Proposal

Our initial strategy was to create a rich visual narrative using collaged elements taken from 19th Century illustrations. Inspiration came from the stunning drawings of Ernst Haeckel, a 19th Century naturalist (and biologist, philosopher, physician, artist and professor). These richly observed and detailed renderings of scientific wonder felt perfectly suited to the context – a scientific institution with a need to provoke wonder in their patients. Our graphic designers artfully dissected these beautifully observed forms, rearranging them into fantastical plants and creatures. Spontaneous bursts of laughter in the studio would signal the creation of another wonder, which would then have to be named.

Interactive Narrative

One of the creative challenges of the Kinect camera is that the participant, or ‘interactive character ‘, always feels fixed within a scene. It’s hard to use the language of cinema when there’s no exit stage left or right. This is why we developed the idea of interactive characters that could propel the narrative along themselves – autonomous animations with just enough AI to have a personality and show motivational behavior. The Kinect cameras are installed below the display, pointing at the players. By using the Kinect to count how many people are engaged at any one time, we can help steer the course of the story, altering the quality of the interactivity to better suit the number of participants.

 

Video : Making the Interactive Wall

 

Motivating Physical Therapy

The Living Garden was also designed to enhance the hospital’s physical therapy program, helping motivate children to move their bodies. We worked with physical and occupational therapists to understand what kind of movements should be encouraged, and where others should be avoided. We achieved full accessibility by testing the experience thoroughly in our studio. The Kinect camera is a surprisingly robust platform, able to pick up sitting and standing people, and usually does a good job tracking bodies as they pass in front of one another.

The Creature Creation Workshop

One of the most rewarding aspects of the design process was the ‘Creature Creation Workshop’, in which children from the hospital’s Youth Advisory Council designed their own characters, describing their behavior and imagining their personalities. We learned that the children wanted to see creatures that reflected themselves, which might be happy and extroverted, but also moody and reclusive at other times. These profiles inspired a broad range of flying and ground-dwelling creatures, brought to life as animated 3D characters.    

The Chromakin

In order to create an extended interactive narrative, we used the Kinect cameras to observe participants, choosing our moment to launch certain effects. We used a variety of characters to initiate these effects, so people understood why they were happening and what the consequences might be. At a certain stages in the game, the flying ‘Chromakin’ will appear, using his wand to cast a spell on people’s silhouettes, transforming them into flying avatars.  

Controlling a Flying Creature

Once a person’s silhouette has been transformed into a flying creature, they take control using their upper body. The Kinect camera can see standing and seated children, so everyone is able to participate. Children use their avatar to release fruits hanging above their heads. These drop through the air, and a series of hungry creatures emerge from the plants to quickly gobble them up.    

Combining Interactivity with Animation

Traditional animation is a linear experience, with a defined start, middle and end. The interactive animations in this installation are different, made up from short animations that are triggered by user interactions. This prototype model shows a flying creature that is programmed to follow a ball. Clicking on the screen changes the position of the ball, and the flying creature follows behind. The entire wall is based on this principle of combining pre-defined animations with open-ended interactions, enabling us to craft believable characters that feel alive to the user.

Interactive Creature in a Scene

Once the character has been placed in a scene it starts to look much more convincing. The ball has been made invisible, but this flying creature is still responding to it’s movements. It’s a relatively easy step to replace the ball with the position of someone’s head, as seen through the ‘eye’ of the Kinect camera. This would make it feel like the creature is following them as they walk in front of the wall.

Animating Plant Growth

The plants needed to grow rapidly, so they felt responsive to being fed water and sunshine. The hard part was figuring out which aspects of the plant’s growth should be animated versus algorithmically driven. Hand-animation provides more control, potentially producing more character. The problem is that it will only work for a particular plant, and we wanted hundreds, even thousands of varieties. This hand-made animation was used to start prototyping the parameters we needed in the algorithmically driven plants. In other words, we started with the ‘look and feel’, and worked backwards into the engineering.

Algorithmically Driven Plant Growth

This algorithmically driven model has about 30 parameters, enabling a wide variety of plant forms to be created from the same code. For example, we can control the diameter of a branch, the number of branches, how much each branch twists, how many leaves appear, where the leaves appear, etc. When users combine seeds from different plants, the ‘child’ seed embodies parameters inherited from both parents, resulting in a new plant that looks like its parents.