Lucid Possession, a multimedia performance, premiered at Roulette in Brooklyn, and combines musicians, VJ mashing, and stage-controlled robotic projection screens to present a contemporary ghost story – a poetic musing on managing the mass of information “noise.” Todd Reynolds live scores and composed the finale. Hai-Ting Chinn stars in the video and sings onstage. I run all cues from the stage and scrub video across multiple screens. Elliott Sharp composed the song cycle. The technologies of Lucid led me to a new installation project.

Lucid Possession and The Dress That Eats Souls will be part of a retrospective on my interactive work to be presented in early 2018 at The Ringling Museum in Sarasota, Fla.

First a little on Lucid Possession. The central character, Bean, is a young artist who designs virtual personalities and is plagued by ghosts. Her mind is like a live Twitter feed that “picks up people” …but without technology. She creates an avatar, an exaggerated alter ego that goes viral on the Internet and makes her a minor celebrity. People stop her on the street. They want something, and she isn’t sure what it is. Anxiety exponentially increases her paranormal sensitivities, and a ghost from the past emerges from the noise.

I was thinking about the way we live in analog and virtual worlds simultaneously – how we live an augmented reality through social networks and online connections that merge with our life in the physical world. It’s ordinary – but fantastic. A TV remote, social networks, ATM machines – they’re like pedestrian spoonbending. With telepresent agency – our bodies extend beyond their edges. Our boundaries blur. I was also thinking about how this landscape has turned us all into performers grasping for attention in a sea of market share. Everything becomes a popularity contest. How many likes for your cat video? Here’s a clip from the premiere.

Making Lucid Possession involved creating a complex intranet – an engine of linked technologies with many elements that included the production of LED costumes and robotic screens. The expansion of my technical vocabulary combined with further experiments with embodied interface inspired me to focus more closely on some of the possibilities these elements presented.

Building Lucid’s robotic screen: from sketches, to construction, to the projection of Bean’s Avatar. The arms and legs are selectable video loops scrubbed in real time. The head uses a neural net and vocal analysis to lip synch live to a performer. The robotic screen is controlled by motion sensing onstage or by OSC on an iPad.

The costumes were constructed with LEDs. This was before the current explosion of Raspberry Pis and micro controllers, conductive thread and flexible electronics for wearables. We had a guy with a soldering gun running around after our actors during the film shoot. In 5 years there’s been enormous advances in wearable technologies and the availability for easy use is everywhere. Karen Young: Costume design, Leif Krinkle: technology and controls.

I began thinking about the explosion of technologized wearables. It seemed to be dominated on many fronts by the fusion of narcissism and commodification. A new installation project started to percolate. The first sketch:

THE DRESS THAT EATS SOULS: An Interactive Robotic Dress Installation

progress, n. a forward movement: an advance: a continuation: an advance to something better or higher in development: a gain in proficiency: a course: a passage from place to place: a procession: a journey of state: a circuit. – v.i. progress, to go forward: to make progress: to go on, continue: to go in progress, travel in state: to go.Chambers 20th Century Dictionary

Xenophanes: “The gods did not reveal to men all things in the beginning, but men through their own search find in the course of time that which is better.”

Or not:

The Dress has a 14′ layered scrim skirt that acts as a projection screen and a cinemascope rear projection screen that hangs overhead. The robotics of the bodice and skirt are controlled by a kinect gaming interface. As a viewer standing in front of the Dress moves, the Dress mirrors their movement. It behaves as if you are wearing it. The Dress speaks to you. It’s rather chilly – like a lizard. It’s the combined human agendas that drive technology. The installation cycles between the Dress speaking and POV experiences on the overhead screen and skirt that put you inside the minds of the people who have worn the Dress as it evolved – 200 years of the human body shaped, molded and colonized by technology. Viewer head movement navigates the cinema space.

So I’ve started to build and script it. Brooklyn Research is working on the Robotics. Rene Steinke, the novelist, is working with me on the voices. Tommy Martinez is programming. Paul Geluso is designing the spatialized sound. Ben Light is prototyping and constructing elements of the Dress design. Leif Krinkle is technical director. Karen Young is helping with costume construction. Andy Dintenfass and Art Jones are shooting the video material with me. And many thanks to students from Parsons Design and Technology for their labor and brainstorming. Here’s some of the prototyping and building process:

First: lots of testing and discarding elements to discover the best way forward. Finally: building the robotic screen system. Robotics building and testing is going on at Brooklyn Research. The first Kinect test with Johnny Lu. Then the more evolved version – a serious robot!

We’re simultaneously building the skirt and bodice elements, testing media and programming navigation on a small mockup of the installation at the studio. This video shows Tommy using the kinect. The white cube in the first section shows how the Kinect sees and tracks his head movement. The second section shows how he navigates between 5 video streams with head motion.

Stay tuned for further developments as The Dress That Eats Souls evolves!

Toni Dove’s LUCID POSSESSION

A co-production of Issue Project Room, HERE and Roulette, premieres April 25, 26, 27, 2013 at 8pm at Roulette. Buy Tickets.

Photo: Melissa C. O’Brien

Batten down the hatches for a wild ride with robotic screens, live mix video, Todd Reynold’s live score on digital violin, Hai-Ting Chinn on vocals and sampling, songs by Elliott Sharp, with Toni Dove using real time motion sensing to control characters and animate video in real time!

It’s a crazy three dimensional video pop up book with performers playing the automated stage machine like an instrument. Technical Directors Ed Bear and Matt Tennie improvise on robotics and control an 8 channel sound system that combines improvisation and pre-recorded sound. Software design by R. Luke DuBois. Robotics by Leif Krinkle and Ed Bear. Staging co-directed by Bob McGrath. Costume design by Karen Young.

Hai-Ting Chinn stars as Bean, the main character onscreen, and performs onstage as Bean’s Avatar and the show vocalist. Bean is a designer of smart personalities. She’s created something…or somebody? That’s gone viral. And the noise, the noise is really getting to her. You’ll have to come to the show to figure out what that means. And even then…

“She Won’t Know”: Music by Elliott Sharp, lyrics by Elliott Sharp and Toni Dove, vocals by Hai-Ting Chinn.

Todd Reynolds with his amazing digital violin system “plays” the movie creating a live score. An array of samples, machines, all manner of digital noise fuse with amplified violin to create a dynamic 8 channel sound environment. Additional sound design by Medianoise.

Photo: Lynn Lane

Toni Dove on show controls and live mix video uses an innovative motion sensing software interface. She animates and inhabits characters across multiple screens using body movement, triggers lights, sound, and robotics and manages the show control system that orchestrates and synchs a complex system of cues with the live performers actions onstage.

Photo: Melissa C. O’Brien

Co-starring Bora Yoon onscreen as Theo, the best friend with a twist…

Photo: Melissa C. O’Brien

…and Andrew Schneider onscreen as Kal, the inner hardware geek you wish you had.

Photo: Melissa C. O’Brien

And of course there’s a ghost in the machine. Arathusa, a spirit from the past traveling across time in the same space on some strange electricity tapped by Bean’s paranormal programming skills. Just another day downtown in NYC and a new take on augmented reality – and this time it’s not just a floater stuck in your eye. Arathusa is played onscreen by Wendy Vierow with VO by Carolyn McCormick. Sound spatialization by Paul Geluso.

Don’t miss us at Roulette! It’s going to be a lot of fun.

Matt Tennie and Ed Bear squash some bugs to make this a more perfect experience for you.
They mean business. Come and see.

Bustlelamp Productions Lab had a great time recently visiting with Flora Lichtman from NPR’s Science Friday. A video from the visit is online on the Science Friday site and you can listen to the radio show from Feb 15, 2013.

Performance skirt worn by Hai-Ting Chinn                                         photo: Melissa O’Brien

Bustlelamp is preparing to launch Toni Dove’s Lucid Possession, our new live mix cinema music performance. We’ll be going to Virginia Tech in March to do 5 preview performances March 16-19 2013. Come and see us if you’re in the area. We will also be Premiering in NYC at Roulette in Brooklyn April 25, 26 and 27. Co-produced by Issue Project Room, HERE and Roulette. If you’re in the NY area come and see us there. We’re very excited about the show. Performers: Hai-Ting Chinn, Toni Dove, Todd Reynolds (musical direction), tech direction by Ed Bear and Matt Tennie, co-director of staging Bob McGrath, songs composed by Elliott Sharp, software design R. Luke DuBois.

Say hello to the petbot. A virtual robot that responds to a human voice (vo by Simon Jones) and to Toni’s movement onstage via video motion sensing – this version is a straight animation as it is interactive in performance. We also have video characters with human faces that lip synch to a performer’s voice using vocal analysis to trigger video visemes – small video loops that represent all the phonemes of speech. They learn to recognize a performer’s voice and get better at doing it the more you use them. Luke says they’re dumb neural nets – they’re smart, but not very. Look at the last work in progress video on the Lucid Possession site to see Bean’s Avatar in action.

The costumes for Lucid were created with costume designer Karen Young and the LED savvy of Leif Krinkle. It’s just the beginning. We will be creating new characters – each with their own costume that manifests personality using LEDs and robotic movement. They will appear in upcoming episodes from the Lucid story world. We’re chewing on how to make it available to you. Maybe an App?

Hai-Ting Chinn and supporting cast Andrew Schneider and Bora Yoon.  photo: Melissa O’Brien

Bean, our heroine (played by Hai-Ting Chinn), is an artist who designs virtual personalities. She  is plagued by ghosts. Her mind is like a live Twitter feed that “picks up people”…but without technology. People stop her on the street. They want something, and she isn’t sure what it is. The anxiety exponentially increases her paranormal sensitivities, and a ghost from the past emerges from the noise…here’s a video scene from Lucid Possession.

The show combines cinema with live music, robotic screens and video motion animated in real time!  Below, Ed Bear, one of our engineers, is working on the robotics of a three-dimensional projection screen at the Bustlelamp lab. Leif Krinkle and Karen Young complete the design and fabrication team.

Below is another screen – a box in which Bean sleeps. It rises from the floor using a computer-controlled motor system. Everything is controlled by networked laptops onstage and triggered by motion, sound or keyboard and iPad commands. The performance is part improvisation, part automation.

Bean sleeps in the net – recharging her battery. Simultaneous multiple realities. Lucid Possession plays with our continual navigations of the real and the virtual. We all live an Augmented Reality, one much more subtle and complex than current AR technology.

The ghost screen has robotics that close up like an umbrella, open, shake and allow the screen to be controlled by video motion sensing onstage by Toni or with an iPad  by Matt Tennie, a technical director at Bustlelamp,. Multiple image loops projected onto the screen are controlled with motion sensing. The idea is to make the dimensional characters seem to breath and become animate.

Here’s a corner of the Bustlelamp Lab that became part of the set for the Lucid Possession shoot. We like it so we work in it this way. Another form of AR?

LUCID POSSESSION is a live mix cinema performance. Musicians, a video DJ, and stage-controlled robotic screens combine to present a contemporary ghost story – a poetic musing on managing the mass of online information “noise”. Lucid Possession draws the audience into a world  in  which video characters come to life: the wave of a hand moves a video body, and video characters lip synch live to a singer. The players onstage collectively perform the movie, which spills off the dynamic, dimensional screens onto the stage. The result is like a complex three-dimensional, automated video pop-up book, and as characters are brought to life through motion, voice, and robotics, the boundaries of the real and virtual are blurred.

We’re happy to announce some new developments for Lucid Possession! Issue Project Room and Roulette will be co-producing the New York premiere at the new Roulette theater in the spring of 2013. Stay tuned for further news and details. Follow us on Facebook or Twitter for updates and announcements (see bar to the right for follow info).

Bob McGrath of Ridge theater is co-directing with us. We’re thrilled to have his vision enhancing the project. Todd Reynolds, violinist. composer and technologist, will be joining us as musical director and performer. Todd will be composing noise segments for the piece as well as playing violin and some crazy custom interface instruments we are cooking up with Ed Bear, our resident multi-talented engineer and artist. Ed is also building some new robotic screens. Leif Krinkle, artist and robotics engineer – who built the incredible main robotic ghost screen (above) for Lucid – is working with us on multiples and other ingenious hardware designs.

This is the Petbot. A virtual screen-based video robot that lip-synchs live to Toni Dove onstage.

It’s a great team as always, with the brilliant R. Luke DuBois on software design. Incandescent Hai-Ting Chinn, mezzo soprano, sings and has the lead role onscreen, and the glamorous singer/performer Bora Yoon (above) is her onscreen nemesis.

In the sample above from work in progress shows at The Spielart Festival, Munich and at Republique, Copenhagen, we see an opening video clip, then move between stage and screen to meet Bean, our lead character, and get a hint of what she’s dealing with: ghosts from the past, noise in the present. It’s a juggling act and she has some big problems to solve. Some of you may find her problems familiar.

In this clip we meet Bean’s Avatar and hear some backstory. What kind of ghosts are we dealing with? Bean’s Avatar sings “She Won’t Know” from a song cycle composed for Lucid Possession by our long time friend and collaborator Elliott Sharp, composer and multi instrumentalist.

It’s clear there are some blurry boundaries between real and virtual, person and doppleganger. Images seem to inhabit multiple worlds in multiple forms. Is it surprising that Bean is having an identity crisis?

Visit us again for more about Bean and her friends. Contact us for presentation and touring information at td@tonidove.com