@BUSTLELAMP

A NEW WORK IN DEVELOPMENT

I began by thinking about two characters across oceans of time and how they would communicate. One a mathematician, one an engineer. They might speak using mathematics. I was looking at our current moment and the relentless, exhausting polarization. This piece is really about the tension between the fear of knowledge and the constructs humans use to build language and symbol systems that speak about what we can’t see, what we can’t touch. Mathematics, physics, the evolution of models of the cosmos…and the forces of attraction between heavenly bodies…or humans.

The core of Hypatia is a hybrid AI that responds to the movement and gestures of multiple viewer-operators, and reacts to them with what appears to be a complex personality.

This innovative format is a fusion of sculpture, installation, cinema and performance for viewers moving in a space. Versions will exist for linear performance and for installation. 

Viewer/operators move around the triangular scrim and engage with Hypatia and other characters and elements of the narrative. 

The project is a visual novel, a movie, and a machine that tells stories. A poetic and dreamlike experience, viewers activate the process and become embedded in another dimension.

–

TECHNICAL REQUIREMENTS AND EQUIPMENT FOR INSTALLATION:

The screen has 3 scrims in a triangle formation. It breaks down and reassembles for travel. There are 3 short throw projectors hung – one for each side of the triangle.

3 speakers and 6 cameras are embedded in the screen structure, a cube of 8 speakers: 4 hung in the upper room corners, 4 on floor stands, and 2 additional Ultrasonic speakers on servo motors hung. Equipment travels with the show unless a venue supplies specified projectors or speakers. 

The AI combines Azure Kinect skeletal tracking and game engine AI with proprietary software that includes blob detection and interactive and emergent behavior design. 

The installation version is an infinity loop that runs for approx. 15 min. The longer narrative performance will be approx. 40 min. There are a variety of ways the installation can be presented using both formats or one and this can be shaped with the venue to best suit presentation needs. Spaces: Black box theaters or gallery spaces that have the ability to be dark and hang speakers and projectors. The screen structure is 10’ high with 3 projection surfaces of 8’x10’. The minimum ceiling height for the room is approx. 12’

SUNJAMMER 6 IS SUPPORTED BY: 

• Foundation for Contemporary Arts 2026
• Doris Duke Foundation Performing Arts Technologies Lab Grant, 2024/2025
• Fellow: Yale University CCAM, 2023-ongoing, Research and Development
• Artist in Residence: Integrated Digital Media, Tandon School of Engineering, NYU, ongoing
• N.E.A. Grant with Harvestworks and Paul Geluso, Director of Music Technology, Steinhardt School of Music, NYU, for sound follower instrument technology development, 2022
• Artist in Residence: Bell Labs, E.A.T. Program, 2020/21
• Artist in Residence: Pioneer Works Virtual Environments Lab,
• 2020•N.Y.S.C.A.: Independent Artist Film and Video Grant 2020

CREATIVE TEAM: 

• Siyuan Qiu: Lead Unity Developer for Visual and Interactive Design
• Nina Demirjian: Unity and AR Developer
• Tommy Martinez: Sound Technologist
• Quentin Chiapetta, Medianoise: Sound Design
• Paul Geluso: Spatialized Sound Design System
• Chorus: VO by Christina Campanella and John Rose
• Kepler: VO by Jim Fletcher
• Wufeng Kai: 3D Modeling and Unity Development
• Indira Ardolic: 3D Modeling

RECENT WORK BY TONI DOVE: The Dress That Eats Souls 

• Vimeo documentation includes project description and links to press: https://vimeo.com/268335347
• NYTimes Profile, Ted Loos: http://nytimes.com/2017/10/28/arts/toni-dove-dress.html

Selected Reviews: 

• Art News, Barbara MacAdam:  http://www.artnews.com/2018/03/15/vasari-diary-stuart-klipper-toni-dove-william-h-gass-reader/
• Herald Tribune, Marty Fugate: heraldtribune.com/entertainmentlife/20180322/art-review-multimedia-artist-toni-dove-imagines-brave-new-world-at-ringling

Lucid Possession, a multimedia performance, premiered at Roulette in Brooklyn, and combines musicians, VJ mashing, and stage-controlled robotic projection screens to present a contemporary ghost story – a poetic musing on managing the mass of information “noise.” Todd Reynolds live scores and composed the finale. Hai-Ting Chinn stars in the video and sings onstage. I run all cues from the stage and scrub video across multiple screens. Elliott Sharp composed the song cycle. The technologies of Lucid led me to a new installation project.

Lucid Possession and The Dress That Eats Souls will be part of a retrospective on my interactive work to be presented in early 2018 at The Ringling Museum in Sarasota, Fla.

First a little on Lucid Possession. The central character, Bean, is a young artist who designs virtual personalities and is plagued by ghosts. Her mind is like a live Twitter feed that “picks up people” …but without technology. She creates an avatar, an exaggerated alter ego that goes viral on the Internet and makes her a minor celebrity. People stop her on the street. They want something, and she isn’t sure what it is. Anxiety exponentially increases her paranormal sensitivities, and a ghost from the past emerges from the noise.

I was thinking about the way we live in analog and virtual worlds simultaneously – how we live an augmented reality through social networks and online connections that merge with our life in the physical world. It’s ordinary – but fantastic. A TV remote, social networks, ATM machines – they’re like pedestrian spoonbending. With telepresent agency – our bodies extend beyond their edges. Our boundaries blur. I was also thinking about how this landscape has turned us all into performers grasping for attention in a sea of market share. Everything becomes a popularity contest. How many likes for your cat video? Here’s a clip from the premiere.

Making Lucid Possession involved creating a complex intranet – an engine of linked technologies with many elements that included the production of LED costumes and robotic screens. The expansion of my technical vocabulary combined with further experiments with embodied interface inspired me to focus more closely on some of the possibilities these elements presented.

Building Lucid’s robotic screen: from sketches, to construction, to the projection of Bean’s Avatar. The arms and legs are selectable video loops scrubbed in real time. The head uses a neural net and vocal analysis to lip synch live to a performer. The robotic screen is controlled by motion sensing onstage or by OSC on an iPad.

The costumes were constructed with LEDs. This was before the current explosion of Raspberry Pis and micro controllers, conductive thread and flexible electronics for wearables. We had a guy with a soldering gun running around after our actors during the film shoot. In 5 years there’s been enormous advances in wearable technologies and the availability for easy use is everywhere. Karen Young: Costume design, Leif Krinkle: technology and controls.

I began thinking about the explosion of technologized wearables. It seemed to be dominated on many fronts by the fusion of narcissism and commodification. A new installation project started to percolate. The first sketch:

THE DRESS THAT EATS SOULS: An Interactive Robotic Dress Installation

progress, n. a forward movement: an advance: a continuation: an advance to something better or higher in development: a gain in proficiency: a course: a passage from place to place: a procession: a journey of state: a circuit. – v.i. progress, to go forward: to make progress: to go on, continue: to go in progress, travel in state: to go.Chambers 20th Century Dictionary

Xenophanes: “The gods did not reveal to men all things in the beginning, but men through their own search find in the course of time that which is better.”

Or not:

The Dress has a 14′ layered scrim skirt that acts as a projection screen and a cinemascope rear projection screen that hangs overhead. The robotics of the bodice and skirt are controlled by a kinect gaming interface. As a viewer standing in front of the Dress moves, the Dress mirrors their movement. It behaves as if you are wearing it. The Dress speaks to you. It’s rather chilly – like a lizard. It’s the combined human agendas that drive technology. The installation cycles between the Dress speaking and POV experiences on the overhead screen and skirt that put you inside the minds of the people who have worn the Dress as it evolved – 200 years of the human body shaped, molded and colonized by technology. Viewer head movement navigates the cinema space.

So I’ve started to build and script it. Brooklyn Research is working on the Robotics. Rene Steinke, the novelist, is working with me on the voices. Tommy Martinez is programming. Paul Geluso is designing the spatialized sound. Ben Light is prototyping and constructing elements of the Dress design. Leif Krinkle is technical director. Karen Young is helping with costume construction. Andy Dintenfass and Art Jones are shooting the video material with me. And many thanks to students from Parsons Design and Technology for their labor and brainstorming. Here’s some of the prototyping and building process:

First: lots of testing and discarding elements to discover the best way forward. Finally: building the robotic screen system. Robotics building and testing is going on at Brooklyn Research. The first Kinect test with Johnny Lu. Then the more evolved version – a serious robot!

We’re simultaneously building the skirt and bodice elements, testing media and programming navigation on a small mockup of the installation at the studio. This video shows Tommy using the kinect. The white cube in the first section shows how the Kinect sees and tracks his head movement. The second section shows how he navigates between 5 video streams with head motion.

Stay tuned for further developments as The Dress That Eats Souls evolves!

“Fans of offbeat cinema or science fiction might find aspects of Spectropia intriguing, because moments suggest an eerie blend of The Big Sleep, Brazil and La Jetee.”

The adventure continues as Spectropia inside Verna’s body in 1932 tries to solve some perplexing problems about her father’s disappearance in time. A mystery and a romantic triangle unfold across centuries as two women in one body drive a man crazy. More info in the Spectropia category to your right.

“Shifting eras and moods are powerfully evoked by the design, from the rich cinematography and costumes to Elliott Sharp’s eclectic soundtrack.”

– The Columbus Dispatch

Featuring the song “This Time, That Place” with vocals by Debbie Harry.

SPECTROPIA Episode #6: “Time is slipping”

Ghosts appear and the mystery deepens. What the Duck knows.

SPECTROPIA Episode #7: “Keeping secrets?”

An argument and a visit to Sally Rand. Collecting clues.

SPECTROPIA Episode #8:

“It’s not a straight line at all”

Stumped and stymied. Nothing is what it seems. Who is William?

SPECTROPIA Episode #9: “The triangle ends”

William lies and Verna gets fed up.

SPECTROPIA Episode #10:

“A private detective of memory”

Looked at in a certain way, time is random access.

Elliott Sharp composed the soundtrack for Spectropia. Check out his CD Spectropia Suite featuring the song “This Time, That Place”, Vocals by Debbie Harry and his CDIncident that includes songs written for Toni Dove’s Lucid Possession with vocals by Hai-Ting Chinn and Bora Yoon. More info below.

You can also find Spectropia on
Vimeo, Youtube and Reel House.

Stay Tuned for an app that will release Toni Dove’s Lucid Possession is 6 episodes. It will be released when we figure out how to do it!

We’ll keep you posted here on our progress and on some other projects involving robotic clothing.

Over two nights, MAY 4 and 5,  ROULETTE will present two radically different aspects of Toni Dove’s Spectropia.

FRIDAY MAY 4, Roulette presents Toni Dove’s Spectropia, a feature-length live-mix cinema event—a scratchable movie performed by Toni Dove and  R. Luke DuBois, artist and project software designer. Buy Tickets.

SATURDAY MAY 5, Elliott Sharp and The ’31 Band perform “Spectropia Suite”. Guest vocalist Barbara Sukowa sings the Spectropia song “This Time, That Place”. Toni Dove and R. Luke DuBois craft live video improvisations: a silent movie to accompany the score. Buy Tickets.

A sci-fi hybrid with themes of time travel, telepathy, and elements of film noir, Spectropia features live VJs orchestrating onscreen characters through a mix of film, performance, and a system of motion sensing that serves as a cinematic instrument.

Dove and DuBois scrub and navigate up to six layers of narrative video and sound—it’s like swimming through a movie! It’s a mystery, a puzzle, a time travel drama, and a romantic triangle—and it’s never quite the same twice. The first night you’ll see the full feature film in all its crazy complexity, then join Elliott Sharp, the composer of the music soundtrack with his ’31 Band, a nine-piece band from a parallel universe in 1931. It’s a thrilling cinematic happening. On Saturday, the film becomes an improvised silent movie.

Aleksa Palladino as Spectropia, Carlolyn McCormick as Verna, Richard Bekins as William, Simon Jones as The Duck,Helen Pickett as Sally.

See Episodes 1 and 2 from the soon to be released serial version of Spectropia at Streaming Museum.

“…it’s just plain cool to watch. Highly recommended.”

– Jeremy Barker, Culturebot

“Ms. Dove, together with co-performer and software engineer R. Luke Dubois, employs her “rig” to present “Spectropia,” an interactive film as immersive for its two real-time performers as it is for the audience. “

Photo: Brian Derballa for The Wall Street Journal

“The multilayered presentation of “Spectropia” is…unlikely to sound or appear familiar to anyone who hasn’t already witnessed it—or to anyone unfamiliar with Ms. Dove. Since the early 1990s, the artist has explored the intersection of narrative experience and audience participation through complex, interactive installations aided by advancing technology.”

-Bruce Bennett, Wall Street Journal

ELLIOTT SHARP

Photos: Peter Cherches, Sascha Rheker

The musicians of The ’31 Band, all of whom have extensive experience in many realms of music, have worked with Sharp on many projects, from the various Western traditions of jazz and classical music to the farthest reaches of contemporary music, free jazz, and improvisation. For this performance, The ’31 Band will include E# playing Bb & bass clarinets, tenor saxophone, guitar & computer processing;  Briggan Kraus – alto saxophone; Nate Woolley – trumpet; Art Baron; Curtis Fowlkes & Steve Swell – trombones; Anthony Coleman – piano; David Hofstra – string bass; and Don McKenzie – drums.

Photo: Andreas Sterzing

Barbara Sukowa is known for her performances onstage and in some of the most iconic films of the New German Cinema with directors such as Fassbinder and von Trotta. She has a career as a classical music narrator and singer and is the lead singer of the band the X-Patsys, which she founded with visual artists Jon Kessler and Robert Longo.

From Douglas Detrick -About.com/Jazz:

“Sharp’s compositions, which often evolve through repetition and micro-variation, mirror mathematical processes, but don’t sound dryly scientific. Sharp is interested in discovery, and similar to the way math attempts to capture the nature of the world through the study of patterns and deduction, Sharp’s music seeks musical truth by extracting the essence of a musical concept and exploring it through various perspectives.

“Spectropia Suite… features mosaics of noise, string quartet pieces, dark and alluring jazz piano solos, and music for large ensemble, with growling blues riffs reminiscent of the Fletcher Henderson band or the early Duke Ellington orchestra… a wide-ranging tour through jazz, contemporary concert music, and avant-garde noise, three main facets of Sharp’s work as a musician over the last 30 years.”

Purchase CD

Come and see us over two nights! They should be two very different experiences – and we think you’ll want to see them both.