A NEW WORK IN DEVELOPMENT
I began by thinking about two characters across oceans of time and how they would communicate. One a mathematician, one an engineer. They might speak using mathematics. I was looking at our current moment and the relentless, exhausting polarization. This piece is really about the tension between the fear of knowledge and the constructs humans use to build language and symbol systems that speak about what we can’t see, what we can’t touch. Mathematics, physics, the evolution of models of the cosmos…and the forces of attraction between heavenly bodies…or humans.
Hypatia, a Hellenistic astronomer, mathematician and philosopher assassinated in 415 AD returns 10 years in our future as a furious ghost. As she navigates her new environment she absorbs what has happened over time and encounters a NASA engineer building a solar array off world power station.
A mixed reality installation for six or more viewers, Sunjammer 6 uses custom AI to allow virtual characters to see and react to multiple people in a room by tracking and responding to their movement with interactive visuals and sound.
The project is a visual novel, a movie, and a machine that tells stories. A poetic and dreamlike experience, viewers activate the process and become embedded in another dimension.
The core of Hypatia is an AI that responds to body language, to the movement and gestures of multiple viewers, and reacts to them with what appears to be a complex personality. She sees how many people are in the room and can interpret types of movement or motivation and react accordingly.
This innovative format is a fusion of installation, cinema and performance for viewers moving in a space. Versions will exist for linear performance and for installation. Viewers move around the triangular scrim and engage with Hypatia and other characters and elements of the narrative.
The installation version is an infinity loop that runs for approx.15 min. The longer narrative version is approx. 40 min. There are a variety of ways the installation can be presented using both formats or one and this can be shaped with the venue to best suit presentation needs.
The installation is designed for black box theaters or gallery spaces that have the ability to be dark and hang speakers and projectors. The screen structure is 10’ high with 3 projection surfaces of 8’x10’. The minimum ceiling height for the room is approx. 12’
In addition to the speakers and cameras embedded in the screen structure the installation requires 8 speakers: 4 in the upper corners, 4 on floor stands in the corners of the room, three short throw projectors hung, 2 additional Ultrasonic speakers on servo motors hung. Venues would supply 4 or 8 of the specified corner speakers. The rest of the equipment travels with the show unless a venue supplies specified projectors.
The tracking engine is comprised of Azure Kinects, RealSense cameras, AI, a Unity animation environment, directional speakers, robotics and surround sound designed in Ableton Software with custom MAX elements for dynamic and interactive sound. The AI combines Azure Kinect skeletal tracking with proprietary software that includes blob detection and interactive and emergent behavior design.
SUNJAMMER 6 IS SUPPORTED BY:
- Artist in Residence: PioneerWorks Virtual Environments Lab, Director: Tommy Martinez, 2020
- Artist in Residence: Bell Labs, E.A.T. Program, Director: Domhnaill Hernon, 2020/21
- Artist in Residence: Integrated Digital Media, Tandon School of Engineering, NYU, Director: Luke DuBois, ongoing
- Fellow: Yale University CCAM, Director: Dana Karwas, 2023/2024, Research and Development
- N.Y.S.C.A.: Independent Artist Film and Video Grant 2020
- N.E.A.: Grant with Harvestworks and Paul Geluso, Director of Music Technology, Steinhardt School of Music, NYU, for sound technology development 2022
- Kim Whitener: Creative Producer
- Tommy Martinez: Producer and Sound Technologist
- Siyuan Qiu: Lead Unity Developer for Visual and Interactive Design
- Nina Demirjian: Unity and AR development
- Wufeng Kai: 3D Modeling and Unity Development
- Quentin Chiapetta, Medianoise: Sound Design
- Rene Steinke: Novelist/ Script Development
- Paul Geluso: Spatialized Sound Design System
- Chorus: VO by Christina Campanella and John Rose
- Danielle McPhatter: E.A.T. Program, Bell Labs, EY: Software Design
- Indira Ardolic: 3D Modeling
- Vimeo documentation includes project description and links to press: https://vimeo.com/268335347
- NYTimes Profile, Ted Loos: http://nytimes.com/2017/10/28/arts/toni-dove-dress.html
- Art News, Barbara MacAdam: http://www.artnews.com/2018/03/15/vasari-diary-stuart-klipper-toni-dove-william-h-gass-reader/
- Herald Tribune, Marty Fugate: heraldtribune.com/entertainmentlife/20180322/art-review-multimedia-artist-toni-dove-imagines-brave-new-world-at-ringling