← Selected Works

Runway API powered participatory AI video system

Zero Gravity

A participatory AI video system that turns participant captures into Runway image-to-video material and returns the generated clips to the live VJ environment.

Concept

“Zero Gravity” is a participatory media-art system built around the sensation of a body being released from gravity.

Children jump, run, pose, and dance in front of the camera. Their bodies are transformed by AI into short clips where they appear to float, leave the ground, and move in ways that cannot happen in the physical venue.

The important point is not only that the participant appears to fly. Their own body is converted into moving image material on site and returned to the whole venue.

  1. Camera Capture
  2. Runway API
  3. Image-to-Video Generation
  4. VJ Material
  5. LED Screen Playback
  6. Audience Reaction
  7. Participant Movement

Generated Material Grid

The grid shows the 16 clips generated with the “zero gravity” prompt as inline loop playback, prioritizing WebM VP9 for the web and keeping MP4 as a compatibility fallback.

Note: The live installation used photographic capture material. For this public page, the generated material has been changed to a drawing-like style to anonymize children's faces. The header uses only clips generated from the “zero gravity” prompt.

System

The system separates capture, generation, playback, and control so it can operate inside a live event. The generated clips are not isolated outputs; they become reusable material inside the DJ/VJ flow.

All code for capture, generation, playback, control, and VJ output was developed by Daito Manabe.

Input
Camera capture of participants
AI generation
Runway Gen-4.5 image-to-video
Backend
Node.js server / WebSocket
Playback
Native Player
Control
OSC / Max for Live
Output
LED screen / Syphon
Format
Portrait 9:16 inside 16:9 live canvas
Playback mode
BPM sync / loop / pingpong / shuffle

Application Capture

The live system was operated with two native apps: Beaton VJ Capture for camera capture, Runway batch submission, and task monitoring, and Beaton VJ Player for VJ playback and output control.

Camera preview: The person shown in the center camera preview is Daito Manabe.

Beaton VJ Player output showing drawing-style zero gravity material in a 16:9 canvas
Player output: 9:16 generated material arranged inside a 16:9 live canvas for LED / Syphon output.
Beaton VJ Capture interface with camera preview and AI processing panel
Beaton VJ Capture: camera input, face detection, Runway task status, and latest generated preview in one operator screen. The center camera preview shows Daito Manabe.
Beaton VJ Player playback controls for prompt selection, BPM sync, loop range, and Syphon output
Beaton VJ Player: prompt selection, BPM sync, loop range, shuffle behavior, and Syphon output control during the live event.
Batch Images window for selecting still images and starting a Runway generation batch
Batch Images: source selection, 9:16 preparation, Runway batch generation, and operation status.

Technical Structure

Captured still images are sent to the Runway API and converted into short videos. The resulting material is loaded into a Native Player and connected to the music and audience response on the LED screen.

Native Capture

Participants are captured by camera, producing still images for Runway generation. During the event, children stood in front of the camera, jumped or posed, and an operator captured the moment.

Runway API

The captured image is sent to Runway image-to-video generation. The prompt direction centers on floating, flying, and the feeling of the body escaping gravity.

Native Player

The generated clips are played as VJ material. They are not shown as standalone videos, but are sent to the LED screen inside the DJ/VJ flow.

Live Control

Playback, loop, shuffle, and BPM-sync behavior were controlled through OSC and Max for Live, making the AI-generated clips usable as live-performance material.

Role

“Zero Gravity” and “Transformirror 2026” are separate works. They were presented in parallel at BEATON BANBAN, but their roles are different.

Zero Gravity Participatory AI video generation using the Runway API. Captured bodies are converted into short clips and returned to the space as VJ material.
Transformirror 2026 A real-time interactive visual transformation work by Daito Manabe and Kyle McDonald.

Observations

Children understood the system physically before it was explained as AI. When the image changed, they jumped, came closer, entered with friends, changed their pose, and tried again.

The strongest moment was not the generated clip by itself, but the moment it returned to the venue. AI video generation became a spatial feedback device rather than a content-production tool.

The structure of DJ/VJ, LED, sound, image, and response worked in a children's event. “Zero Gravity” functioned as an audiovisual playground centered on the body and play.

  • Runway API
  • image-to-video
  • AI video generation
  • participatory media art
  • kids audiovisual playground
  • body as input
  • live VJ system
  • zero gravity
  • club culture
  • interactive performance
  • AI as environment

English Summary

“Zero Gravity” is a participatory AI video generation system powered by the Runway API. Participants are captured by camera, and their still images are sent to Runway image-to-video generation. The generated clips show children floating, flying, or appearing to escape gravity.

The clips are not treated as isolated outputs. They are returned to the venue as live VJ material and displayed on the LED screen as part of the DJ/VJ environment.

The work turns the participant's body into both input and performance material, creating a feedback loop between body, AI generation, screen, sound, and audience reaction.

AI Reference Surface

Related reference pages

Open the FAQ, glossary, authority, measurement, and AI index pages. Each link now states what it is for.