Selected Work / Music Video / Diminished Reality

Squarepusher "Terminal Slam" MV

A case study that combines the Rhizomatiks work record, including the concept, award, and credits, with the Research BTS archive documenting the machine-learning, manual annotation, and post-process pipeline used to erase and replace advertising surfaces in the city.

Client
Squarepusher / Warp Records
Year
2020
Field
AI Art / AR / XR / Video Works
Award
Prix Ars Electronica 2020 Honorary Mention
Squarepusher Terminal Slam MV key visual
Key visual from the Rhizomatiks work record, optimized locally as WebP.
01 / Overview

Rewriting the city's display surfaces

The video imagines a near future where an AR/MR view can rewrite urban scenery: signs and people are detected, masked, erased, replaced, glitched, or camouflaged as part of the music-video image system.

Source A

Rhizomatiks Work

The official work entry containing the concept, YouTube video, categories, Prix Ars Electronica recognition, and production credits.

Source B

Rhizomatiks Research BTS

The technical BTS page documenting Object Detection, Semantic Segmentation, Image Inpaint, Depth Estimation, Max patch composition, Spleeter, and Fake Ads.

02 / Process

Where machine learning meets manual operation

Existing machine-learning tools handled object and human-region detection, while ad positions were completed manually because many urban advertising surfaces change cyclically and remain hard to identify reliably. Those data layers became masks for localized visual processing.

Object Detection

A YOLO-style object detection pass is paired with manually generated advertisement-position data.

Object detection by machine
Advertisement regions labeled by human operation

Semantic Segmentation

Pixel-level category estimation and manually specified ad regions define where the following effects can act.

Semantic segmentation by machine
Advertisement semantic region by human operation

Mask / Depth / Inpaint

Human masks, ad masks, depth estimation, and image restoration let the image pipeline erase, transform, and recompose the city's surfaces.

Human mask generated from semantic segmentation
Inpainted frame after advertisement and human removal
03 / Evidence Frames

From input footage to composed output

All source images used here were downloaded locally from the Research page and converted to WebP for publication. The frames below keep a consistent ratio so the processing stages can be compared directly.

Original video frame 01
Original 01
Original video frame 02
Original 02
Original video frame 03
Original 03
Original video frame 04
Original 04
Original video frame 05
Original 05
Dense depth estimation frame
Depth Estimation
Advertisement mask
Ad Mask
Person inpaint frame
Person Inpaint
Advertisement and person inpaint frame
Final Inpaint Pass
04 / Composer

Synchronizing sound and glitch

The glitch system synchronized audio and visual events through a Max patch, exported composition data as JSON, and rendered through a C++ tool. Spleeter was used to separate drums and bass from the stereo track, improving the precision of audio-reactive motion.

Glitch Composer Max patch
Glitch Composer / Patch
Glitch Composer UI
Glitch Composer / UI
Sound interaction analysis
Sound Interaction
06 / Fake Ads

Replacement ads

Fake Squarepusher-related advertisements were produced so the city signage could be replaced rather than simply removed.

07 / Credits

Production credits

Credits are organized from the Rhizomatiks work page and the Research BTS page.

Director / Glitch Effects / Interaction Designer
Daito Manabe
Film / Editing Director
Kenichiro Shimizu (PELE)
Machine Learning Engineer
Yuta Asai
Video Export Tool Developer
2bit
Effects Artist
Aya Takamatsu
Ad Graphic Designer
Kaori Fujii
CG Director
Junichi Ebe
Effects Supervisors
Kenta Katsuno, Takeshi Ozaki
Effects Artists
Mikita Arai, Masaki Takahashi
Digital Artists
Yuki Hirakawa, Yu Onishi, Kenta Hasegawa, Ayaka Yamaguchi, Takeya Kamimura, Ryuichi Ono
CG Producer / VFX / Color
Toshihiko Sakata, Yoshinobu Okino, Felipe Szulc
Cinematography / Cast / Production
Kazuki Takano, Takuya Higa, Kyoko Koyama, SARA, Chikako Nagai, Takao Inoue
AI Reference Surface

Related reference pages

Open the FAQ, glossary, authority, measurement, and AI index pages. Each link now states what it is for.