1. sensing (input)

componentfunctionsource
ceiling-mounted azure kinect (qty depends on room size — e.g. 2 units for a 25×38 ft room) depth tracking to detect presence/position of visitors to be sourced — compatible with standard azure kinect hardware
mini-pc per kinect (1:1 in prior install; can run 1 pc per 2 kinects if a sync cable and a sufficiently long high-speed data cable are used) runs kinect capture/processing locally; accessed via ssh from the main gallery computer for setup, coding, and monitoring during install and exhibition windows pc with latest kinect sdk installed, able to process numpy arrays, wifi-enabled. exact specs otherwise flexible.
high-speed data cable, kinect to pc ~40gbps usb-c cable for reliable data transfer between each kinect and its pc to be sourced

2. processing (logic)

componentfunctionsource
custom tracking software computes presence (ratio of white-to-black pixels after masking the depth array to the range where shoulders typically sit); motion (sum of pixel-value differences between consecutive frames); and center of mass (average x/y/z position of all detected pixels, used to drive the virtual camera's position in the 3d world) artist-built, maintained, and supplied
osc data bridge sends real-time presence, motion, and position values from tracking software to touchdesigner artist-built, maintained, and supplied

3. content pipeline (generation)

componentfunctionsource
youtube api + automated queryer + database pulls randomized search terms, retrieves matching videos daily, logs video ids to a database artist-built; requires api key, supplied by artist
yt-dlp download routine downloads video files from the database artist-built
content refresh (current vs. planned) prior presentation used a pre-downloaded pool — as much footage as fit on 2tb of storage, each file tagged by unique id, sequentially selected and removed from the pool in touchdesigner as it was used. a live cyclic download/delete system, enabling continuous, unbounded content refresh during the exhibition run, is in development and would be implemented for this presentation, time permitting. if implemented, this would also require a self-updating routine for yt-dlp, to keep the automated download process functional through youtube-side platform changes. artist-built
storage (2tb+ external ssd or equivalent) temporary storage for downloaded video before/during exhibition gallery or artist, to be confirmed

4. output (display)

componentfunctionsource
touchdesigner (pro license) builds and renders the real-time 3d video environment license to be secured — purchase or gallery-provided
360° projection setup (projectors + edge-blending/calibration) projection surface for the walkable 3d archive expected from venue. this is the component the artist has the least in-house expertise in — prior presentation took place in a gallery with an existing 360° projection infrastructure already built and calibrated; the artist did not set this up. detailed specs from that prior install (projector count, placement, calibration approach) could be gathered and shared post-award if selected, but are not available ahead of the application deadline.
gallery/presentation computer runs touchdesigner and tracking software in real time gallery-provided or artist-supplied; highest available gpu/ram recommended — exact spec to be confirmed against prior installation's reference machine ahead of install

5. maintenance / reliability

  • daily operation: minimal. in the prior presentation, the kinect tracking script was automated to start each morning and shut down each evening; gallery staff only needed to open the touchdesigner file daily. no maintenance issues occurred over several weeks of exhibition.
  • known failure point: occasional lag or freezing under high-presence conditions (large crowds), caused by gpu load. mitigated by either capping the maximum tracked presence value or running on a higher-spec gpu.
  • platform dependency: if the live cyclic download system (see content pipeline, above) is implemented, yt-dlp would need a self-updating routine to remain functional through youtube-side platform changes. not required for the pre-downloaded-pool approach used previously.
  • network: requires an isolated wifi network, primarily to ensure smooth, low-latency communication between the kinect mini-pcs and the main gallery computer. if the live cyclic download system is implemented, network isolation would additionally help prevent bandwidth competition from affecting real-time download and playback performance.