Open Black Box
Many digital artworks use computer vision techniques that are unable to be appreciated on a technical level by a large portion of their audience. It is unlikely that a human can decipher many of the logics of a computer vision system purely through experience, even if they are interacting with it in real time. Due to the potential to convert and add layers to data from sensors, the output of a digital interactive piece is likely to be visually unrelated to the triggering processes that the user is interacting with. Apprehension towards digital images is reinforced by the ease with which digital images can be manipulated, copied and distributed. That invisible process such as motion detection and facial recognition increasingly extend into our day to day lives, where our most personal interactions with technology occur (cameras, phones and social media), highlights the need for greater depth of understanding of the tools and code that shape our worlds. It is important for us to understand the points at which we are detectable by machines so the we have the potential to navigate the environments we exist in with autonomy.
The visual representation of image-based calculations and smart camera readings close to their simplest form are often aesthetically interesting in themselves. Working with OpenCV and OpenFrameworks allows for interaction with digital images as data to the point that you can see the results of numerical manipulations on them. The opportunity to see the result of subtracting pixel colour values from each other over time, as in frame differencing, visually gives an idea of how machines approach the task of detecting changes. It also shows certain limitations, for example just having one angle with no depth image, as everything is happening on the same 2d plane. The background is only (temporarily) visible when something in front of it moves. Similarly, interacting with the 3d point cloud generated from a depth camera such as the Kinect also quickly helps understand the potential and limits of the technology.
Different parts of the point cloud are revealed as the viewer stands at different distances, as it rotates and zooms in and out accordingly, while the sensor tilt angle is affected to emphasise the effects of frame differencing on the whole frame. The partially transparent and reflective surfaces also have some function in extending possibilities for perceptible interrogation. The structure recalls older forms of technology with related genealogies, while querying the usefulness of smoke and mirrors in technical practice.