white clouds in blue sky

(work-in-progress)

A video installation by Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola

 

white clouds in blue sky is a 24 minute, two-channel video installation that juxtaposes a performative engagement with the materiality of gallery refuse with the poetics and politics of machine vision. In one video channel, artists Hennessy and Smith methodically construct a sculptural heap of utilitarian objects like stacks of chairs, and scrap materials that have been gathered after an exhibition and are destined for the landfill. As they create and then deconstruct the pile of mundane and broken objects, a second video channel shows a poetic text that is edited by the artists from sentences generated by DenseCap machine vision and description system, which repeatedly seeks to identify and interpret shapes created as Smith and Hennessy work. The system uses a neural network language model to generate language sequences (Johnson et. al. 2017), yet the abstract shapes created as the artists arrange waste materials confound the system (see Martineau, 2019), and its interpretations become textual waste of a new order. The text in the second video channel is drawn from sentences generated by DenseCap’s interpretation of video frames, approximately every 15 seconds. The displayed text has been edited by the artists as an act of human intervention, a salvaging of meaning through human reinterpretation. Shown alongside and in time with the performance video, these edited texts reveal an alternative reading of the material associations and coincidental configurations that Hennessy and Smith create in the video. DenseCap’s reading of the object assemblage creates a new worlding, instructing the viewer that a chair may also be a fire hydrant, a cement gallery floor may also be wood, and that piles of trash evoke white clouds and blue sky. At the same time, the machine’s reading of assemblages of gallery waste create new resource-hungry digital waste that the artists are compelled to sort, rearrange, and delete to create new human-readable narratives.

The installation highlights tensions between individual human structures of memory and imagination, and contemporary computational image recognition systems. As they sort through and re-arrange both material refuse and digital waste, the artists draw parallels between the planetary proliferation of material waste, the proliferation of digital imagery being mobilized in artificial intelligence data training sets, and the energy resources required and wasted to power these interactions. By drawing attention to current limitations of machine vision in recognizing and describing objects (and unique assemblages of objects), the work points to significant possibilities and difficulties as humans and machines increasingly mutually constitute, reinforce and re-write classifications and meanings of things. How is the experience of contemporary art practice, such as abstract sculpture and performance, either corrupted or propelled by machine vision? How will machines read images and artworks in the future, and what stories will be told about them? What stories will humans be able to tell and imagine in the future, in relation to new intelligent storytelling machines? What kind of planet will we inhabit? Will the skies be blue? Will the clouds be white?

white clouds in blue sky, video still. Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola, 2019.

Working image 1, “white clouds in blue sky”, 2019. Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola

 

Working image 2, “white clouds in blue sky”, 2019. Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola

 

Working image, “white clouds in blue sky”, 2019. Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola

 

References

Johnson, J., Karpathy, A., Li, F. (2016). DenseCap: Fully Convolutional Localization Networks for Dense Captioning. Accessed at https://cs.stanford.edu/people/karpathy/densecap/

Martineau, K. (2019) This object-recognition dataset stumped the world’s best computer vision models. MIT News, Dec. 10th, 2019. Accessed at http://news.mit.edu/2019/object-recognition-dataset-stumped-worlds-best-computer-vision-models-1210

Biographies

Kate Hennessy and Trudi Lynn Smith are anthropologists and practicing artists that have worked together as curators and collaborators since 2009 as a part of Ethnographic Terminalia, an international curatorial collective exhibiting and creating works at the intersection of art and anthropology. Hennessy is an associate professor at Simon Fraser University’s School of Interactive Arts and Technology, where she leads the Making Culture Lab, an interdisciplinary research and production studio. Smith is an adjunct assistant professor in the School of Environmental Studies at the University of Victoria, and recently held the position of artist-in-residence with Hennessy in the Making Culture Lab. Together they explore cultural practices of media, museums, and archives in the context of technoscience. Their art practice in video, photography, and text has engaged with entropy in diverse collections and the environmental, social, and political impacts of new digital memory infrastructures. Their art work and writing has been featured in journals such as PUBLIC: A journal of arts and culture (2018); the literary magazine Geist (2018) and Visual Anthropology Review (2020). Their photographic and video work was recently shown in a solo exhibition at the Royal British Columbia Museum (2018-19), at the Bard College Center for Experimental Humanities in upstate New York (2019), and at Gallery 500x in Dallas, Texas (2017). As ethnographers as well as artists, Hennessy and Smith highlight collaboration as a central aspect of their work, working with both human and non-human entities to represent the politics of the material world and its relationships with human agents.

Hennessy and Smith collaborated with cognitive scientist, artist, and AI scholar Steve DiPaola (Professor, School of Interactive Arts and Technology at Simon Fraser University) to use AI-based image description tools to generate the text of white clouds in blue sky. DiPaola’s primary research areas are cognitive, character and expression based artificial intelligence, interaction and computer graphics. His computational artwork was notably commissioned by video artist Nam Jun Paik for his work Fin de Siecle II, which was recently reinstalled as part of the new computer programmed art retrospective at the Whitney Museum of American Art (NYC, “Programmed: Rules, Codes, and Choreographies in Art, 1965–2018”).