Excerpt from white clouds in blue sky (2022). Preview 1 min 50 sec.
Original work 6 mins. 25 sec. By Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola.

white clouds in blue sky

white clouds in the blue sky is a three-channel video installation by artist-anthropologists Kate Hennessy and Trudi Lynn Smith and artist and artificial intelligence scholar Steve DiPaola. The premiere of the first iteration of this work was in February, 2022 at the Surrey UrbanScreen, as a part of the program Body as Border, curated by Gabriela Aceves-Sepulveda, Freya Zinovieff, and prOphecy Sun. Edited by J Tseng; AI systems support by Amineh Ahmadinejad.

Installation view, white clouds in blue sky, 2022. Surrey UrbanScreen, Feb. 2022.

The work juxtaposes a performative engagement with the materiality of gallery refuse with the poetics and politics of machine vision. Hennessy and Smith methodically construct a sculptural heap of utilitarian objects like stacks of chairs and scrap materials that have been gathered after an exhibition and are destined for the landfill. As they create and then deconstruct the pile of mundane and broken objects, these assemblages are interpreted by DenseCap machine vision and description system, which is confounded in its attempts to accurately identify and interpret assemblages of objects created. The system uses a neural network language model to generate language sequences (Johnson et. al. 2017), yet the abstract shapes created as the artists arrange waste materials are misinterpreted by the system (see Martineau, 2019), and its articulations become textual waste of a new order. A second channel shows a poetic text that is edited by the artists from sentences generated by the system.  A third channel displays a montage of AI-generated images based on  the poetic text, a further attempt by the machine to make sense of what it sees. The soundtrack is yet another AI-generated response to the performance-initiated texts. 

The title of the work, white clouds in the blue sky is frequently repeated throughout the work and is audible in the phrasing of the soundtrack. It is evocative in its misinterpretation of the scene; its association of landfill refuse with an idealized image of nature; and its reference to the metaphor and materiality of the cloud, where data processing  and rendering of the generated images is taking place. This video work highlights tensions between individual human structures of memory and imagination, contemporary computational image recognition systems, and the capacity of generative systems to create new representations of the world. As the artists sort through and re-arrange both material refuse and digital waste, they draw parallels between the planetary proliferation of material waste, the proliferation of digital imagery being mobilized in artificial intelligence data training sets, and the energy resources required and wasted to power these interactions. By drawing attention to current limitations of machine vision in recognizing and describing objects (and unique assemblages of objects), the work points to significant possibilities and difficulties as humans and machines increasingly mutually constitute, reinforce and re-write classifications and meanings of things. How will machines read images and artworks in the future, and what stories will be told about them? What stories will humans be able to tell and imagine in the future, in relation to new intelligent storytelling machines? What kind of planet will we inhabit? Will the skies be blue? Will the clouds be white?  

Video still, white clouds in blue sky, Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola, 2022.

Ai-generated GANS image,  white clouds in blue sky, Kate Hennessy, Trudi Lynn Smith, and Steve DiPaola, 2022.

Conference Presentation at the 2023 Royal Anthropological Institute Conference “Anthropology, AI, and the Future of Human Society”, June 2022.

References

Johnson, J., Karpathy, A., Li, F. (2016). DenseCap: Fully Convolutional Localization Networks for Dense Captioning. Accessed at https://cs.stanford.edu/people/karpathy/densecap/

Martineau, K. (2019) This object-recognition dataset stumped the world’s best computer vision models. MIT News, Dec. 10th, 2019. Accessed at http://news.mit.edu/2019/object-recognition-dataset-stumped-worlds-best-computer-vision-models-1210

Biographies

Kate Hennessy and Trudi Lynn Smith are anthropologists and practicing artists that have worked together as curators and collaborators since 2009 as a part of Ethnographic Terminalia, an international curatorial collective exhibiting and creating works at the intersection of art and anthropology. Hennessy is an associate professor at Simon Fraser University’s School of Interactive Arts and Technology, where she leads the Making Culture Lab, an interdisciplinary research and production studio. Smith is an adjunct assistant professor in the School of Environmental Studies at the University of Victoria, and recently held the position of artist-in-residence with Hennessy in the Making Culture Lab. Together they explore cultural practices of media, museums, and archives in the context of technoscience. Their art practice in video, photography, and text has engaged with entropy in diverse collections and the environmental, social, and political impacts of new digital memory infrastructures. Their art work and writing has been featured in journals such as PUBLIC: A journal of arts and culture (2018); the literary magazine Geist (2018) and Visual Anthropology Review (2020). Their photographic and video work was recently shown in a solo exhibition at the Royal British Columbia Museum (2018-19), at the Bard College Center for Experimental Humanities in upstate New York (2019), and at Gallery 500x in Dallas, Texas (2017). As ethnographers as well as artists, Hennessy and Smith highlight collaboration as a central aspect of their work, working with both human and non-human entities to represent the politics of the material world and its relationships with human agents. Steve DiPaola (Professor, School of Interactive Arts and Technology at Simon Fraser University) is a cognitive scientist, artist, and AI scholar whose primary research areas are cognitive, character and expression based artificial intelligence, interaction and computer graphics. His computational artwork was notably commissioned by video artist Nam Jun Paik for his work Fin de Siecle II, which was recently reinstalled as part of the new computer programmed art retrospective at the Whitney Museum of American Art (NYC, “Programmed: Rules, Codes, and Choreographies in Art, 1965–2018”).