New materialities in the digital age, narrated through sound experiments
By: M.A. Doreen A. Ríos
For almost 30 years the consumption, creation and distribution of digital material among the masses has been made possible through the Internet. This has meant that, as proposed by William J. Mitchell, the WWW has taken charge of “replacing spaces, reconfiguring the social body and the architectural contexts.” (Mitchell 2003). “Mitchell offers an analysis about how people use the metaphors and paradigms of creating space to inform the construction of their electronic agoras. For Mitchell, the Internet is the first serious contender to do some of the things that Architecture has always done: create places to do, to see and to be seen.” (Lunenfeld 1999). Taking this reflection as a starting point, it is interesting to ask what happens with these three decades of digital materialities that move extensively within a series of platforms and devices that, now more than ever, are deeply integrated in our daily lives.
This essay focuses on addressing the contemporary notion of new materialities, emphasizing those that have been enunciated within different contexts and have tried to make sense of the material exchanges of contemporary creative production and, especially, to establish a new language that encompasses its complexities.
New materialities in the digital age
New materialities, as seen from creative production, are a mixture between the use of digital technologies, computerized drawing, digital modelling, etc. and all that this entails within a broader creative process. This essay focuses on the use of sound experiments as examples of each concept to make a deeper case about how, within such classic material transactions as sound and music, there are sensory layers that, in contemporary production, adopt different forms. On the other hand, it is key to state that the condition of digital objects, for purposes of this research, moves away from the premise of thinking of them as immaterial because, although this notion comes from the fact that are somewhat intangible, they are always accessed through an interface that works as a link to traditional objectuality. Taking this as a starting point, I will define three concepts that are located within the notion of new materialities and I will complement them with examples that come from sound experiments.
One of the first concepts that encompasses the idea of new materialities is that of hypermateriality. This term was coined by Bernard Stiegler in his book The Hypermaterial Economy and Psychopower, to refer to the processes of exchange between digitization and rematerialization of an object, i.e in architectural design when the final outcome is eternally linked to the use of CAD technologies. Stiegler seeks to respond to what happens when a material transaction of this nature is carried out, especially in areas that have to eventually point towards the materialization of a tangible object, but within which there is a niche of production that is 100% digitized.
"I call hypermaterial a complex of energy and information where it is no longer possible to distinguish its matter from its form [...] a process where information – which is presented as form – is in reality a sequence of states of matter produced by materials and apparatuses, by techno-logical devices in which the separation of form and matter is totally devoid of meaning."(Stiegler, 2009). If we think about a simple design process carried out in 2018, it will likely start with several hand-drawn sketches + the creation a series of mockups and preliminary models + a digitized version of these ideas + a final digital model and its details + the final rematerialization seen in its construction/production process. It is important to point out that sometimes the digitization process does not necessarily happen because the creators consciously seek it but because the production conditions that surround them push it to be part of a mandatory process. Within other areas of creative production these conditions are more or less the same, especially for objects than plan to be produced more than once, where if we want that the original design is preserved, there should exist a key file which will not change its form no matter how many times it is replicated. Under the current production conditions, this key file is mainly digital because its forms will not change without a direct intention.
Therefore, it is not surprising that there are situations in which it is possible to identify aesthetic elements of the software used to create the object in question (building, poster, table, etc.). This is a consequence of how, through the daily use of these computerized programs, we become aware of their possibilities and their limits (i.e how it might be harder to work with organic shapes in Revit than in 3DMax). This, although it is already a common theme within areas such as Architecture, is clearly seen in multiple areas of creative production. Another big example of this is Graphic Design where it is common to identify not only the tools with which a design was created but also, probably, the process that was followed to develop its final form. From this perspective, hypermaterials advocate to the notion that it is not only the final product that represents its complete objecthood but also its digital file. This goes hand in hand with understanding these objetcs as expanded materials, where they have a physical form but also a digital file which can be mutable, sharable, scalable, etc. In other words, a single object that can have diverse material conditions which allows it to be interacted with in various spaces.
What Stiegler raises here is that if we do not think of these objects recognizing their different materialities, we are losing part of what they are or can be. In this sense, the concept of hypermateriality seeks to make sense of this new material condition as an inseparable whole.
A good example of this is the project Speculative Communications (2017 - 19) by the collective Interspecifics. This project transforms the movement of bacteria into sound and image with the possibility of manipulating it live onstage, in the words of its creators: "A machine that is able to observe and learn from a microorganism to create an audiovisual score based on the patterns it finds in its behavior. This project focuses on the creation of an artificial intelligence, which recognizes repetitive organized behaviors in biological entities and transforms them into impulses that provide specific audiovisual gestures in order to achieve a self-generative composition that responds to a logic that this intelligence builds over time. To achieve this, it is necessary to develop a device for the search and transmission of analogue signals from microorganisms and an audiovisual platform that allows the expression of biological signals which can also take charge of their own biological maintenance. The result is transmitted in real time through a digital channel creating the possibility of observing the coevolution process of the machine live. "(Interspecifics, 2017).
In technical matters, and approaching its condition of hypermaterial, what happens here is a cycle in which part of a tangible biological entity is digitized through a series of impulses driven by external stimuli for later rematerialize in image and sound. It is precisely the whole of its parts what gives it meaning, that is, there is no difference in the importance of its tangible parts compared to its intangible parts, there is a balance between its components that endow it with a broad language that develops as these material exchanges take place. In other words, "this project seeks to highlight the gestures of real time expressed in life and of that life as a generator of change and tangible instability from both visual and sound representations. Matter has in itself an expressive performativity so we seek to create an interface that allows the other to express its complexity and diversity through a language that is tangible for both a machine and for us humans. A narrative that is constructed from the intersection of science, fiction, life and engineering and that seeks to appropriate the social imaginary to create links with what seems so alien, but that conforms in itself the ontology of our species. "(Interspecifics, 2017). That is why its objectives as a piece and as a material exploration give Speculative Communications what Stiegler emphasizes in his conceptualization of hypermateriality.
This term was coined by Christiane Paul in 2015. According to Paul "the concept of neomateriality is proposed here to describe an objecthood that incorporates networked digital technologies, and embeds, processes, and reflects back the data of humans and the environment, or reveals its own coded materiality and the way in which digital processes see our world. " (Paul, 2015). Christiane Paul's critique of what Stiegler proposes with hypermaterials is that this concept does not take into account what is gained and lost within the material exchanges between rematerialization and digitization. If we are not aware that in this cycle of material exchanges something changes, we are losing sight of all the processes that takes place inside black boxes and, by doing so, we perpetuate its condition of little to no transparency.
Paul proposes the inclusion of this terminology from a territory of critical thinking and especially from an era, 2015, where digital devices are no longer seen as allies. By this year we, as users and consumers, are mildly aware of certain irregularities and injustices that are propitiated from digital platforms and devices. From corrupt facial recognition systems, hypervigilance and loss of privacy, the debate surrounding the early adoption of such technologies is part of daily life. Moreover, the areas that concern this debate have expanded exponentially and have found an interesting field for exploration within contemporary artistic practices.
It is this mixture of ideas among the hypermaterialities that Bernard Stiegler proposes along with the New Aesthetic of James Bridle that spark the notion of neomateriality. Bridle is an artist and media theorist who, for several years, has been very critical about the speed with which we accept certain conditions of use within the technologies that surround us. Hence, what the New Aesthetic proposes is related to the recognition of digital systems off-screen, but it also has to do with the way in which we recognize certain type of images which carry a strong aesthetic imprint that relates them to digital platforms and software. James Bridle states that there are aesthetic aspects in the visual content that we consume on a daily basis that have created a language that allows us to access information that goes beyond the content of the image in question (for example its origin, some processes behind its acquisition, location within the web, etc.).
Returning to Bridle "(...) the New Aesthetic project is undertaken within its own medium. It is an attempt to ‘write’ critically about the network in the vernacular of the network itself: in a Tumblr, in blog posts, in YouTube videos of lectures, tweeted reports and messages, reblogs, likes, and comments. In this sense, from my perspective, it is as much work as criticism: it does not conform to the formal shapes—manifesto, essay, book—expected by critics and academics. As a result, it remains largely illegible to them, despite frequent public statements of the present kind. "(Bridle, 2014).
Its exploration, and link with the concept of neomateriality, comes from understanding a condition that arises from a broad, and almost uncontrollable, use of images (mainly) that navigate within digital platforms and dominate the way we consume information. The question posed by the New Aesthetic about what do we understand beyond the digital image we observe? is embedded with the idea of hypermaterials and gives rise to the definition of neomaterials where Paul states that we should look for a cognitive process beyond the objective. This refers to recognizing a cycle, again, of materialization - digitization - rematerialization where each change should reveal what changes within the object in question, so "neomateriality describes the embeddedness of the digital in the objects, images, and structures we encounter on a daily basis and the way we understand ourselves in relation to them. It finds different kinds of expression within contemporary culture and artistic practice in the form of objects or artworks that 1) use embedded networked technologies, reflecting back their surrounding human and non-human environment; 2) reveal their own coded materiality as part of their form, thereby becoming themselves a residue of digital processes; 3) reflect the way in which digital machines and processes (seemingly autonomously) perceive us and our world." (Paul, 2015).
One piece that clearly reflects this condition is The Dark Age of Connectionism: Captivity (2017) by Wesley Goatley who "(...) explored the limitations of devices such as the Amazon Echo through a multi-channel audio and sculptural installation. Through this, it suggested new forms of behaviour for living amongst these technologies, and how the opacities encountered in their function can be responded to."(Goatley, 2017). This installation exposes how some personal assistants marketed widely not only detect our voices but also our movements. This installation has many layers - from the configuration of the circular physical space, through the pre-recorded questions designed to be launched every time these devices detect a movement and even the way in which the viewer must learn to communicate with the piece in order to experience it - are part of a set of actions that expose the way in which the devices in question work. There is in its creation process a deep understanding of the physical parts of this personal assistants, at the same time that there is a digital reconnection for its use and a rematerialization in the exhibition space. In other words, what this piece teaches us how to be invisible to these machines, an act that unites the characteristics of what Christiane Paul proposes within the idea of neomateriality.
With this proposal we navigate closer to speculation because, unlike the two previous concepts, part of its affective relationships are rooted in its form and not so much in its materials. This terminology appears within the essay Metamaterialism by Timur Si-Qin, where he establishes a series of links with the ideas put forward by Manuel de Landa, where he states that "(…) the token material entity of current textual theory—just to back track a bit—the ‘60’s in France was the great period of virtualization. Everything became text. Kristeva and Derrida and so on were just talking about intertextuality. Even the weather doesn’t exist, it is what we make of it, what we interpret of it. Everything became virtual in a way. Baudrillard says that everything is just simulacra, just layers of neon signs on top of layers of television images on top of layers of film images and more and more virtual stuff. The computer games and simulations. We need an antidote to that. We need to acknowledge that we’ve built these layers of virtuality and that they are real, they are real virtual. They might not be actual but they are real still but that all of them are running on top of a material basis that ultimately informs the source of power and the basis of society."(De Landa, 2006). Hence, a metamaterial is one that can become or take another's place regardless of its structure, Timur uses this term to refer to the materiality of art as a concept.
These links are generated from entities that could be understood as shape-shifters since they are able to join a discourse in which they take the place and functions of an entity different them, but are able to cope in a functional way. What Si-Qin establishes within the concept of metamaterialism, and taking the meta prefix to refer to an infinity of possibilities, is that these materials are capable of adopting various forms depending on the situation. Under this logic, and taking from non-human intelligences, this terminology can be attributed to bots and AIs. Which, in some extent, adopt a series of behaviors that allow them to act like other entities, without necessarily fully replacing what they imitate. In that sense, a Twitter bot is using a platform designed for humans, and what it does is emulate their beheaviour to offer a series of interactions that are understood as natural within this context. However, if we wanted to question or create a dialogue with this bot based on its attributions, we could not do it, since its functions are limited to a single type of interaction possible within this platform. Taking a more complex scenario, we could argue that it is a similar case what happens with AIs where they are able to carry out more than one type of interaction within their space and learn from them. However, at the end of the day, this AI will respond only to the data inside its neural network, and in that sense, its interventions will be equally limited to the objectives of its creator.
When we think about how these relationships are produced and how they evolve, we must necessarily consider the new affective conditioning, the development of non-human intelligences and the fulfillment of creative qualities as the basis for understanding their materialities and their role within the ecology of contemporary production. A project that includes the latter is Spawn (2016 - 19), by Holly Herndon in collaboration with Mat Dryhurst. Here, Holly Herndon researches and humanizes technology through music that sounds intimate and alien at the same time. If we are going to locate ourselves in a digital environment, she seeks to communicate instead of succumbing to our fears surrounding the AI. This project by Herndon and Dryhurst has been in process for two years and has revealed its activations, mainly through sound, where the artificial neural network reproduces the voices of its creators and interacts with the sounds of the spectators in the installation. Among the central questions of this project are: should we back away from technological developments and establish limitations in the capacity of non-human entities, such as Spawn, to witness the things we want to protect? Is mimicry the logical end point of a new data-based musical ecosystem, designed to give people more of what they like, with less and less emphasis on the origin or identity of their creator(s)? Or is there a more interesting and symbiotic path of machine / human collaboration where these developments are seen as an opportunity to reconsider who we are and to dream of new ways of creating? The latter, and its basic form, is where this project creates a direct relationship with the proposal of Si-Qin around the search for resilient and adaptable nodes within the notion of metamateriality.
It is clear that the digital era has opened various doors of opportunity for creative production. However, if we are unable to understand these processes and enunciate what they represent for the contemporary creative ecosystem, it is likely that, on one hand, we are not getting the most out of these technologies, and,on the other, we are not being able to recognize their limitations, vulnerabilities and sociopolitical consequences. The latter is crucial for technological innovation, in addition to representing a fertile area for the integration of creative thoughts for greater accessibility to these technologies and a critical questioning of their imminent ubiquitous domain that hides behind black boxes which are waiting to be emptied.
Bishop, R., Gansing, K., Parikka, J. et. al. (2016) Across and Beyond: Post-digital Practices, Concepts, and Institutions. Transmediale festival reader.
Cook, S., Graham, B. and Gfader, V. (eds.) (2010) A brief history of curating new media art: Conversations with curators. Berlin: The Green Box Kunstedition.
Cook, S., Graham, B. (2002) Curating New Media. Art Monthly. Issue 261. Available at: http://www.artmonthly.co.uk/magazine/site/article/curating-new-media-by-beryl-graham-sarah-cook-november-2002/ (Accessed: 22 November 2017).
Cook, S., Graham B. (2010) Rethinking Curating: Art After New Media. The MIT Press..
Cramer, F. (2013) What is ‘Post-digital’? A Peer Reviewed Journal About Post-Digital Research. Available at: http://tm-resource.projects.cavi.au.dk/?p=1318 (Accessed: 24 January 2018)
Cueto, B., Hendrikx, B. (eds.). (2017) Authenticity? Observations And Artistic Strategies In The Post-digital Age (Making Public). Publisher Valiz.
Hookway, B. (2014) Interfase. The MIT Press
Paul, C. (2015) From Immateriality to Neomateriality: Art and the Conditions of Digital Materiality. ISEA 2015
Paul, C. (2010). The Myth of Immateriality -- Presenting & Preserving New Media. Available at: http://intelligentagent.com/writing_samples/CP_Myth_of_Immateriality.pdf [Accessed 26 Jan. 2018].