Umwelten

[photo credits: Puneet Jain]

Umwelten is an interactive installation that initiates a collaboration between a human, a non-human (a touch-me-not plant/Mimosa Pudica), and a machine-learning (ML) algorithm depicting a playful circulation of agency among the three entities over time. This work borrows the concept of ‘umwelt’ (environment) coined by the German biologist Jacob Johann Von Uexküll , inviting the human, the plant, and the ML algorithm to interact with one another in the same physical space but with different sensory envelopes (or umwelten). Uexküll believed that one should not perceive an animal as an objective entity (a physio-chemical being) but rather as a being that can be discerned based on their behavior in the environments they inhabit. Therefore according to Uexküll, an animal together with its environment form a whole system that he called an “Umwelt”.

[photo credits: Kaldi Moss]

The three entities, the human, the plant, and the algorithm initiate an interaction on a text-writing interface where they have an agency to change others’ past and future actions, and hence, their umwelts. For instance, as the human enters a text on the interface, both the plant and the algorithm alter the typography of the text depending on the umwelts they inhabit. Eventually such an interplay of interactions result into a dynamically morphing non-linear and confusing typographical text on the screen based on the collective actions taken by the three entities - symbolizing the overlap, collision, and entanglement of the umwelts of these organisms, Hence, depicting a dance of agency among the human, the non-human and the machine learning algorithm over time.

[photo credits: Puneet Jain]

This research intervention posits questions like - How do the umwelts of human and more than human entangle, overlap and are morphed with one another in such an exchange? Does such an invitation of bringing all these entities together also adds a confusion - a confusion emerging from different spatial and temporal perceptions of the three entities? If so, how do we see new futures where such an interweaving of entities is not only crucial but also help us to bring ourselves back to the ontological question of being V. becoming. Here is a experimental video describing the interplay between the three entities in “Umwelten”.

Concretely, through this project, the human and the non-human are brought in the same physical space (with different umwelts) to interact with one another and an ML algorithm, each having an agency to change others’ environment and hence, their umwelts. While the human changes the typography on the screen (using a text writing platform developed as a part of this project) through the typing speed and their choice of letters, both the plant and the (computer vision) machine learning algorithm alter this typography dynamically depending on the images clicked by a camera pointing to the plant. Such interactions manifested on the screen encapsulate a space where a non-linear and confusing typography symbolizes the overlapping, collision, and entanglement of the umwelts of these organisms – depicting the shifting of agencies among the human, the non-human and the machine learning algorithm over time.

The set up

[photo credits: Puneet Jain]

The set up consists of a human, a Mimosa Pudica (or touch-me-not) plant, and a camera (that acts as a machinic eye) that asks the human to type while the typography (color, font size, and spacing between the letters) dynamically evolves based on the typing speed of the human. Meanwhile, a camera points to the touch-me-not plant capturing images of the plant frame by frame, sending it to machine learning model trained on the image data set of the plant - classifying the plant into two categories (closed or open) with respective probabilities. While the leaves of the ’touch me not’ plant are susceptible to close or open depending on the external stimuli (sunlight, a human hand or a fly), the machine learning algorithm continously predicts the leaves to be open or closed acting as a “sensing” machinic eye. The probabilities of the classification again influence the typography on the screen thus disrupting the human sensory input-output on the screen. Hence, this dynamically evolving typography on the screen manifests in itself the different time scales that make up the umwelts of the different entities - the plant (which takes approximately 10 minutes to open its leaves once closed because of an external stimulus), the computational clock time of the predictive machine learning algorithm, and the physiological time of a human being.

[video conception and editing: Kaldi Moss and Puneet Jain]