This artwork is a visualisation of the moment of a well-formed shape of a wave, ideal for surfing, which retains its aesthetically pleasing shape forever, through a creative process that mixes reality and virtual reality.  It attempts to generate an experience in nature with its audience by presenting it and the artwork reaffirms the supple strength of human cognitive abilities and expresses a sense of human existence that will remain unchanged forever, even in a future where reality is in chaos.

The artwork mixes reality and virtual reality through a playful creative process that moves between digital and physical, 2D and 3D. It visualises the moment of a shape of a well-formed wave as a perpetual presence, that is aesthetically pleasing and gives us pleasant confusion at the same time.

It presents the fleeting moment of a wave, a form of natural force by blending human hand-creation, which generates randomness, with ocean physics, which always works correctly, in the chaos of real and virtual reality. And by building interaction between human beings and nature and reaffirming our identity from a trans-human perspective, like meditation, we can ensure that our sense of being is forever unchanged.

Modern technology makes it easy to replicate visual representations that are indistinguishable from reality. The metaverse has become one of the spaces in which we humans exist, and the definition of existence is being distorted. However, we live in ever-changing times and continue to evolve, recognising and accepting flexibly the chaos of reality and the unreality that technology brings with positive hope.

Waiting for the Wave in Metaverse

Solar System is an audio-visual live performance that marries data visualization and sonification. The installation system employs sidereal period data from the eight planets in our Solar System to generate a live soundtrack and visualization.

The mission of this endeavor is to use audible and visual media to allow humans to uncover and cognize with the always surrounding, yet invisible, Solar System. Each planet's unique data patterns (sidereal periods) contribute to this exploration by being translated with audible and visible media. The media stimulate audiences' cognitive senses, enabling audiences to tangibly experience the aesthetic wonder of the cosmic world, which is deeply interconnected with the entirety of the Universe and humanity on Earth.

The inspiration and message of the Solar System project resonate with this year's theme, Perpetual Presence. Throughout history, data from the Universe has been employed to uncover the existence of unseen worlds, the Cosmos. The discoveries gleaned from the process have influenced numerous sectors of our humanity, communicated through various languages and approaches. Scientists use numerical data and textual explanations to communicate information about the Cosmos. On the other hand, artists use sensory media, such as visuals and sounds, to narrate the story of the Cosmos. This art project aims to manifest the continuous and timeless presence of the Universe where it surrounds humanity, employing artistic transformation of universe data to illuminate its enduring existence. 

The sidereal periods of the eight planets are the raw data. These relative orbital periods were transformed into eight distinct sound loops, each with its unique cycle. This transformation was accomplished through the installation system incorporating algorithms. While producing a soundtrack, changes in the sound influence the visualization. The sound signals are transferred to the visual tool, impacting the visualized elements.

Solar System

Spotlight is an interactive data visualization project on the topic of government-imposed internet shutdowns. Although originally created in a book format, it is presented as a poster series for the purposes of this exhibition. This project explores the effects of internet shutdowns, some of their common triggers, and their relation to sensitive events such as protests and elections. The pages are made interactive through the use of glow-in-the-dark paint, which requires the viewer to use a UV flashlight provided to them along with each page to reveal additional information. The glow-in-the-dark-paint was not only an interesting material to experiment with in terms of creating light-based interactions, but it also symbolically complements the storytelling aspect of this project since internet shutdowns are often associated with phrases such as “being left in the dark”. The overall experience aims to replicate a sense of living through an internet shutdown by creating a low-light environment where the viewer is faced with a barrier in order to access some information as it is difficult or at times impossible to access certain sources of information during internet shutdowns. 

Spotlight

This paper presents "Body Cosmos", an artwork that creates a symbiotic relationship between the human body and a simulated cosmic environment through volumetric rendering and particle system. Drawing from DICOM data to simulate the human body and nebulae, we create an interactive and dynamic virtual environment. The real-time bio-data of users, collected via heart rate sensors and EEG devices, is integrated into the visualization, fostering a personal engagement and unity within this 'cosmos.' Body Cosmos provokes curiosity and expands users' imagination, and deepens their understanding of life's macrocosm and microcosm. This exploratory project redefines traditional perceptions of the human body in relation to the universe, creating a unique lens to view selfhood, embodiment, and identity. As we look to the future, the system's evolution will include incorporation of more bio-data sensors, an investigation into its potential psychological and physiological benefits, and the development of social interactive features through multi-user capabilities.

Body Cosmos: An Immersive Experience Driven by Real-Time Bio-Data

Body Cosmos is an artwork that explores the interconnectedness of the human body in a digital cosmic environment. We
create a surreal virtual reality that reflects the intricate structures of human anatomy and celestial nebulae, learning from volumetric rendering techniques. This immersive experience connects the macrocosm of the universe with the microcosm of human life, revealing the enduring resonance of our biological rhythms in the cosmic dance (Figure 1). “Body Cosmos” integrates real-time bio-data gathered through
heart rate monitors and EEG devices. This life data influences the visualization, creating an intimate, personal connection with the cosmos. The artwork transcends immediate presence, reflecting on the intertwined relationships that humans can form with digital data, thus nurturing a perpetual presence within the cosmic expanse.

Body Cosmos

This generative work draws data from 2,499 queer independent games for about 12 seconds each. Each game adds a unique shape and colour onto the canvas, and plays a unique string of notes. Over 8 hours, the canvas will be filled with infinite colours to celebrate LGBTQIA+ independent videogames.

History has always been queer. Through this generative visual and sound work, we aim to demonstrate the collective activism, movement, and creative expressions that queer folks are making to be visible, heard, and to say that we are here. 

But queer movement does not happen over night; queer resistance is accumulative and built over generations of self-sacrifice and self-acceptance. The multitude intersectionality of the unruly times slowly bleeds colour into the world, blends motion into the landscape, and accumulatively becomes a canvas of ever-moving colourful light.

Infinite Colours

In the novel The Vast Territory (El vasto territorio. Alfaguara, 2021; Caja Negra, 2023), by Chilean author Simón López Trujillo, a mycologist analyses the way a certain fungus infects the mind of a forest worker called Pedro. An abstract image of countless white dots against a black background, in the form of waves or a mountain range, appears to explain the infection: the genetic origin of Pedro’s language, when the fungus starts speaking through him. 
That image, included in the revised edition of the book, is also a depiction of the novel’s genetic origin, as it was generated by a visualization of different drafts of the novel. In every literary reading, two texts are involved: the actual text, that we can smoothly read with our eyes, and a second, invisible text, made of all the deletions, editions, and additions of words involved in the process of writing. This project conceives the visualization of The Vast Territory as a visual novel on its own that explores the unconscious of the book: that black, secret space, where the words involved in the writing process emerge as ghostly presences. There, the data of the previous draft are manifested in the following one, as a latent presence in the actuality of the text we read.
In this work, every white dot represents a word in the juxtaposed draft sequence. Its construction follows only two straightforward rules: (1) words are sequentially arranged in a horizontal line; (2) the vertical position of each word-dot is determined by its initial appearance within the entire sequence of drafts. The result is a “data-palimpsest” where each draft leaves its imprint on the next through their cumulative determination of the spatial order. By using the order of first appearance as the guiding principle, the visualisation emphasises the inherited structure of each draft from its predecessors, akin to looking at the fossil record or geological strata, with the most ancient elements appearing at the greatest depth.

The Vast Territory

We present an examination of organic spaces through print, book making, data, and machine learning. Artists created a book that explored the idea of organic and machine-made interpretations of a place that were generate using 3 different processes: a generative adversarial network, traditional printmaking and a camera. The artists found the results unexpected, discussing how the AI-generated image changed and complicated their understanding and constructed narrative about the original image and space. This caused them to think outside the box with over half of the students changing their print matrix and/or ink choices after seeing their AI-generated image. This supported the learning objective for students to collaborate with technology that is uninhibited by perspective or expectation, adapting and responding productively and creatively within a new framework. With the newfound accessibility to AI-generated images we encourage art teachers to explore how it fits into their curriculum.

Reinterpreted Spaces, an AI Printmaking Collaboration

"Plastic Landscape - The Reversible World" is an AI-generated 3D animated video design that shows the apocalyptic and surreal world surrounded by artificial plastic mixtures and objects in the ocean, urban city, Antarctica, and forest. Four different scenes are animated, with the camera panning slowly from left to right. Viewers can observe how the plastics are decomposed at a slower speed by looking at particle animations. In collaboration with engineers at Yonsei University, reverse neural network algorithms were applied to create style transferred images and those images were used for textures for surreal 3D objects in the world-making process. In doing so, the piece raises awareness about microplastics, which are a cause of environmental issues in the present day. Data like different types of plastics and speed of decomposition are applied in this animation and sound design. This scene animation is inspired by Ilwalobongbyeong (a folding screen) behind the king’s throne of the Joseon Dynasty. This animation depicts the twist of the landscape. Surreal objects/buildings in this animation made from plastic look beautiful and mesmerizing at first glance. However, the viewers can notice that they are the decayed objects and destroyed nature impacted by human beings. This new multi-sensory artwork addresses the awareness of plastic pollution through the apocalyptic lens. 
This work explores how humans and non-humans can be mingled in the plastic-impacted situations with an artistic lens and critical point of view. The piece brings audiences face to face with the reality of climate change, while at the same time foreboding a future that has been destroyed by microplastics; the piece will inspire audiences to take action of their own accord.

Plastic Landscape - The Reversible World

The lives and works of artists inscribe themselves into the cultural, material and media environments of their times, and some of them leave traces living with us in perpetual presence. Similar to the arts, the means to study and document such biographical inscription processes are under constant development: From the heroic accounts of Renaissance artists to feminist visualizations of co-existing urban subjects we have seen the arts and humanities’ methods portfolio evolve.

In this context, the artwork showcases how the intangible and ephemeral lives of artists can be brought into the physical presence of material exhibition spaces. To that end, it builds on the visualization framework of time geography and remodels biographies diagrammatically as three-dimensional trajectories, drawing unique curves into a translucent space-time sculpture.

Focusing on the life and work of the Austrian painter Herwig Zens (1943-2019), the data sculpture shows major steps and movements of his career. Over many decades, Zens kept a multimodal diary and etched logbook-like notes on art projects and his teaching activities, encounters with other artists, collectors and gallery owners, as well as travel impressions on copper plates. A complete print of the diary from 2005—which Zens referred to as “monster” due to its sheer informational and productional complexity—is considered the longest etching in the world at 40 meters. In lieu to this artifact, the sculpture translates the ‘monstrous’ complexity of a modern-day biography into a distant reading object, oscillating between an epistemic image and a kid’s toy—similar to snow globes which are used to contain memorable figures or sites, and which have been invented in Vienna at the end of the 19th century.

Acknowledgements: The European Union’s Horizon 2020 research and innovation programme partly supported related research and development under the project No. 101004825.

Monster in a Snow Globe: Biographies as Data Physicalizations

The Heart is a site-responsive, slow Artificial Intelligence artwork to be lived with over decades. It reveals the pulse of a superorganism: the community visiting, living, and working in Melbourne Connect, a city-block size building, home of businesses, university departments, a kindergarten, accommodation, and a science museum. The Heart beats indefinitely for and with the life of the building and its community.

The Heart is connected to 4800 Building Information Modelling sensors. These monitor CO2, humidity, occupancy, movement, light, and more. The building adjusts the environment to create the optimum conditions for human comfort and safety. Normally, the automated work of building sensors and systems is dispersed and imperceptible. The Heart stages the building’s ‘sensations’ in a way people can perceive and begin to empathise with. It does this by taking form in Melbourne Connect as a 10-metre-tall volume of brass droppers, reconstituted brick fragments, and LEDs in the shape of a giant human heart.

The Heart operates within a perpetual present, responding live to the activity occurring in proximity to its sensors in perpetuity, and responds through changes in heart rate and animation. Visitors can also interact directly with The Heart by touching a heart rate monitor and adding their pulse to the building.

The Heart uses Manifold Learning to become accustomed to the live ‘sensations’ it receives from its ‘body’ (the building) over the course of each day. At night when The Heart sleeps, it creates a new manifold merged with previous days’ manifolds. When awake, it compares what it ‘feels’ in the present against its habituated experience of the past.

The Heart also uses a Generative Adversarial Network (GAN) to develop a capacity to anticipate what it is about to ‘feel’ based on its ‘lifetime’ of habituated experiences. 

Visit: 700 Swanston Street, Melbourne

Artist: Robert Walton 
AI Programming: Zaher Joukhadar 
Design: ADDITIVE 
Heart OS: Brad Hammond

The Heart (Video Documentation)

“Bitter Data” transforms 100,000 distress postings from Chinese social media into a multi-sensory experience using data edibilization. We’ve mapped distress data quantity to the bitterness and color of tea through data analysis and experimentation. Participants taste, smell, and observe 11 cups of tea, each embodying a year’s distress data, in our workshop. Their facial expressions, recorded upon tasting, visually indicate emotional states. This project explores benefits and pragmatic solutions to challenges of data edibilization.

Bitter Data: An Exploration into Data Edibilization of Negative Emotion

This project aims to transform the nano-scale of a striking biological phenomenon, the relationship between the SARS-CoV-2 virus and human molecules, into an interactive audiovisual simulation. In this work, Atomic Force Microscopy (AFM) touching and imaging a single molecule measures the interaction between the spike protein of SARS-CoV-2 and human cellular proteins and measures the dynamic of the spike protein. We create a comprehensive scientific model based on diverse datasets and theories presenting a real-time interactive complex system with efficient rendering and sonification using a single C++ platform. This project invites the audience into an immersive space where they can control the behavior of biomolecules, allowing them to intuitively perceive biological properties. This project is not only a demonstration of scientific data but also attempts to look at the interspecies relationship in parasitism which particularly deals with our current and post-pandemic life with coronavirus and how we might control our coexistence in a virtual space.

Parasitic signals: Multimodal Sonata for Real-time Interactive Simulation of the SARS-CoV-2 Virus

This project aims to transform the nano-scale of a striking biological phenomenon, the relationship between SARS-CoV-2 coronavirus and human molecules, into an interactive audiovisual simulation. In this work, the interaction data between the spike protein of SARS-CoV-2 and human cellular proteins is measured by Atomic Force Microscopy, which can touch and image a single molecule. We are creating an interactive audiovisual installation and performance from a set of interaction data. The audience is invited to an immersive space where they can control the biomolecules’ behavior so that they can intuitively recognize the biological characteristics. This project is not only a demonstration of scientific data but also tries to look at the interspecies relationship in parasitism. This project particularly deals with our current and future life with coronavirus and demonstrates how we might control our coexistence in virtual space.

Parasitic Signals: Coexistence with the SARS-CoV-2 virus

Loneliness and isolation are eternal emotions in human beings. Technological advancements create ample avenues, like social medias, for individuals to articulate themselves and record emotions. However, the sense of loneliness has never vanished, as their expressions are easily buried in the digital stream. We analyze social media posts that express loneliness during holiday seasons but receive few responses. By superimposing digital charts on physical models, we visualize these lonely posts and generate the island of loneliness. We aim to reveal the complexities of human emotions in the digital age and reflect on the interconnections between technology, solitude, and social communication.

Posts with No Response: The Island of Loneliness

Eco-Mending juxtaposes old and new to tell a story of ecological regeneration. This project finds stories of human-mediated restoration in longitudinal data, and stitches modern data into imagery from the past, creating decorative wall and ceiling hangings.
With perpetuality of data, comes the possibility of changing it. In this collection of art pieces, Eco-Mending pairs past ecological destruction with present and future projections of data that highlight successful ecological reconstructions.

Eco-Mending

Latent Prism is a visually captivating and thought-provoking piece that incorporates artificial intelligence (AI) and data visualization. It presents a projection of an imagined environment, created through a generative adversarial network (GAN) trained on thousands of aerial photographs from royalty-free stock photo websites. The sculpture takes the form of a polished transparent lucite prism, within which layers of translucent mylar film are suspended. These films display frames extracted from an AI-generated video known as a “latent walk,” showcasing undulating ocean and forest landscapes captured from an aerial perspective. Frames are selected at regular intervals along the linear interpolation of images from the generative model, such that light from below is still able to permeate up to the viewer. The aggregated effect of these layered video frames results in an eerie visual sensation of peering down at a forested landscape that is being submerged in water. Surrounding the prism is a haphazardly piled 120ft. (36.5m)-long roll of credits, listing the names and photographers for every image used to train the model on a 2.5in (6.35cm) wide strip of drafting paper.

Latent Prism

We are constantly decoding and assimilating the clues our from our natural environment provides through sensory experiences and recoding them into our own experiential repertory.  Research of environmental and climate-based change through data measurement and analysis separates us from the sensory experience of our immediate environment. 
Our research objective is to facilitate the merging of our sensory perception of the environment with the data representing that environment, through the development of a new visual vocabulary that speaks both to the scientific and artistic representations of our environments, translating data into a sensory perception, making us perpetually present in the deeper, richer reality of our environment.

Current visualization glyphs used in climate sci¬ence represent data with simple geometric primitives. These generic geometries bear no relationship to the sensory aspects of our environmental experiences, and they fail to evoke any personal connection to the data. 

Here we present handcrafted organic 3D glyph designs, drawn specifically from the forms, textures, metaphors, and narratives we observe in nature. We demonstrate how this vocabulary can not only accurately and efficiently depict data but also bridge the gap between our current experiences with environmental data and our sensory experience and memory of our environment. 


Associative Forms for Encoding Multivariate Climate Data

As humans, we are perpetually present in nature. We experience
nature through sensory perception of everything that surrounds us:
the breeze on our skin, the warmth of the sun, the scent of flowering
plants, the tactile variations in the surfaces of our environment. We are
constantly decoding and assimilating the clues our natural environment
provides through sensory experiences.
In contrast, expression of environmental and climate-based change
through data measurement and analysis separates us from the sensory
experience of our immediate environment. The current simulations and
depictions of climate change data – ocean chemistry, coupled ecosystem
models, the atmosphere far above and the ocean currents far below –
are viewed as separate from our personal environments: distant, not
part of us, not connected to us, and unrelated to our perpetually present
sensory experience.
The separation of our emotional connection to nature from our
intellectual study inhibits our ability to absorb the growing impact
on our daily lives. The work presented here seeks be a conduit assisting
us to close the gap between our human emotional connection to nature
from our intellectual study and the sterile analytical imagery we use to
understand the invisible physical changes underway.
We rely on the interplay of art, technology and science, and the
dance of these disciplines as they augment one another to create an
emotional connection between the audience and the data. Prior work
focused on building out artistic vocabulary for clear, engaging science
exploration and communication. The series presented here, melds the
science inquiry, data representation, artistic contextual content to create
new layers of meaning and connection.


Contributions

Artworks, Pictorials, and Papers