Prev     /     Next


Parasitic signals: Multimodal Sonata for Real-time Interactive Simulation of the SARS-CoV-2 Virus

Myungin Lee, Sabina Hyoju Ahn, Yoojin Oh, JoAnn Kuchera-Morin

Installation of Coexistence with the SARS-CoV-2 virus
Schematic image of components in the simulation with the actual sizes
Four stages of SARS-CoV-2 infection toward the lung along the airway: Appearance, Diffusion, Penetration, and Binding for Entry
The installations are made in various environments and diverse audiences are invited for the interaction.
From (a) spike trimer of the SARS-CoV-2 virus observed with AFM to (f) rendered dynamic spikes with the labels
Overall structure of multimodal design
Linear-scale spectrogram of an exemplary narrative over 4 minutes

This project aims to transform the nano-scale of a striking biological phenomenon, the relationship between the SARS-CoV-2 virus and human molecules, into an interactive audiovisual simulation. In this work, Atomic Force Microscopy (AFM) touching and imaging a single molecule measures the interaction between the spike protein of SARS-CoV-2 and human cellular proteins and measures the dynamic of the spike protein. We create a comprehensive scientific model based on diverse datasets and theories presenting a real-time interactive complex system with efficient rendering and sonification using a single C++ platform. This project invites the audience into an immersive space where they can control the behavior of biomolecules, allowing them to intuitively perceive biological properties. This project is not only a demonstration of scientific data but also attempts to look at the interspecies relationship in parasitism which particularly deals with our current and post-pandemic life with coronavirus and how we might control our coexistence in a virtual space.


Artists bio
Image of undefined

Myungin Lee

Dr. Myungin Lee is a researcher designing multimodal instruments based on scientific theory, composition, signal processing, machine learning, and gestural interface. He pursues cohesive multimodal instrument design and compositions by establishing crossmodal correspondence between modalities, including physics, audio, graphics, and interface. He is a faculty member in the Immersive Media Design program at the University of Maryland, College Park.

Image of undefined

Sabina Hyoju Ahn

Sabina Hyoju Ahn is an artist engaging with various media represented through auditory perception, tactile sense, visual elements and a mixture of digital and analog technology. Her research seeks to find hidden rules and patterns in natural elements and multi-layered relationships between human and non-human beings by translating imperceptible data in natural elements into different perceptual experiences. In her work, biological materials are often used, combined or connected to machines, and transformed.

Image of undefined

Yoojin Oh

Yoojin Oh is currently senior scientist at the Institute of Biophysics, Johannes Kepler University Linz with a technical focus on microscopic imaging in nano-scale. She studied physics by developing scanning probe microscopic techniques and obtained a doctoral degree in Korea. In 2009, she moved to Europe as a post-doctoral researcher and applied microscopic techniques to living organisms and pathogenic systems. With her broad scientific activities, she is also interested in interdisciplinary artwork by collaborating with artists.

Image of undefined

JoAnn Kuchera-Morin

Dr. JoAnn Kuchera-Morin, Director and Chief Scientist of the AlloSphere Research Facility, is a Professor of Media Arts and Technology and Music at the University of California, Santa Barbara. She created and was Chief Scientist of Digital Media for the University of California. This led to the creation of a Digital Media Center in the California NanoSystems Institute and the AlloSphere instrument designed for immersive, interactive scientific and artistic investigation of multi-dimensional datasets.

Previous work     /     Next work