johnbcarpenter // heart // ibm // gesture // morph // study // info


New Ways of Seeing the Pattern film by USC BASA // released 10.08.2019

[ All work on this page is a collaboration with Dr Rusty Lansford from
Childern's Hospital Los Angeles (CHLA) and the USC Keck School of Medicine ]

There exists a critical need for new tools and techniques to visualize, filter, interpret, and share high-dimensional image datasets. New sensors allow scientists to collect such massive amounts of information from events such as cell migration in cardiogenesis that it can be difficult to perceive and understand all the hidden patterns and narratives using traditional visualization strategies. John Carpenter (Oblong and USC School of Cinematic Arts MA+P) and Dr. Rusty Lansford (USC Keck School of Medicine) demonstrate that immersive visualizations and spatial navigation can play an essential role in understanding complex multidimensional datasets.

Using g-speak, Rusty and John designed a 320° immersive workspace that permits gesture-based navigation of volumetric image sets from early heart formation. The application runs in real-time on five computers over 45 screens (93,312,000 pixels), and the ultrasonic emitters and wand provide precise spatial interactions with the system. The raw data is multispectral 4D (xyzt) confocal microscopy image sets (11,088 images: 126 images every 7.5 minutes for 11 hours) that underwent quantitative analysis to generate a multi-dimensional data set for 460.6K points. The goal of the work was to enable the visualization of individual cell movements en masse while maintaining the integrity of the data and providing an intuitive new way to navigate and explore early cardiogenesis.

Gesture -- which by its nature is spatial -- is an ideal mode for interacting with multidimensional datasets because it provides a seamless, intuitive way to connect with and navigate the datascape. Placing the data in a room-sized (human-scaled) pixel space provides new opportunities for visualization and allows for exploration of the data from multiple perspectives at different spatial and temporal resolutions. The social nature of the space, and the ability to have multiple users driving the interactions with the system creates opportunities for new forms of scientific collaboration and the communication of research.

John Carpenter and Dr. Rusty Lansford were recently awarded a grant from the USC Bridge Art + Science Alliance (BASA) to document their work. This video was directed by Lily Darragh Harty and produced by Kyle McClary in collaboration with Oblong, USC, and CHLA.

More info: Oblong blog post + Medium Article



Early visualization study looking at the data in Oblong Industries' lab space: 320° Immersion Room


Early visualization study looking at the data in Oblong Industries' lab space: Linear Digital Wall


Multi-user exploration of the 320° Immersion Room
john carpenter + dr rusty lansford (children's hospital los angeles + the USC keck school of mediecine)


Screen capture from the software that depicts an aggregate view of all cells and cell tracks in the data set after quantitative analysis using Imaris software

Proximity-based exploration of heart data in the 320° Immersion Room

View this post on Instagram

#photoshoot documenting some of the #immersive #heart #dataviz yesterday at #oblongindustries // thanks to @ilikeitwhatisit + @petehawkes for photos // #science #research with @4midbrain from @childrensla + @uscedu // #datavisualization #creativecoding #gspeak #cplusplus #immersiveexperience #immersiveinstallation #immersivedesign #interactiveinstallation #interactiondesign # uxdesign #gesture #gesturalinterface #spatialoperatingenvironment #ifyouhigh

A post shared by john carpenter (@johnbcarpenter) on


View this post on Instagram

photo wrapping up the heart #datavisualization filming last night... thanks so much to #USC BASA (bridging the #arts & #sciences) team @kyle.mcclary + director @lilyharty #pixels #immersive #spatial #design #oblongindustries + @4midbrain

A post shared by john carpenter (@johnbcarpenter) on


View this post on Instagram

depth-based #shader work and #color studies on the cell migration #datavis, reminds me of some of the generative #wildflowers, #artscience collab w rusty lansford #processing #creativecoding #biology

A post shared by john carpenter (@johnbcarpenter) on


View this post on Instagram

the compression on the previous video is making it difficult to see some of the details in the #datavis... these stills capture the triangle scaling better. the ideas that the larger triangles draw your attention to areas of faster cell movement #creativecoding #processing #biology

A post shared by john carpenter (@johnbcarpenter) on


View this post on Instagram

added trailing light paths to the yfp stained cell data #creativecoding #processing #shaders #dataviz #biology w rusty lansford

A post shared by john carpenter (@johnbcarpenter) on


View this post on Instagram

#interaction #study for the heart #cell migration #datavisualization #creativecoding #biology #science #design #shader #gspeak #cplusplus #dataviz

A post shared by john carpenter (@johnbcarpenter) on