Approximately 1% of all births involve congenital heart defects, and no two cases are alike.

Children born with a hole between the lower chambers of the heart—or whose single pumping chamber is kept alive by a leaky valve—require care tailored to their precise anatomy. Historically, the devices used by surgeons were rarely designed with individual children in mind.

“You have a unique child and an off-the-shelf device,” says Dr. Matthew Jolly, a cardiologist and researcher at Children’s Hospital of Philadelphia (CHOP). “Our job is to find the fit, and modeling allows us to do that before anyone even walks into the cath lab or operating room.”

AI creates accurate pediatric heart models in seconds

CHOP’s cardiac modeling services are built on MONAI, an open-source medical imaging framework co-founded by NVIDIA. MONAI uses images that children’s care teams already have, including CT scans, MRIs and 3D ultrasounds, to generate anatomically accurate heart models in seconds.

Workflows that once required up to four hours of work by a skilled researcher can now be completed quickly enough for routine clinical use.

This approach is spreading. More than 20 children’s hospitals across the United States currently have cardiac modeling programs. At Boston Children’s Hospital, more than half of all cardiac surgeries—approximately 500 per year—are supported by modeling. CHOP expects about 200 cases to be modeled this year.

Although the initiative began with cardiac care, CHOP is now aiming to apply the same tools to multiple areas through the hospital’s Idea Lab, part of the Morgan Research and Innovation Center.

From research to standard treatment: A 10-year journey

When Jolly joined CHOP in 2015, 3D echocardiography was just coming online. Although tools existed for modeling adult valves, few were designed for the complex and small anatomy he was treating.

His lab collaborated with the open-source community to build SlicerHeart, an extension of the open-source 3D Slicer software for visualizing, segmenting and analyzing 3D medical images. The team also began developing a workflow to model pediatric hearts and valves from multiple imaging modalities.

For many years, creating a single model required skilled research assistants to spend hours at a workstation. Machine learning has changed that.

Using MONAI Label and NVIDIA’s Auto3DSeg implementation, Jolly’s team trained a segmentation network on previous image-model pairs. The output meets the same quality standards as a model produced by a trained human, but is generated in seconds instead of hours.

“Machine learning has become just bread and butter,” Jolly said. “Once you have 10 to 20 image-model pairs, you can immediately start training and applying the model.”

3D heart modeling helps guide complex repairs

Clinical effects were immediate. CHOP now routinely models complex ventricular septal defects—holes between the lower chambers of the heart—before surgery.

One early case showed the value of the technology. A child had already undergone two unsuccessful repair attempts, and surgeons were unable to locate the defect using traditional methods. The 3D model revealed the anatomy, and the repair was successful on the first try.

In cases like this, cardiac modeling has moved from research to standard of care, Jolly said.

Near-real-time cardiac simulations with Newton and NVIDIA Warp

Visualization is not the only goal. Jolly’s team wants to understand not only what a child’s heart looks like, but also what happens when a device is placed inside it before a procedure begins. That is where Newton comes in.

Newton uses NVIDIA Warp, a Python framework for running physics simulations on the GPU. CHOP is working with NVIDIA and the open-source community to build a biomechanics-focused simulation framework using Warp that can be deployed on Newton, which was originally intended for simulation-based AI robot training.

Integrating these frameworks with 3D Slicer and SlicerHeart helps physicians understand the tissue properties that determine how a device is deployed in a specific patient. With GPU acceleration, the time required to simulate a cardiac device can be reduced from up to four hours—sometimes running all night—to near real time.

In practice, clinicians can compare how different devices fit a child’s anatomy and receive results quickly enough to make a decision within the same day.

CHOP has begun implementing functionality built on Warp and Newton for closure devices used to close holes in children’s hearts. The team hopes to apply similar methods to simulating transcatheter valves. The open-source architecture of Warp and Newton is connected with SlicerHeart, with the long-term goal of bringing real-time simulation into clinical workflows.

Digital twins could bring pediatric heart simulations into clinical workflows

Couplers using SlicerHeart and NVIDIA Omniverse, equipped with digital twin capabilities and OpenUSD, are also under development.

OpenUSD’s open-source 3D interoperability allows different data sources and solvers to be integrated into simulations built from patient images. These simulations can flow into virtual reality environments and take advantage of embedded systems.

A vision language model (VLM) could allow clinicians to intuitively interrogate and manipulate pediatric heart anatomy in a simulation before acting on it.

How open source supports care for rare and complex conditions

Approximately 2.4 million people in the United States have congenital heart defects. Historically, this population has been too small and diverse to attract investment from traditional device companies at the scale needed by families. No single company has built all the tools Jolly’s team needs, and no single institution can build them alone.

Open source offers a way forward.

SlicerHeart’s tools are free to use and build on. Researchers at Stanford University and Boston Children’s Hospital are providing additional tools alongside CHOP. A national consortium of children’s hospitals is currently forming to build the next generation of shared modeling infrastructure, using open source as the connective tissue across institutions.

“The population is too small to support traditional commercial development with normal economics,” Jolly said. “But this is such an important issue that people are trying to get behind it, both in the research community and in philanthropy. Open source defies traditional economics for small, heterogeneous populations by enabling collaboration and progress without barriers.”

NVIDIA’s investment in open platforms and collaboration with other companies—including MONAI for medical imaging AI, Newton for physics simulation and OpenUSD for 3D interoperability and virtual reality—gives teams like Jolly’s access to infrastructure maintained at industrial scale.

Children’s hospital labs can now benefit from continued innovation across the open-source community, gaining access to tools that would otherwise require an enterprise-wide engineering team to build and maintain.

Learn how MONAI is evolving medical imaging AI.