MIDL 2021

International Conference on
Medical Imaging with Deep Learning
5. bis 7. Juli 2021

Patient-Individual 4D Virtual Reality Simulation of Punctures and Radio-Frequency Ablations in Virtual Body Models with Breating Motion

Within the project we research methods for realistic 4D visuo-haptic VR simulation Virtual body models are animated by respiratory motion models. Applications are the patient-individual planning and training of punctures and radio-frequency ablation under respiratory motion. Static 3D image data sets of the patient are animated both by individual as well as mean-4D motion models that have been extracted from 4D image data and serve as a voxel-based description of real breathing movements. Using surrogate based 4D motion models regards also the variability of breathing in different respiratory cycles. By means of non-linear registration methods, the anatomical differences between the model and the patient's anatomy are compensated and the motion fields are transferred to animate the static 3D patient data. The 4D-motion models are integrated in a visuo-haptic framework the haptic-visual-driven interaction allows the puncture and ablation needle to interact with the breathing virtual body. For visuo-haptic 4D representation of moving 3D image data in real time special volume-based 4D rendering techniques are developed and optimized for the GPU. Furthermore near the diaphragm region, the effects of the respiratory movement on the biophysical simulation of RF ablations and the 4D needle path planning are compared to the planning and simulation in static 3D data. In addition to the evaluation of the different methods and system components, a user study about the VR training simulator enhanced by breathing motion is finally conducted.

Fig. 1: 4D image sequences showing needle and breathing motion.

The project is funded by the German Research Foundation (DFG: HA 2355/11-2).

Project team:

Dr. Andre Mastmeyer
Prof. Dr. Heinz Handels


Created at June 12, 2015 - 3:00pm by Mastmeyer.

Patient-specific Virtual Reality Simulation of Punctures Using Puncture Atlases

The aim of this project is to enable the patient-specific virtual reality training of minimal-invasive puncture procedures using special atlases. One topic of the project is the development of multi-altas methods for automated segmentation of the relevant organs and structures. Furthermore, efficient algorithms for volume-based haptic, visual simulation and soft tissue deformation are developed. Soft-tissue deformations should be computed in real-time using a volume-based simulation method. The implementation of computationally expensive algorithms on graphics hardware guarantees the real-time capability of the simulation algorithms.

Within the project a prototype of a VR simulator is set up (Fig. 1). The methods are developed and evaluated in cooperation with clinical partners.

Fig. 1: Immersive VR workstation with shutter glasses and haptic feedback device for puncture training.

The project is funded by the German Research Foundation (DFG: HA 2355/11-1).

Project team:

Prof. Dr. Heinz Handels
M.Sc. Dirk Fortmeier
Dr. Andre Mastmeyer

Created at August 26, 2011 - 3:22pm.

Finite Element Modeling of Respiratory Lung Motion


Respiratory motion is a main source of error in radiation therapy of thoracic tumors. In current clinical practice treatment planning is usually based on 3D CT imaging. Organ and tumor motion is accounted for by increasing safety margins which in turn increases the volume of healthy tissue to be irradiated. Development and optimization of methods to adequately account for breathing motion require detailed knowledge about respiratory dynamics. Consequently, computer aided modeling and model based simulation gain in importance. Within this project we aim at modeling respiratory (lung) motion taking the physiology of breathing as modeling starting point. As an approved method in biophysical model-ing we therefore apply Finite Element Methods (FEM). Since FEM enable to directly model certain (bio-) physical aspects by means of e.g. boundary conditions it provides for a high-precision analysis and simulation.


Starting point of modeling is the process of lung ventilation. During breathing the thoracic cavity is expanded by contraction of the diaphragm and outer intercostal muscles. This causes changes in the intrapleural pressure which acts as a force upon the lung surface: Lung expands, and (since the pleu-ral cavity is filled with a fluid) during this process the visceral pleura is sliding frictionlessly down the internal surface of the thoracic cavity.
This process is modeled by means of Finite Element Methods: A uniform negative pressure is applied to a lung surface representing some initial state of breathing (e.g. state of end-expiration). This causes the lung to expand. We limit expansion by a geometry representing the lung shape at a final state of breathing (e.g. end-inhalation); see fig. 1 for illustration. If there occurs a contact between the two geometries, the contact is modeled to be frictionless. For simplistic purposes lung tissue is assumed to be an isotropic linear elastic and homogeneous medium. As FEM software we use COMSOL Mul-tiphysics (FEMLAB, Sweden).


Fig. 1: For illustration of the modeling approach.

The modeling approach enables to generate patient specific models. Therefore we use 4D CT image data with high spatial and temporal resolution to create such models. Modeling accuracy is evaluated comparing simulated patient specific motion patterns of inner lung landmarks and corresponding mo-tion patterns observed in the 4D CT data. The influence of biomechanical and geometrical parameters (elasticity modulus, Poisson’s ratio; mesh quality) on the modelling process and modelling accuracy is evaluated.


Fig. 2.1: Surface mesh of a left lung at end-expiration. The small picture in the right upper corner represents the distance between the surface of the lung to be deformed and the limiting geometry surface (red: distance up to 20 mm; dark blue: approx. no distance).


Fig. 2.2: Corresponding surface mesh at end-inspiration. The arrows indicate the direction and magnitude of the motion. As in fig. 2.1 the small picture represents the surface distance between the deformed initial geometry and the limiting geometry.

Selected Publications:

  1. René Werner, Jan Ehrhardt, Rainer Schmidt, Heinz Handels
    Patient-Specific Finite Element Modeling of Respiratory Lung Motion using 4D CT Image Data
    Medical Physics, 36, 5, 1500-1511, 2009.
  2. René Werner, Jan Ehrhardt, Rainer Schmidt, Heinz Handels:
    Modeling Respiratory Lung Motion: A Biophysical Approach using Finite Element Methods
    In: Hu X.P., Clough, A.V. (eds.), Physiology, Function, and Structure from Medical Images, SPIE Medical Imaging 2008, San Diego, Vol. 6916, 0N-1-0N-11, 2008.

Project Team:

Dipl.-Inf. Dipl.-Phys. René Werner
Dr. Jan Ehrhardt
Prof. Dr. Heinz Handels


Created at July 15, 2010 - 1:41pm by Kulbe.

Virtual Reality Simulator for the Training of Lumbar Punctures

Lumbar punctures are performed for diagnosis and therapy by inserting a needle into the spinal chord of the patient to inject medicaments or extract liquor. The training of this procedure is usually done guided by experienced supervisors directly in contact with the patient. A virtual reality lumbar puncture simulator has been developed in order to minimize the training costs and the patients risk and to give new insights into the human anatomy.

We use a haptic device with six degrees of freedom (6DOF) to feedback forces that resist needle insertion and rotation. An improved haptic volume rendering approach is used to calculate the forces. This approach makes use of label data of relevant structures like skin, bone, muscles or fat and original CT data that contributes information about structures that can not be segmented.

Fig. 1: Haptic I/O-device with six degress of freedom (Sensable Phantom Premium).

A realtime 3D visualization with optional stereo view gives an overview of the punctured region. 2D visualizations of orthogonal slices enable a detailed impression of the anatomical context. The input data consisting of CT and label data and surface models of relevant structures is stored in an XML file together with haptic rendering and visualization parameters. In a first step the Visible Human Male and the Visible Korean Human data sets have been used to generate the virtual bodies.

Fig. 2: 3D-view showing the virtual user-steered needle and the virtual body.

Fig. 3: 2D-view showing orthogonal slices of the data. The needle position is shown by a yellow line.

Several users with different medical experience tested the lumbar puncture trainer. The simulator gives a good haptic and visual impression of the needle insertion and the haptic volume rendering technique enables the feeling of unsegmented structures. Especially, the restriction of transversal needle movement together with rotation constraints enabled by the 6DOF device facilitate a realistic needle behavior.

Selected Publications:

  1. Matthias Färber, Julika Heller, Heinz Handels
    Simulation and Training of Lumbar Punctures Using Haptic Volume Rendering and a 6DOF Haptic Device,
    In: Cleary K.R., Miga M.I. (eds.), Visualization, Image-Guided Procedures, and Display, SPIE Medical Imaging 2006, San Diego, SPIE Vol. 6509, 0F1-0F8, 2007
  2. Matthias Färber, Julika Heller, Hummel F., Gerloff C. Heinz Handels
    Virtual Reality based Training of Lumbar Punctures using a 6DOF Haptic Device,
    In: Buzug T., Holz D., Weber W., Bongartz J., Kohl-Bareis M., Hartmann U. (eds.), Advances in Medical Engineering, Springer Verlag, Berlin, 236-240, 2007

Project Manager:

Prof. Dr. Heinz Handels


Created at July 12, 2010 - 2:53pm by Kulbe. Last modified at July 27, 2010 - 11:00am.


Program of Study

Study Medical Informatics
at the University of Lübeck

read more ...


Susanne Petersen

Tel+49 451 3101 5601
Fax+49 451 3101 5604

Ratzeburger Allee 160
23538 Lübeck