Dr. med. Sandro-Michael Heining

Chirurgische Klinik und Poliklinik - Klinikum Innenstadt
Nussbaumstr. 20
80336 München

Publications with CAMP

2009
	L. Wang, J. Traub, S.M. Heining, S. Benhimane, R. Graumann, E. Euler, N. Navab Long Bone X-ray Image Stitching using C-arm Motion Estimation Proceedings of Bildverarbeitung fuer die Medizin (BVM 2009), Heidelberg, Germany, March 2009 (to be published) (bib)
	L. Wang, S. Weidert, J. Traub, S.M. Heining, C. Riquarts, E. Euler, N. Navab Camera Augmented Mobile C-arm: Towards Real Patient Study Proceedings of Bildverarbeitung fuer die Medizin (BVM 2009), Heidelberg, Germany, March 2009 (to be published) (bib)
2008
	C. Bichlmeier, B. Ockert, S.M. Heining, A. Ahmadi, N. Navab Stepping into the Operating Theater: ARAV - Augmented Reality Aided Vertebroplasty The 7th IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, Sept. 15 - 18, 2008. (bib)
	J. Traub, T. Sielhorst, S.M. Heining, N. Navab Advanced Display and Visualization Concepts for Image Guided Surgery IEEE/OSA Journal of Display Technology; Special Issue on Medical Displays, Volume 4, Issue 4, Dec. 2008 (bib)
	O. Kutter, A. Aichert, C. Bichlmeier, J. Traub, S.M. Heining, B. Ockert, E. Euler, N. Navab Real-time Volume Rendering for High Quality Visualization in Augmented Reality International Workshop on Augmented environments for Medical Imaging including Augmented Reality in Computer-aided Surgery (AMI-ARCS 2008), USA, New York, September 2008 (bib)
	C. Bichlmeier, B. Ockert, O. Kutter, M. Rustaee, S.M. Heining, N. Navab The Visible Korean Human Phantom: Realistic Test & Development Environments for Medical Augmented Reality International Workshop on Augmented environments for Medical Imaging including Augmented Reality in Computer-aided Surgery (AMI-ARCS 2008), USA, New York, September 2008 (bib)
	A. Ahmadi, N. Padoy, S.M. Heining, H. Feußner, M. Daumer, N. Navab Introducing Wearable Accelerometers in the Surgery Room for Activity Detection 7. Jahrestagung der Deutschen Gesellschaft f{\"u}r Computer-und Roboter-Assistierte Chirurgie (CURAC 2008) (bib)
	J. Traub, A. Ahmadi, N. Padoy, L. Wang, S.M. Heining, E. Euler, P. Jannin, N. Navab Workflow Based Assessment of the Camera Augmented Mobile C-arm System International Workshop on Augmented Reality environments for Medical Imaging and Computer-aided Surgery (AMI-ARCS 2008), New York, NY, USA, September 2008 (bib)
	L. Wang, J. Traub, S.M. Heining, S. Benhimane, R. Graumann, E. Euler, N. Navab Long Bone X-ray Image Stitching Using Camera Augmented Mobile C-arm Medical Image Computing and Computer-Assisted Intervention, MICCAI, 2008, New York, USA, September 6-10 2008 (bib)
	F. Wimmer, C. Bichlmeier, S.M. Heining, N. Navab Creating a Vision Channel for Observing Deep-Seated Anatomy in Medical Augmented Reality Proceedings of Bildverarbeitung fuer die Medizin (BVM 2008), Munich, Germany, April 2008 (bib)
	J. Traub, S.M. Heining, E. Euler, N. Navab Two camera augmented mobile C-arm – System setup and first experiments Proceedings of The 8th Computer Assisted Orthopaedic Surgery (CAOS 2008), Hong Kong, China, June, 2008 (bib)
	S.M. Heining, C. Bichlmeier, E. Euler, N. Navab Smart Device: Virtually Extended Surgical Drill Proceedings of The 8th Computer Assisted Orthopaedic Surgery (CAOS 2008), Hong Kong, China, June, 2008 (bib)
	M. Feuerstein, T. Mussack, S.M. Heining, N. Navab Intraoperative Laparoscope Augmentation for Port Placement and Resection Planning in Minimally Invasive Liver Resection IEEE Trans. Med. Imag., vol. 27, no. 3, pp. 355-369, March 2008 (bib)
2007
	C. Bichlmeier, S.M. Heining, M. Rustaee, N. Navab Laparoscopic Virtual Mirror for Understanding Vessel Structure: Evaluation Study by Twelve Surgeons The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007. (bib)
	C. Bichlmeier, F. Wimmer, S.M. Heining, N. Navab Contextual Anatomic Mimesis: Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007. (bib)
	J. Traub, H. Heibel, P. Dressel, S.M. Heining, R. Graumann, N. Navab A Multi-View Opto-Xray Imaging System: Development and First Application in Trauma Surgery Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2007), Brisbane, Australia, October/November 2007. (bib)
	C. Bichlmeier, M. Rustaee, S.M. Heining, N. Navab Virtually Extended Surgical Drilling Device: Virtual Mirror for Navigated Spine Surgery Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2007), Brisbane, Australia, October/November 2007. (bib)
	M. Feuerstein, T. Mussack, S.M. Heining, N. Navab Registration-free Laparoscope Superimposition for Intra-Operative Planning of Liver Resection 3rd Russian-Bavarian Conference on Biomedical Engineering, Erlangen, Germany, July 2/3 2007 (bib)
	P. Stefan, J. Traub, S.M. Heining, C. Riquarts, T. Sielhorst, E. Euler, N. Navab Hybrid navigation interface: a comparative study Proceedings of Bildverarbeitung fuer die Medizin (BVM 2007), Munich, Germany, March 2007, pp. 81-86 (bib)
	T. Klein, S. Benhimane, J. Traub, S.M. Heining, E. Euler, N. Navab Interactive Guidance System for C-arm Repositioning without Radiation Proceedings of Bildverarbeitung fuer die Medizin (BVM 2007), Munich, Germany, March 2007, pp. 21-25 (bib)
	C. Bichlmeier, T. Sielhorst, S.M. Heining, N. Navab Improving Depth Perception in Medical AR: A Virtual Vision Panel to the Inside of the Patient Proceedings of Bildverarbeitung fuer die Medizin (BVM 2007), Munich, Germany, March 2007 (bib)
	M. Feuerstein, T. Mussack, S.M. Heining, N. Navab Registration-Free Laparoscope Augmentation for Intra-Operative Liver Resection Planning SPIE Medical Imaging, San Diego, California, USA, 17-22 February 2007 (bib)
2006
	N. Navab, S. Wiesner, S. Benhimane, E. Euler, S.M. Heining Visual Servoing for Intraoperative Positioning and Repositioning of Mobile C-arms Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2006), Copenhagen, Denmark, October 2006 (bib)
	J. Traub, P. Stefan, S.M. Heining, T. Sielhorst, C. Riquarts, E. Euler, N. Navab Hybrid navigation interface for orthopedic and trauma surgery Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2006), Copenhagen, Denmark, October 2006, pp. 373-380 (bib)
	T. Sielhorst, C. Bichlmeier, S.M. Heining, N. Navab Depth perception a major issue in medical AR: Evaluation study by twenty surgeons Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2006), Copenhagen, Denmark, October 2006, pp. 364-372 The original publication is available online at www.springerlink.com (bib)
	J. Traub, P. Stefan, S.M. Heining, T. Sielhorst, C. Riquarts, E. Euler, N. Navab Towards a Hybrid Navigation Interface: Comparison of a Slice Based Navigation System with In-situ Visualization Proceedings of International Workshop on Medical Imaging and Augmented Reality (MIAR 2006), Shanghai, China, August, 2006, pp.179-186 (bib)
	S.M. Heining, P. Stefan, L. Omary, S. Wiesner, T. Sielhorst, N. Navab, F. Sauer, E. Euler, W. Mutschler, J. Traub Evaluation of an in-situ visualization system for navigated trauma surgery Journal of Biomechanics 2006; Vol. 39 Suppl. 1, page 209 (bib)
	S.M. Heining, S. Wiesner, E. Euler, N. Navab CAMC (camera augmented mobile c-arm) - first clinical application in a cadaver study Journal of Biomechanics 2006; Vol. 39 Suppl. 1, page 210 (bib)
	J. Traub, P. Stefan, S.M. Heining, T. Sielhorst, C. Riquarts, E. Euler, N. Navab Stereoscopic augmented reality navigation for trauma surgery: cadaver experiment and usability study International Journal of Computer Assisted Radiology and Surgery, 2006; Vol. 1 Suppl. 1, page 30 - 31. The original publication is available online at www.springerlink.com (bib)
	S.M. Heining, S. Wiesner, E. Euler, N. Navab Pedicle screw placement under video-augmented fluoroscopic control. First clinical application in a cadaver study International Journal of Computer Assisted Radiology and Surgery, 2006; Vol. 1 Suppl. 1, page 189-190. The original publication is available online at www.springerlink.com (bib)
	S.M. Heining, P. Stefan, F. Sauer, E. Euler, N. Navab, J. Traub Evaluation of an in-situ visualization system for navigated trauma surgery Proceedings of The 6th Computer Assisted Orthopaedic Surgery (CAOS 2006), Montreal, Canada, June, 2006 (bib)
	S.M. Heining, S. Wiesner, E. Euler, W. Mutschler, N. Navab Locking of intramedullary nails under video-augmented flouroscopic control: first clinical application in a cadaver study Proceedings of The 6th Computer Assisted Orthopaedic Surgery (CAOS 2006), Montreal, Canada, June, 2006 (bib)

Research Projects with CAMP

Research Projects

MeTaTop A Multi Sensory Table Top System for Medical Procedures A tabletop system in medical environments can be used for interactive and collaborative analysis of patient data but also as a multimedia user interface within sterile space. For preoperative planning physicians in charge with a particular patient meet to discuss the medical case and plan further steps for therapy. For this reason, they could collaboratively view and browse through all kind of available medical imaging data with the tabletop system. Alternatively such a system could be a central interaction device for all kind of equippment within the OR requiring user input, however, can not be operated by the sterile surgeon. We believe that the projection of all kind of user and information interfaces on a sterile glass plane would facilitate the clinical workflow. This project is strongly related to the Tangible Interaction Surface for Collaboration between Humans project.

Discovery and Detection of Surgical Activity in Percutaneous Vertebroplasties In this project, we aim at discovering automatically the workflow of percutaneous vertebroplasty. The medical framework is quite different from a parallel project , where we analyze laparoscopic surgeries. Contrary to cholecystectomies where much information is provided by the surgical tools and by the endoscopic video, in vertebroplasties and kyphoplasties, we believe that the body and hand movement of the surgeon give a key insight into the surgical activity. Surgical movements like hammering of the trocar into the vertebra or the stirring of cement compounds are indicative of the current workflow phase. The objectives of this project are to acquire the workflow related signals using accelerometers, processing the raw signals and detecting recurrent patterns in order to objectively identify the low-level and high-level workflow of the procedure.

Camera Augmented Mobile C-arm The problem of positioning mobile C-arms, e.g. for down the beam techniques, as well as repositioning during surgical procedures currently requires time, skill and additional radiation. The Camera-Augmented Mobile C-arm (CAMC) is able to speed up the procedure, simplify its execution and reduce the necessary radiation. The C-arm is extended by a CCD camera. The CCD camera is attached to the C-arm such that it is virtually at the same location than the x-ray source. This is done by a double mirror construction. A two step calibration routine has be done only once at the moment the camera is attached. Extensions for a lightwight, but complete navigation system for pedile screw placement or implant positioning is currently under investigation.

3D user interfaces for medical interventions This work group aims at practical user interfaces for 3D imaging data in surgery and medical interventions. The usual monitor based visualization and mouse based interaction with 3D data will not present acceptable solutions. Here we study the use of head mounted displays and advanced interaction techniques as alternative solutions. Different issues such as depth perception in augmented reality environment and optimal data representation for a smooth and efficient integration into the surgical workflow are the focus of our research activities. Furthermore appropriate ways of interaction within the surgical environment are investigated.

ARAV Augmented Reality Aided Vertebroplasty In today’s ORs more and more operations are performed employing minimally invasive procedures. Surgical instruments are inserted through a tiny cut on the patient’s skin, the port to the inside of the patient. In some cases endoscope cameras record video images of the operation site that are presented on a monitor. As a consequence of this technique, the surgeon’s field of view is divided into several work spaces, the monitor, the patient and information of medical imaging data presented on a third station. The missing direct view on the workspace complicates intuitive control of surgical tools. In contrast to open surgery the physician has to collect information from several fields of view at the same time and fuse information mentally to create a complete model of his working space, the operation site. The minimally invasive intervention vertebroplasty was determined as a suitable medical application to bring an Head Mounted Display (HMD) into the OR for augmentation of surgical instruments and medical imaging data. In-situ visualization with an HMD presents all available imaging data and navigational information in one field of view. The objective of vertebroplasty is the insertion of cement into weak and brittle vertebrae through a trocar for stabilization. In this case the view on the inside of the patient is not provided by an endoscope camera. However, since the operation is performed under a CT scanner, imaging data is permanently updated to check position of the trocar and amount of inserted cement. Imaging data is presented on a monitor and has to be mapped mentally by the surgeon on the real operation site.

Virtual Mirror: Interaction Paradigm for Augmented Reality Applications Augmented Reality offers a higher degree of freedom for the programmer than classical visualization of volume data on a screen. The existing paradigms for interaction with 3D objects are not satisfactory for particular applications since the majority of them rotate and move the object of interest. The classic manipulation of virtual objects cannot be used while keeping real and virtual spaces in alignment within an AR environment. This project introduces a simple and efficient interaction paradigm allowing the users to interact with 3D objects and visualize them from arbitrary viewpoints without disturbing the in-situ visualization, or requiring the user to change the viewpoint. We present a virtual, tangible mirror as a new paradigm for interaction with 3D models. The concept borrows its visualization paradigm in some sense from methodology used by dentists to examine the oral cavity without constantly changing their own viewpoint or moving the patients head. The virtual mirror improves the understanding of complex structures, enables completely new concepts to support navigational aid for different tasks and provides the user with intuitive views on physically restricted areas.

Improving Depth Perception and Perception of Layout for In-Situ Visualization in Medical Augmented Reality In-situ visualization in medical augmented reality (AR) using for instance a video see-through head mounted display (HMD) and an optical tracking system enables the stereoscopic view on visualized CT data registered with the real anatomy of a patient. Data can aligned with the required accuracy and the surgeons do not have to analyze data on an external monitor or images attached to the wall somewhere in the operating room. Thanks to a medical AR system like mentioned before, surgeons get a direct view onto and also ”into” the patient. Mental registration of medical imagery with the operation site is not necessary anymore. In addition surgical instruments can be augmented inside the human body. Bringing medical imagery and surgical instruments in the same field of action provides the most intuitive way to understand the patient’s anatomy within the region of interest and allows for the development of completely new generations of surgical navigation systems. Unfortunately, this method of presenting medical data suffers from a serious lack. Virtual imagery, such as a volume rendered spinal column, can only be displayed superimposed on real objects. If virtual entities of the scene are expected behind real ones, like the virtual spinal column beneath the real skin surface, this problem implicates incorrect perception of the viewed objects respective their distance to the observer. The strong visual depth cue interposition is responsible for misleading depth perception. This project aims at the development and evaluation of methods to improve depth perception for in-situ visualization in medical AR. Its intention is to provide an extended view onto the human body that allows an intuitive localization of visualized bones and tissue.

Laparoscope Augmentation for Minimally Invasive Liver Resection In recent years, an increasing number of liver tumor indications were treated by minimally invasive laparoscopic resection. Besides the restricted view, a major issue in laparoscopic liver resection is the precise localization of the vessels to be divided. To navigate the surgeon to these vessels, pre-operative imaging data can hardly be used due to intra-operative organ deformations caused by appliance of carbon dioxide pneumoperitoneum and respiratory motion. Therefore, we propose to use an optically tracked mobile C-arm providing cone-beam computed tomography imaging capability intra-operatively. After patient positioning, port placement, and carbon dioxide insufflation, the liver vessels are contrasted and a 3D volume is reconstructed during patient exhalation. Without any further need for patient registration, the volume can be directly augmented on the live laparoscope video. This augmentation provides the surgeon with essential aid in the localization of veins, arteries, and bile ducts to be divided or sealed. Current research focuses on the intra-operative use and tracking of mobile C-arms as well as laparoscopic ultrasound, augmented visualization on the laparoscope's view, and methods to synchronize respiratory motion.

Port Placement in Minimally Invasive Endoscopic Surgery Optimal port placement is a delicate issue in minimally invasive endoscopic surgery. A good choice of the instruments' and endoscope's ports can avoid time-consuming consecutive new port placement. We present a novel method to intuitively and precisely plan the port placement. The patient is registered to its pre-operative CT by just moving the endoscope around fiducials, which are attached to the patient's thorax and are visible in its CT. Their 3D positions are automatically reconstructed. Without prior time-consuming segmentation, the pre-operative CT volume is directly rendered with respect to the endoscope or instruments. This enables the simulation of a camera flight through the patient's interior along the instruments' axes to easily validate possible ports.

Teaching with CAMP

• Recent Advances in Computer Assisted Surgery: Hauptseminar (2008WiSe)
• Recent Advances in Computer Assisted Surgery: Hauptseminar (2008SoSe)
• Recent Advances in Computer Assisted Surgery: Hauptseminar (2007SoSe)
• Bildverarbeitung und Computer-Graphik mit C++: Projekt Systementwicklung (2007SoSe)
• Recent Advances in Computer Assisted Surgery: Hauptseminar (2006WiSe)
• Current Research and Challenges in Surgical Navigation: Doktorandenseminar (2006SoSe)
• Recent Advances in Computer Assisted Surgery: Hauptseminar (2005WiSe)