Development of an Intraoperative Pipeline for Holographic Mixed Reality Visualization During Spinal Fusion Surgery

Vivek P. Buch, Kobina G. Mensah-Brown, James W. Germi, Brian J. Park, Peter J. Madsen, Austin J. Borja, Debanjan Haldar, Patricia Basenfelder, Jang W. Yoon, James M. Schuster, Han Chiao I. Chen

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


Objective. Holographic mixed reality (HMR) allows for the superimposition of computer-generated virtual objects onto the operator’s view of the world. Innovative solutions can be developed to enable the use of this technology during surgery. The authors developed and iteratively optimized a pipeline to construct, visualize, and register intraoperative holographic models of patient landmarks during spinal fusion surgery. Methods. The study was carried out in two phases. In phase 1, the custom intraoperative pipeline to generate patient-specific holographic models was developed over 7 patients. In phase 2, registration accuracy was optimized iteratively for 6 patients in a real-time operative setting. Results. In phase 1, an intraoperative pipeline was successfully employed to generate and deploy patient-specific holographic models. In phase 2, the registration error with the native hand-gesture registration was 20.2 ± 10.8 mm (n = 7 test points). Custom controller-based registration significantly reduced the mean registration error to 4.18 ± 2.83 mm (n = 24 test points, P <.01). Accuracy improved over time (B = −.69, P <.0001) with the final patient achieving a registration error of 2.30 ±.58 mm. Across both phases, the average model generation time was 18.0 ± 6.1 minutes (n = 6) for isolated spinal hardware and 33.8 ± 8.6 minutes (n = 6) for spinal anatomy. Conclusions. A custom pipeline is described for the generation of intraoperative 3D holographic models during spine surgery. Registration accuracy dramatically improved with iterative optimization of the pipeline and technique. While significant improvements and advancements need to be made to enable clinical utility, HMR demonstrates significant potential as the next frontier of intraoperative visualization.

Original languageEnglish (US)
Pages (from-to)427-437
Number of pages11
JournalSurgical Innovation
Issue number4
StatePublished - Aug 2021
Externally publishedYes


  • 3D visualization
  • augmented reality
  • holographic mixed reality
  • HoloLens™
  • spine surgery

ASJC Scopus subject areas

  • Surgery


Dive into the research topics of 'Development of an Intraoperative Pipeline for Holographic Mixed Reality Visualization During Spinal Fusion Surgery'. Together they form a unique fingerprint.

Cite this