Video Optical See-Through Augmented Reality surgical System


The idea of integrating the surgeon’s perceptive efficiency with the aid of new augmented reality (AR) visualization modalities has become a dominant topic of academic and industrial research in the medical domain since the 90’s. AR technology appeared to represent a significant development in the context of image-guided surgery (IGS). The quality of the AR experience affects the degree of acceptance among physicians and it depends on how well the virtual content is integrated into the real world spatially, photometrically and temporally. In this regard, wearable systems based on head-mounted displays (HMDs), offer the most ergonomic and easily translatable solution for many surgeries. Most of the AR HMDs fall into two categories according to the see-through paradigm they implement: video see-through (VST) and optical see-through (OST) HMDs. In OST systems, the user’s direct view of the real world is augmented with the projection of virtual information into the user’s line of sight. Differently, in VST systems the virtual content is merged with images captured by two external cameras anchored to the visor. With respect to technological and human-factor issues, both the approaches have their own strengths and shortcomings. In this project, we identified in a hybrid OST/VST HMD, a disruptive solution for improving surgical outcomes. The application driven device will be developed from existing systems and exploiting the knowhow acquired within the consortium on photonics KET technologies. The resulting device will undergo three clinical trials whose results will be fundamental towards a straight industrial exploitation comprising economic viability analysis. Video-Optical See Through AR surgical System (VOSTARS) will be the first hybrid see-through HMD surgical navigator. Further, albeit VOSTARS will be specifically designed for medical procedures, its design is aimed to evolve into a multipurpose AR platform for HMDs.

Project details

Unibo Team Leader: Claudio Marchetti

Unibo involved Department/s:
Dipartimento di Scienze Biomediche e Neuromotorie

Università di PISA(Italy)

Other Participants:
ALMA MATER STUDIORUM - Università di Bologna (Italy)
Cea-Commissariat A L'Energie Atomique Et Aux Energies Alternatives (France)
Scuola Sup. di Studi Univ. e Perfezionamento S.Anna di PISA (Italy)
Sankt Gertrauden-Krankenhaus GmbH (Germany)
Charite' Universitatsmedizin Berlin (Germany)
Pilotfish Gmbh (Germany)
Optinvent SA (France)
Scopis GmbH (Germany)
Technische Universitat Munchen (Fg Obstbau) (Germany)
Vreo Limited (United Kingdom)

Total Eu Contribution: Euro (EUR) 3.816.440,00
Project Duration in months: 36
Start Date: 01/12/2016
End Date: 30/11/2019

Cordis webpage
Project website

Industry, innovation and infrastructure This project contributes to the achievement of the Sustainable Development Goals of the UN 2030 Agenda.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731974 This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731974