Deep Learning-Assisted Laparoscopic Surgery
Projects | | Links:
Deep learning has proven to be an extraordinary useful tool in multiple applications. One of these applications is minimally invasive laparoscopic surgical intervention. This type of operations highly depend on surgeon’s ability to interpret the 2D camera image in 3D to correctly perform the surgical procedure. Deep learning can improve surgeon’s abilities by enhancing their skills by the use of this technology to identify the laparoscopic forceps in real time and supporting them with Augmented Reality (AR). The deep learning instance recognition model (YOLACT++) had already been implemented in Python, but it showed to be too slow for online execution.
The project consisted on the development of a C++ inference program to perform multiple object instance segmentation of the laparoscopic forceps to achieve real-time processing speed while maintaining a high accuracy. The program consisted of three main stages: the pre-processing of the frames, the inference into the model (using ONNX Runtime C++ API), and the final post-processing of the model outputs.
I finally achieved a 20% increase in inference speed with respect to the original Python program. I was awarded the UPM Scholarship to conduct research at Miura Lab under the Summer Exchange Research Program (SERP).
