Artificial intelligence anticipates how instruments are used during surgery – Innovation Origins

In the operating theater of the future, computer-based assistance systems will make work processes simpler and safer and thereby play a much greater role than today. However, such support features are only possible if computers are able to anticipate important events in the operating room and provide the right information at the right time, explains Prof. Stefanie Speidel. She is head of the Department of Translational Surgical Oncology at the National Center for Tumor Diseases Dresden (NCT/UCC) in Germany.

Together with the Centre for Tactile Internet with Human-in-the-loop (CeTI) at TU Dresden, she has developed a method that uses artificial intelligence (AI) to enable computers to anticipate the usage of surgical instruments before they are used.

This kind of system does not just provide an important basis for the use of autonomous robotic systems that could take over simple minor tasks in the operating theater, such as blood aspiration. It could also issue early warnings of complications if these are inherent to the use of a particular instrument. Furthermore, it would increase efficiency where preparing instruments is concerned. However, our vision is not to replace the surgeon with a robot or other assistants. The intelligent systems should merely act as a helping hand and lighten the load for both the doctor and the entire surgical team, says Prof. Jrgen Weitz, Managing Director at NCT/UCC and Director of the Clinic for Visceral, Thoracic and Vascular Surgery at the University Hospital Carl Gustav Carus in Dresden.

In order to teach computers how to anticipate the use of surgical instruments on a situation-specific basis a few minutes before they are actually put to use, scientists at NCT/UCC and CeTI used an artificial neural network that mimics the human ability to learn by example. By using a continuous analysis of video images of a surgical procedure, the usage of certain instruments was shown a few minutes before they were actually used. They then trained the neural network with 60 videos of gall bladder removal surgery. These were recorded by default with a laparoscope in the abdomen. Five different instruments were highlighted in these videos.

Afterwards the neural network had to demonstrate its knowledge on 20 more videos without any markers. The scientists were able to verify that the system had made important advances in learning. Plus, in many cases it was able to correctly anticipate how the instruments would be used.

Compared to other methods, this neural network proved to be much more suitable for practical applications. Consequently, this also means that is capable of solving complex tasks. Other methods treat the timing of a specific situation as a matter of routine and the network just needs to decide between various possible situations. In contrast, we have been able to show that an artificial neural network with specific adaptations and a suitably formulated mathematical function is capable of making sensible assessments about the type of instrument that should be selected and the time frame of its application with a minimum of coding effort, says Dominik Rivoir from the Department of Translational Surgical Oncology at NCT/UCC, first author of the study presented at the International Conference on Medical Image Computing & Computer-Assisted Intervention (MICCAI).

Next, the scientists want to refine the method and add more data sets to the neural network. One focus in this is on surgical videos that show more severe bleeding. Using this image data, the network should be able to learn even better when hemorrhages need to be aspirated with a special instrument. In the presented study, the researchers were already able to show that the network interpreted, for example, the appearance of a clamp for clamping a blood vessel with a high degree of accuracy as a characteristic. This way, it was abe to anticipate the use of scissors soon afterward. In future, this could serve as a basis for timing the use of robot-guided aspiration instruments or for anticipating complications.

The National Center for Tumor Diseases Dresden (NCT/UCC) is a joint venture of the German Cancer Research Center (DKFZ), the University Hospital Carl Gustav Carus Dresden, the Medical Faculty Carl Gustav Carus of the TU Dresden, and the Helmholtz Center Dresden-Rossendorf (HZDR).

Title image: Autonomous robotic systems and other intelligent assistance systems will provide enhanced support for the surgical team in the future. NCT/UCC/Andr Wirsig

Read the original:
Artificial intelligence anticipates how instruments are used during surgery - Innovation Origins