UC Davis Technology Brings Augmented Reality to the Operating Room

View video of augmented reality during head and neck cancer surgery.

For hard-to-reach tumors at the back of the throat, robots can assist surgeons to remove cancers in a less invasive way. But this high-tech approach has a drawback: the surgeon’s inability to assess through touch whether they’ve been able to get all of the cancer out.

An innovative new device developed at UC Davis is helping to change that. Based on fluorescence lifetime imaging microscopy (FLIm), the device can be used in tandem with robotic surgery to distinguish between cancerous and benign tissue, guiding the physician in real time to achieve a more precise surgical outcome for the patient.

In a first-in-human, proof-of-principle study published in Scientific Reports, a Nature publication, UC Davis researchers report on their ability to successfully integrate the FLIm tool with a surgical robot to allow real-time evaluation of the different tissue types in patients undergoing oral cancer surgery without using contrast agents. First studied in animal models, the technology has now been used in about 30 human patients with cancers of the tonsil and base of the tongue.

“We are the first to demonstrate implementation of augmented reality in patients and in conjunction with robotic surgery,” said Laura Marcu, a professor in the UC Davis Department of Biomedical Engineering who developed the technology.

FLIm is a technique that measures the autofluorescence properties of certain molecules in tissue in the form of wavelength (or color) and lifetime (or the time a molecule emits light). Since tumor tissue is made up of molecules different from those in normal tissue, tumors also have different fluorescence properties. This allows the autofluorescence to encode diagnostic information to help the surgeon as he removes tissue.

“The FLIm features are augmented instantaneously in the surgeon’s field of view so the surgeon can see tissue properties that cannot be seen otherwise,” Marcu said.

New optical device takes science from bench to bedside

For this work Marcu teamed up with D. Gregory Farwell, professor and chair of the UC Davis Department of Otolaryngology – Head and Neck Surgery, to use the device to help guide him during complex oral cancer surgeries. When Farwell began using the robot to do complex head and neck surgeries, the two figured out a way to incorporate the device with the robot’s technology.

“It’s a special application for patients with a tumor at back of their throat,” Farwell said. “These tumors are very difficult to access, and they traditionally required extensive surgery that involved cutting the jaw and reconstructing the jaw to allow us to get back there. In selected patients we can use the robot to gain access to tumors that historically we could not get to less invasively.”

He added that head and neck cancers are on the rise because of the spread of human papilloma virus, which can lead to cancer even in people without traditional risk factors such as excessive drinking and smoking.

“We have seen patients in their 30s and 40s, but it is showing up increasingly in middle-age men,” he said. “And it is dramatically increasing in incidence.”

Cancers of the head and neck can be very challenging to treat because there is little margin for error, Marcu added.

“In the head and neck, where tumors are present in functional areas, you don’t want to cut too much,” she said. “You want to preserve functional tissue, but remove the entire tumor. This where this tool optimizes the procedure and provides the surgeon with additional information.”

Specially designed optics augment surgeon’s precision tumor removal   

FLIm uses light to excite molecules in tissue and measure how long they emit light of their own after they are excited. Different types of molecules emit light at different rates. FLIm can measure the rates, and that information is used to distinguish between different types of tissue. The device employs fiber optics to deliver and collect the light.

Marcu and Farwell found a way to insert the fiber optic device through a robotic port to assess the tissue, remove it and then replace it with a standard surgical instrument. Sitting at a robotic console, the surgeon can then use the instantaneous findings from the device to remove additional tissue, as needed.

Marcu explained that the information gathered by the device is more detailed than what can be gleaned from a microscopic analysis of a small tissue sample taken during surgery because it generates diagnostic information based on the intrinsic molecular contrast between normal tissue and cancer tissue. She said it may even be able to help find early cancer growth not detected using regular pathology.

“Pathology sees things in black and white,” she said. “There is a grey area where you don’t know if it is cancer or not cancer,” she said. “You can do the genomic analysis and other tests that can provide additional information, but it’s not done in real time and would take weeks. With this device, the surgeon may be able to cut out more tumor or preserve more normal tissue than what had been anticipated because of the information the device can provide in the operating room.”

For the next phase of their work Marcu and Farwell are recruiting up to 80 more patients to further develop and refine the technique. Marcu also plans to team with urological surgeons to use the device for prostate cancer patients.

In addition to Marcu and Farwell, UC Davis authors included: D. Gorpas, J. Phipps, J. Bec, D. Ma, D. Yankelevich, R. Gandour Edwards and A. Bewley. This study was conducted in collaboration with Intuitive Surgical Inc. in Sunnyvale, Calif.

Funding for the study was obtained from NIH-NCI Grant RO1 CA187427.