web viewword count manuscript text: 2,142. 5 figures and 2 tables. introduction. ... hughes-hallett...
TRANSCRIPT
The current and future use of imaging in urological robotic surgery: A survey of the
European Association of Robotic Urological Surgeons
Archie Hughes-Hallett, MRCS1, Erik K Mayer, PhD1, Philip Pratt, PhD2, Alex Mottrie, PhD3,4, Ara
Darzi, FRS1,2, Justin Vale, MS1,
1. Department of Surgery and Cancer, Imperial College London2. The Hamlyn Centre for Robotic Surgery, Imperial College London3. Department of Urology, OLV Clinic, Aalst, Belgium 4. O.L.V. Vattikuti Robotic Surgery Institute, Aalst, Belgium
Corresponding Author
Erik Mayer,
Department of Surgery and Cancer, Imperial College London, St Marys Hospital
Campus, London, W2 1NY
07984195642
No reprints will be available from the authors
No financial support was received
Article Category: Original Article
Word count abstract: 244
Word count manuscript text: 2,142
5 figures and 2 tables
1
1
2
3
4
56789
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Introduction
Since Röntgen first utilised x-rays to image the carpal bones of the human hand in
1895, medical imaging has evolved and is now able to provide a detailed
representation of a patient’s intracorporeal anatomy, with recent advances now
allowing for 3-dimensional (3D) reconstructions. The visualisation of anatomy in 3D
has been shown to improve the ability to localize structures when compared to 2D
with no change in the amount of cognitive loading [1]. This has allowed imaging to
move from a largely diagnostic tool to one that can be used for both diagnosis and
operative planning.
One potential interface to display 3D images, to maximise its potential as a tool for
surgical guidance, is to overlay them onto the endoscopic operative scene (augmented
reality). This addresses, in part, a criticism often levelled at robotic surgery, the loss
of haptic feedback. Augmented reality has the potential to mitigate for this sensory
loss by enhancing the surgeons visual cues with information regarding subsurface
anatomical relationships [2].
Augmented reality surgery is in its infancy for intra-abdominal procedures due in
large part to the difficulties of applying static preoperative imaging to a constantly
deforming intraoperative scene [3]. There are case reports and ex-vivo studies in the
literature examining the technology in minimal access prostatectomy [3–6] and
partial nephrectomy [7–10], but there remains a lack of evidence determining
whether surgeons feel there is a role for the technology and if so what procedures
they feel it would be efficacious for.
2
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
This questionnaire-based study was designed to assess: firstly, the pre- and
intraoperative imaging modalities utilised by robotic urologists; secondly, the current
use of imaging intraoperatively for surgical planning; and finally whether there is a
desire for augmented reality amongst the robotic urological community.
Methods
Recruitment
A web based survey instrument was designed and sent out, as part of a larger survey,
to members of the EAU Robotic Urology Section (ERUS). Only independently
practising robotic surgeons performing RALP, RAPN and/or robotic cystectomy
were included in the analysis, those surgeons exclusively performing other
procedures were excluded. Respondents were offered no incentives to reply. All data
collected was anonymous.
Survey design and administration
The questionnaire was created using the LimeSurvey platform
(www.limesurvey.com) and hosted on their website. All responses (both complete
and incomplete) were included in the analysis. The questionnaire was dynamic with
the questions displayed tailored to the respondents’ previous answers.
When computing fractions or percentages the denominator was the number of
respondents to answer the question, this number is variable due to the dynamic nature
of the questionnaire.
Survey Content
3
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
Demographics
All respondents to the survey were asked in what country they practised and what
robotic urological procedures they performed, in addition to what procedures they
performed surgeons were asked specify the number of cases they had undertaken for
each procedure.
Current Imaging Practice
Procedure-specific questions in this group were displayed according to the operations
the respondent performed. A summary of the questions can be seen in appendix 1.
Procedure non-specific questions were also asked. Participants were asked whether
they routinely used the Tile Pro™ function of the da Vinci console (Intuitive
Surgical, Sunnyvale, USA) and whether they routinely viewed imaging
intraoperatively.
Augmented Reality
Prior to answering questions in this section, participants were invited to watch a
video demonstrating an augmented reality platform during Robot-Assisted Partial
Nephrectomy (RAPN), performed by our group at Imperial College London. A still
from this video can be seen in figure 1. They were then asked whether they felt
augmented reality would be of use as a navigation or training tool in robotic surgery.
Once again, in this section, procedure-specific questions were displayed according to
the operations the respondent performed. Only those respondents who felt augmented
reality would be of use as a navigation tool were asked procedure-specific questions.
4
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
Questions were asked to establish where in these procedures they felt an augmented
reality environment would be of use.
Results
Demographics
Of the 239 respondents completing the survey 117 were independently practising
robotic surgeons and were therefore eligible for analysis. The majority of the
surgeons had both trained (210/239, 87.9%) and worked in Europe (215/239, 90.0%).
The median number of cases undertaken by those surgeons reporting their case
volume was: 120 (6 - 2000), 9 (1 – 120) and 30 (1 – 270), for RALP, Robot assisted
cystectomy and RAPN respectively.
Contemporary use of imaging in robotic surgery
When enquiring about the use of imaging for surgical planning, the majority of
surgeons (57%, 65/115) routinely viewed preoperative imaging intraoperatively with
only 9% (13/137) routinely capitalising on the TilePro™ function in the console to
display these images, when assessing the use of TilePro™ amongst surgeons who
performed RAPN 13.8% (9/65) reported using the technology routinely.
When assessing the imaging modalities that are available to a surgeon in theatre the
majority of surgeons performing RALP (74%, 78/106)) reported using MRI with an
additional 37% (39/106) reporting the use of CT for preoperative staging and/or
planning. For surgeons performing RAPN and robot-assisted cystectomy there was
more of a consensus with 97% (68/70) and 95% (54/57) of surgeons, respectively,
using CT for routine preoperative imaging (table 1).
5
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
Those surgeons performing RAPN were found to have the most diversity in the way
they viewed preoperative images in theatre, routinely viewing images in sagittal,
coronal and axial slices (table 2). The majority of these surgeons also viewed the
images as 3D reconstructions (54%, 38/70).
The majority of surgeons used ultrasound intraoperatively in RAPN (51%, 35/69)
with a further 25% (17/69) reporting they would use it if they had access to a ‘drop-
in’ ultrasound probe (figure 3).
Desire for augmented reality
In all 87% of respondents envisaged a role for augmented reality as a navigation tool
in robotic surgery and 82% (88/107) felt that there was an additional role for the
technology as a training tool.
The greatest desire for augmented reality was amongst those surgeons performing
RAPN with 86% (54/63) feeling the technology would be of use. The largest group
of surgeons felt it would be useful in identifying tumour location, with significant
numbers also feeling it would be efficacious in tumour resection (figure 4).
When enquiring about the potential for augmented reality in Robot-Assisted
Laparoscopic Prostatectomy (RALP), 79% (20/96) of respondents felt it would be of
use during the procedure, with the largest group feeling it would be helpful for nerve
sparing 65% (62/96) (Figure 2). The picture in cystectomy was similar with 74%
(37/50) of surgeons believing augmented reality would be of use, with both nerve
6
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
sparing and apical dissection highlighted as specific examples (40%, 20/50) (Figure
5). The majority also felt that it would be useful for lymph node dissection in both
RALP and robot assisted cystectomy (55% (52/95) and 64% (32/50) respectively).
Discussion
The results from this study suggest that the contemporary robotic surgeon views
imaging as an important adjunct to operative practice. The way these images are
being viewed is changing; although the majority of surgeons continue to view images
as two-dimensional (2D) slices a significant minority have started to capitalise on 3D
reconstructions to give them an improved appreciation of the patient’s anatomy.
This study has highlighted surgeons’ willingness to take the next step in the
utilisation of imaging in operative planning, augmented reality, with 87% feeling it
has a role to play in robotic surgery. Although there appears to be a considerable
desire for augmented reality, the technology itself is still in its infancy with the
limited evidence demonstrating clinical application reporting only qualitative results
[3,11–13].
There are a number of significant issues that need to be overcome before augmented
reality can be adopted in routine clinical practice. The first of these is registration (the
process by which two images are positioned in the same coordinate system such that
the locations of corresponding points align [14]). This process has been performed
both manually and using automated algorithms with varying degrees of accuracy
[2,15]. The second issue pertains to the use of static preoperative imaging in a
7
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
dynamic operative environment; in order for the preoperative imaging to be
accurately registered it must be deformable. This problem remains as yet unresolved.
Live intraoperative imaging circumvents the problems of tissue deformation and in
RAPN 51% of surgeons reported already using intraoperative ultrasound to aid in
tumour resection. Cheung and colleagues [9] have published an ex-vivo study
highlighting the potential for intraoperative ultrasound in augmented reality partial
nephrectomy. They report the overlaying of ultrasound onto the operative scene to
improve the surgeon’s appreciation of the subsurface tumour anatomy, this
improvement in anatomical appreciation resulted in improved resection quality over
conventional ultrasound guided resection [9]. Building on this work the first in vivo
use of overlaid ultrasound in RAPN has recently been reported [10]. Although good
subjective feedback was received from the operating surgeon, the study was limited
to a single case demonstrating feasibility and as such was not able to show an
outcome benefit to the technology [10].
RAPN also appears to be the area in which augmented reality would be most readily
adopted with 86% of surgeons claiming they see a use for the technology during the
procedure. Within this operation there are two obvious steps to augmentation,
anatomical identification (in particular vessel identification to facilitate both routine
‘full clamping’ and for the identification of secondary and tertiary vessels for
‘selective clamping’ [16]) and tumour resection. These two phases have different
requirements from an augmented reality platform; the first phase of identification
requires a gross overview of the anatomy without the need for high levels of
registration accuracy. Tumour resection, however, necessitates almost sub-millimetre
8
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
accuracy in registration and needs the system to account for the dynamic
intraoperative environment. The step of anatomical identification is amenable to the
use of non-deformable 3D reconstructions of preoperative imaging while that of
image-guided tumour resection is perhaps better suited to augmentation with live
imaging such as ultrasound [2,9,17].
For RALP and robot-assisted cystectomy the steps in which surgeons felt augmented
reality would be of assistance were those of neurovascular bundle preservation and
apical dissection. The relative, perceived, efficacy of augmented reality in these steps
correlate with previous examinations of augmented reality in RALP [18,19].
Although surgeon preference for utilising AR while undertaking robotic
prostatectomy has been demonstrated, Thompson et al. failed to demonstrate an
improvement in oncological outcomes in those patients undergoing AR RALP [19].
Both nerve sparing and apical dissection require a high level of registration accuracy
and a necessity for either live imaging or the deformation of preoperative imaging to
match the operative scene; achieving this level of registration accuracy is made more
difficult by the mobilisation of the prostate gland during the operation [18]. These
problems are equally applicable to robot-assisted cystectomy. Although guidance
systems have been proposed in the literature for RALP [3,4,13,18,20], none have
achieved the level of accuracy required to provide assistance during nerve sparing.
Additionally, there are still imaging challenges that need to be overcome. Although
multiparametric MRI has been shown to improve decision making in opting for a
nerve sparing approach to RALP [21] the imaging is not yet able to reliably discern
the exact location of the neurovascular bundle. This said significant advances are
9
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
being made with novel imaging modalities on the horizon that may allow for imaging
of the neurovascular bundle in the near future [22].
Limitations
The number of operations included represents a significant limitation of the study,
had different index procedures been chosen different results may have been seen.
This being said the index procedures selected were chosen as they represent the vast
majority of uro-oncological robotic surgical practice, largely mitigating for this
shortfall.
Although the available ex-vivo evidence suggests that introducing augmented reality
operating environments into surgical practice would help to improve outcomes [9,23]
the in-vivo experience to date is limited to small volume case series reporting
feasibility [2,3,15]. To date no study has demonstrated an in-vivo outcome advantage
to augmented reality guidance. In addition to this limitation augmented reality has
been demonstrated to increased rates of inattention blindness amongst surgeons
suggesting there is a trade of between increasing visual information and the surgeon’s
ability to appreciate unexpected operative events [23].
Conclusions
This survey depicts the contemporary robotic surgeon to be comfortable with the use
of imaging to aid in intraoperative planning; furthermore it highlights a significant
interest amongst the urological community in augmented reality operating platforms.
10
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
Short to medium term development of augmented reality systems in robotic urology
surgery would be best performed using RAPN as the index procedure. Not only was
this the operation where surgeons saw the greatest potential benefits, but it may also
be the operation where it is most easily achievable by capitalising on the respective
benefits of technologies the surgeons are already using; preoperative CT for
anatomical identification and intraoperative ultrasound for tumour resection.
Conflicts of Interest
None of the authors have any conflicts of interest to declare
References
1. Foo J-L, Martinez-Escobar M, Juhnke B, Cassidy K, Hisley K, Lobe T, Winer
E. Evaluating mental workload of two-dimensional and three-dimensional
visualization for anatomical structure localization. J Laparoendosc Adv Surg
Tech A. 2013;23(1):65–70.
2. Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW,
Vale J. Augmented Reality Partial Nephrectomy: Examining the Current
Status and Future Perspectives. Urology; 2014;83(2):266–73.
3. Sridhar AN, Hughes-Hallett A, Mayer EK, Pratt PJ, Edwards PJ, Yang G-Z,
Darzi AW, Vale J. Image-guided robotic interventions for prostate cancer. Nat
Rev Urol. 2013;10(8):452–62.
11
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
4. Cohen D, Mayer E, Chen D, Anstee A, Vale J, Yang G-Z, Darzi A, Edwards P
“Eddie.” Augmented reality image guidance in minimally invasive
prostatectomy. Lect Notes Comput Sci. 2010;6367:101–10.
5. Simpfendorfer T, Baumhauer M, Muller M, Gutt CN, Meinzer HP, Rassweiler
JJ, Guven S, Teber D. Augmented reality visualization during laparoscopic
radical prostatectomy. J Endourol. 2011;25(12):1841–5.
6. Teber D, Simpfendorfer T, Guven S, Baumhauer M, Gozen AS, Rassweiler J.
In-vitro evaluation of a soft-tissue navigation system for laparoscopic
prostatectomy. J Endourol. 2010;24(9):1487–91.
7. Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F,
Gözen AS, Rassweiler JJ, Simpfendorfer T, Guven EO, Gozen AS.
Augmented reality: a new tool to improve surgical accuracy during
laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur
Urol. 2009;56(2):332–8.
8. Pratt P, Mayer E, Vale J, Cohen D, Edwards E, Darzi A, Yang G-Z. An
effective visualisation and registration system for image-guided robotic partial
nephrectomy. J Robot Surg. 2012;6(1):23–31.
9. Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM. Fused video and
ultrasound images for minimally invasive partial nephrectomy: A phantom
study. Med Image Comput Comput Assist Interv. 2010;13(Pt 3):408–15.
12
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
10. Hughes-Hallett A, Pratt P, Mayer E, Di Marco A, Yang G-Z, Vale J, Darzi A.
Intraoperative Ultrasound Overlay in Robot-assisted Partial Nephrectomy:
First Clinical Experience. Eur Urol. 2013;
11. Nakamura K, Naya Y, Zenbutsu S, Araki K, Cho S, Ohta S, Nihei N, Suzuki
H, Ichikawa T, Igarashi T. Surgical navigation using three-dimensional
computed tomography images fused intraoperatively with live video. J
Endourol. 2010;24(4):521–4.
12. Teber D, Guven S, Simpfendorfer T, Baumhauer M, Guven EO, Yencilek F,
Gozen AS, Rassweiler J. Augmented Reality: A New Tool To Improve
Surgical Accuracy during Laparoscopic Partial Nephrectomy? Preliminary In
Vitro and In Vivo Results. Eur Urol. 2009;56(2):332–8.
13. Ukimura O, Gill IS. Imaging-assisted endoscopic surgery: Cleveland clinic
experience. J Endourol. 2008;22(4):803–9.
14. Altamar HO, Ong RE, Glisson CL, Viprakasit DP, Miga MI, Herrell SD,
Galloway RL. Kidney deformation and intraprocedural registration: A study of
elements of image-guided kidney surgery. J Endourol. 2011;25(3):511–7.
15. Nicolau S, Soler L, Mutter D, Marescaux J. Augmented reality in laparoscopic
surgical oncology. Surg Oncol. 2011;20(3):189–201.
16. Ukimura O, Nakamoto M, Gill IS. Three-dimensional reconstruction of
renovascular-tumor anatomy to facilitate zero-ischemia partial nephrectomy.
Eur Urol. 2012;61(1):211–7.
13
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
17. Pratt P, Hughes-Hallett A, Di Marco A, Cundy T, Mayer E, Vale J, Darzi A,
Yang G-Z. Multimodal Reconstruction for Image-Guided Interventions.
Hamlyn Symposium. 2013.
18. Mayer EK, Cohen D, Chen D, Anstee A, Vale J a., Yang GZ, Darzi AW,
Edwards E. Augmented Reality Image Guidance in Minimally Invasive
Prostatectomy. Eur Urol Supp. 2011;10(2):300.
19. Thompson S, Penney G, Billia M, Challacombe B, Hawkes D, Dasgupta P.
Design and evaluation of an image-guidance system for robot-assisted radical
prostatectomy. BJU Int. 2013;111(7):1081–90.
20. Simpfendorfer T, Baumhauer M, Muller M, Gutt CN, Meinzer H-PP,
Rassweiler JJ, Guven S, Teber D, Simpfendörfer T, Müller M. Augmented
reality visualization during laparoscopic radical prostatectomy. J Endourol.
2011;25(12):1841–5.
21. Panebianco V, Salciccia S, Cattarino S, Minisola F, Gentilucci A, Alfarone A,
Ricciuti GP, Marcantonio A, Lisi D, Gentile V, Passariello R, Sciarra A. Use
of Multiparametric MR with Neurovascular Bundle Evaluation to Optimize the
Oncological and Functional Management of Patients Considered for Nerve-
Sparing Radical Prostatectomy. J Sex Med. 2012;9(8):2157–66.
22. Rai S, Srivastava A, Sooriakumaran P, Tewari A. Advances in imaging the
neurovascular bundle. Curr Opin Urol. 2012;22(2):88–96.
14
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
23. Dixon BJ, Daly MJ, Chan H, Vescan AD, Witterick IJ, Irish JC. Surgeons
blinded by enhanced navigation: the effect of augmented reality on attention.
Surg Endosc. 2013;27(2):454–61.
15
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
Tables
CT MRI USS None OtherRALP (n=106) 39.8%
(39)73.5% (78)
2%(3)
15.1% (16)
8.4%(9)
RAPN (n=70) 97.1% (68)
42.9% (30)
17.1% (12)
0%(0)
2.9%(2)
Cystectomy (n=57) 94.7% (54)
26.3% (15)
1.8%(1)
1.8%(1)
5.3%(3)
Table 1 - Which preoperative imaging modalities do you use for diagnosis and surgical planning?
16
355
356
357
358
359
360
361
362
363
364
365
366
367
368
Axial slices(n)
Coronal slices(n)
Sagittal slices (n)
3D recons.
(n)
Do not view(n)
RALP (n=106) 49.1% (52)
44.3% (47)
31.1% (33)
9.4%(10)
31.1% (33)
RAPN (n=70) 68.6% (48)
74.3% (52)
60% (42) 54.3%(38)
0%(0)
Cystectomy (n=57)
70.2% (40)
52.6% (30)
50.9% (29)
21.1%(12)
8.8%(5)
Table 2 - How do you typically view preoperative imaging in the OR?3D recons = Three dimensional reconstructions
17
369
370
371
372
373
374
375
376
377
378
379
380
Figure Legends
Figure 1 – A still taken from a video of augmented reality robot assisted partial
nephrectomy performed. Here the tumour has been painted into the operative view
allowing the surgeon to appreciate the relationship of the tumour to the surface of the
kidney.
Figure 2 – Chart demonstrating responses to the question - In robotic prostatectomy
which parts of the operation do you feel augmented reality image overlay would be of
assistance?
Figure 3 - Chart demonstrating responses to the question - Do you use intraoperative
ultrasound for robotic partial nephrectomy?
Figure 4 - Chart demonstrating responses to the question – In robotic partial
nephrectomy which parts of the operation do you feel augmented reality image
overlay would be of assistance?
Figure 5 - Chart demonstrating responses to the question – In robotic cystectomy
which parts of the operation do you feel augmented reality overlay technology would
be of assistance?
18
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400