Introduction and Objectives: Recently, different competency-based frameworks have formed the
foundations of surgical training and assessment.1,2 Competency assessment process has been
traditionally performed using subjective in-training evaluation reports, filled out by the program directors.3
However, there is lack of validity and reliability of these subjective reports necessitating the invention of
alternative objective assessment tools.3 Recently, several virtual-reality simulators were introduced,
validated, and used for training and assessment of endourologic and robotic skills.4–11 The objective of
this video was to show the incorporation of virtual-reality simulators into objective structured clinical
examinations (OSCEs) to assess competency of urology postgraduate trainees (PGTs) in basic
endourologic and robotic skills.
Materials and Methods: Between December 2012 and December 2015, PGTs from all four urology
training programs in Quebec were recruited for assessment of their basic endourologic or robotic skills
during five OSCEs using three validated virtual-reality simulators. The GreenLight simulator, invented
by Dr Robert Sweet (American Medical Systems, Guelph, ON), was used to assess their photo-selective
vaporization of the prostate (PVP) skills by vaporizing a 30 g normal prostate during two OSCEs. The
PERC Mentor simulator from Simbionix (Cleveland, OH) was used for assessment of their percutaneous
access skills by performing task 4 during two OSCEs. Finally, the da Vinci surgical skills simulator
(dVSSS) from Intuitive Surgical (Sunnyvale, CA) was used for assessment of their basic robotic skills by
performing Pick and Place task and Energy Dissection level 1 task during an OSCE. Assessment was
performed during a 20-minute station during each OSCE and competency pass scores were calculated
using the norm-referenced method by three experts.6,10–12
Results: For the GreenLight simulator, 37 PGTs participated in the two OSCEs. There were significantly
more competent PGTs among those who had previously practiced on the simulator (32.7% vs 10.2%;
p = 0.009). When comparing global scores from the first and second OSCEs, there was significant
improvement in the number of grams vaporized (2.9 g vs 4.3 g; p = 0.003) and global score (100 vs 165;
p = 0.03). Furthermore, there was good correlation between the number of previously performed PVPs
and global scores (r = 0.4, p = 0.04).6 For the PERC Mentor simulator, 26 PGTs were recruited. When
compared with the 21 PGTs without practice, all 5 PGTs who had practiced on the simulator were
competent (p = 0.03). Competent PGTs performed the task with significantly higher percentage of
successful attempts to access renal calices (p < 0.001), shorter fluoroscopy time (9.8 minutes vs 6.5
minutes; p = 0.01), higher PCNL-GRS scores (p < 0.001), and lower complications (p = 0.01).10 For the
dVSSS, nine PGTs were recruited. There was significant difference among PGY levels in terms of
competency for the basic robotic skills tested (p = 0.01) since all three (33%) competent PGTs were from
the final PGY-5 level.11
Conclusion: High-fidelity virtual-reality simulators could be effectively incorporated into OSCEs to
assess competency of urology PGTs in basic endourologic and robotic skills. Future international studies
should include more complex exercises with larger sample sizes. |
