Open Access | Mini review
This work is licensed under a Creative
Commons Attribution-ShareAlike 4.0 International License.
Artificial intelligence and augmented reality: transforming urology?
* Corresponding author: Riccardo Autorino
Mailing address: Department of Urology, Rush University Medical Center, Chicago, IL, USA.
Email: ricautor@gmail.com
Received: 04 March 2024 / Revised: 08 March 2024 / Accepted: 12 March 2024 / Published: 27 March 2024
DOI: 10.31491/UTJ.2024.03.018
Abstract
The advent of artificial intelligence (AI) and augmented reality (AR) in the medical field has signaled a transformative shift in urological practice. This mini-review encapsulates the current innovations, challenges, and ethical considerations of AI and AR technologies in urology. The potential of AI in urology spans from diagnostic advancements in uro-oncology to predictive modeling in functional urology and urolithiasis, empowering precision medicine with data-driven insights. AR enhances the surgical field with real-time, precision-guided interventions and enriches training with immersive educational experiences. However, the integration of these technologies raises ethical questions about data privacy, potential algorithmic bias, and the impact on the clinician-patient dynamic. Addressing these concerns is essential for a future where AI and AR are not only innovative, but also aligned with patient-centered care.
Keywords
AI, AR, technology, 3D models, image-guided surgery
Introduction
The integration of artificial intelligence (AI) and augmented reality (AR) into medical practice heralds a transformative era
for healthcare, with urology at the vanguard of this revolution. Historically, urology has frequently been an early adopter of
innovative technologies, leveraging advancements to enhance both the precision of surgical interventions and the breadth of clinical
research. These emerging technologies offer unprecedented opportunities to refine diagnostic acuity, customize therapeutic
interventions, and streamline patient care workflows. In urology, AI and AR are not just futuristic concepts but are already
underpinning significant improvements in areas such as surgical planning and surgical training
[1]. Concurrently, the role of these technologies in patient management and clinical
decision making continues to grow, underlined by the promise to personalize treatment pathways and enhance the prognosis of
urological conditions. The evolution of AI from basic diagnostic tools to complex predictive models demonstrates a burgeoning
synergy between technology and clinical expertise. However, the path to a fully integrated AI and AR-enabled urological practice is
fraught with challenges. Issues around costs, patient privacy, and the potential for algorithmic bias must be carefully navigated.
Additionally, there is a need to reconcile the rapid pace of technological innovation with the imperative of maintaining the sanctity
of the clinicianpatient relationship.
This mini-review scrutinizes the state-of-the-art AI and AR applications in urology, unpacks the challenges hindering their
ubiquitous adoption, and sparks a dialogue on the ongoing debates about ethical implications, potential risks, and the overarching
impact on patient care.
Materials and methods
We designed a mini narrative review. A bibliographic search was conducted in January 2024 using PubMed/MEDLINE, Scopus, and Web of Science databases. Special attention was given to articles published in the last 5 years. Additional articles were identified by manually cross-referencing the bibliographies of selected articles. Article selection was conducted by two independent reviewers (L.C.L. and E.B.), and final inclusion was based on a consensus discussion involving a third senior author (R.A.).
Results
Artificial intelligence
AI is commonly defined as the set of computational technologies that emulate intellectual processes typically associated with
human cognition, such as reasoning, learning, and problem solving [2]. Human intelligence
can be metaphorically described as the union of "software" with its "hardware"; the "software" includes the algorithms and
knowledge we use to process information, which can be enhanced throughout life. Conversely, the "hardware" represents the physical
structure of the human brain, which is less susceptible to rapid changes compared to software, due to the gradual pace of
biological evolution over generations. Interestingly, AI is not restricted by such evolutionary constraints. It has the unique
capability to upgrade both its "software" and "hardware" without the need for generational evolution. This reflects the progression
of AI development, which has advanced from merely replicating basic human cognitive functions to enhancing and expediting these
processes while adeptly handling extensive data, culminating in the creation of sophisticated, advanced systems. When integrated
into healthcare, AI aligns with the core principles of scientific research, which involve the meticulous processing of clinical
data—the "input"—and the thorough examination of clinical observations—the "output"— to establish and validate interconnected
patterns. There are several subfields of AI [3,
4]:
● Machine learning (ML) integrates computer science, statistics, and mathematics to develop algorithms capable of predicting
outcomes. These algorithms excel over traditional statistics by focusing on predictions rather than relationships between variables
and continuously improve by learning from additional data.
● Deep learning (DL), an advanced branch of ML, utilizes artificial neural networks to analyze data with a complexity that
resembles the human brain. DL systems identify patterns and features within medical imagery, often discovering diagnostic
indicators that human analysis might miss.
● Big data encompasses vast data sets that traditional software cannot handle. In healthcare, AI processes this data to identify
disease patterns, predict disease progression, and aid in drug development. The unlimited potential offered by the evolution of
these models has, over the years, found extensive applications in the field of urology, ranging from functional urology to
uro-oncology.
In the realm of urolithiasis, the main applications of AI have varied from outcomes prediction to diagnosis and therapy
[5]. The prediction of outcomes is the most extensively studied area; stone-free status,
the detection of infection, the optimization of kidney stone fragmentation, and the prediction of stone patients' health-related
quality of life are some of the outcomes investigated. Predictive applications extend to the differential diagnosis and even to
predictions of stone composition. Advances have also been made in the surgical field, such as improving the surgical performance
by refining protocols, stone localizations, and patient selection
[6, 7]. Urogynecology has also adopted ML
systems, particularly for assessing functional outcomes and postoperative results [8].
Predictive models have been employed to evaluate the occurrence of complications such as stress urinary incontinence following
prolapse surgery, conditions recurrence, and overall outcomes of surgical interventions. Among benign urological conditions, the
heterogenicity of benign prostatic hyperplasia (BPH) has provided a broad scope for the integration of AI. Computer vision-based
systems have been evaluated for their diagnostic accuracy in identifying BPH histologically, with results showing a high accuracy
level. Neural networks have been applied to support the prediction of complications following BPH-related surgeries, to examine
the risk of worsening symptoms, and to analyze the contributing risk factors. Additionally, these networks have been instrumental
in developing models to predict patient responses to medical treatments [6]. Urological
cancer has reaped the greatest benefits from the development of AI. More data, more connections, and generally more inputs allow
the potential of machine learning and its derivatives to be fully realized. Hence, it is clear how urological cancer provides an
immense array of factors that are well-suited to the advantages of these new technologies, ranging from pathogenesis and risk
factors to diagnosis, treatment decisions, and outcome predictions [9].
In the diagnostic field, major advancements have been achieved in the management of Prostate cancer (PCa), including the use of
clinico-pathological data to differentiate between organ-confined and non-organ-confined PCa
[10, 11]. Algorithms have been developed to
predict Gleason Scores based on MRI studies or to detect PCa on digitized pathology images. Similar concepts have been applied
to urothelial and renal cell carcinoma (RCC). Diagnosis and staging of urothelial cancer now potentially rely on imaging and
biomarkers, while metabolomic data assist in differentiating between RCC and healthy tissue
[12]. These concepts have also been applied to the prediction of oncological outcomes
such as disease recurrence, survival analysis, and the guidance of management decision-making and therapy selection
[13]. The continual advancement of AI in urology signifies a transformative shift
towards data-driven precision medicine (Figure 1). By harnessing sophisticated algorithms
and computational analytics, we are not only redefining the paradigms of diagnosis and treatment but also empowering clinicians
with predictive insights that could lead to more effective, individualized patient care strategies, ultimately enhancing outcomes
and elevating the standard of urological practice.
Figure 1. Integrating artificial intelligence into reality: a schematic overview.
Augmented reality
AI algorithms have also a critical role in extended reality (XR) technologies implementation, that provides clinicians with
data-driven insights for decision-making and real-time precision-guided interventions.
XR encompasses a spectrum of immersive technologies that alter and enhance the traditional interface between the user and the
computer, each with distinct applications in urology.
● Virtual reality (VR) immerses users completely within a digital environment, isolating them from the physical world.
● Augmented reality (AR) on the other hand, superimposes digital images onto the user's real-world view, typically through digital
interfaces that allows for clear visibility of the environment.
● Mixed reality (MR) combines elements of both AR and VR, merging real and virtual worlds to produce new environments where
physical and digital objects co-exist and interact in real time.
XR's role in urology extends to education and skill development, where devices like VR headsets and optical headmounted displays
(OHMDs) create immersive learning scenarios [14]. VR has progressed from facilitating
the practice of fundamental, discrete skills to enabling comprehensive procedural simulations. Likewise, AR offers to urology
trainees the opportunity to engage with highly detailed, interactive models, thereby enriching their learning experience and
surgical training. The validated fields of application range from uro-oncology to endourology and andrology; however, there is a
lack of validated VR simulators for the management of urological emergencies [15].
Despite issues with the quality of studies available and the cost-effectiveness of these advanced training devices compared to the
"standard" ones, the early results are largely in favor of the integration of XR simulations in surgical training
[16]. Recent advances have led to an even more revolutionary application of AR and MR in
the field of Urology. As a matter of fact, the integration of these technologies is actively being explored as a potential
game-changer for urologic surgeries [17]. A growing body of evidence suggests that AR
and MR can significantly enhance the interactivity of preoperative planning and patient education by integrating standard imaging
with patient-specific 3D models [18]. These models can be either printed, visualized
through AR, or displayed on 3D/2D computer monitors, offering a more comprehensive and tailored surgical planning and educational
process for both surgeons and patients alike. An area where this technology has found an ideal application is in nephron-sparing
renal surgery, where the integration of advanced information about tumor position and vascularization and relations with vital
structures has been shown to enhance preoperative planning and surgeon's confidence, as well as the patient's understanding of
their condition [19]. Moreover, AR can be applied during a surgical intervention to
permit augmented visualization directly within the surgical field, thereby integrating seamlessly with the clinician's workflow.
By enabling the superimposition of diagnostic preoperative images onto the operative field, AR offers enhanced visual guidance for
identification of anatomical structures during complex procedures. Additionally, it can assist in minimizing the risk of
complications by indicating the real-time location of fragile structures [20]. Once
again, a highly researched field of application has been that of partial nephrectomies, but various uro-oncological and
reconstructive procedures have also been evaluated [21].
Challenges and ethical concerns
As we embrace the integration of AI and AR in urology, we confront several challenges and ethical dilemmas that must be addressed to ensure responsible adoption and application of these technologies [22, 23]. One of the primary challenges is data privacy. As healthcare increasingly relies on digital data, ensuring the confidentiality and security of patient information is paramount. AI systems require vast datasets for training and validation, which raises concerns about the potential for data breaches and misuse. Potential biases in AI algorithms also pose significant challenges. AI models can inadvertently perpetuate existing biases present in their training data, leading to disparities in patient care. This is particularly concerning in medicine, where algorithms may influence decisionmaking in patient diagnosis and treatment. The implications of automation on clinical decision-making are also a critical consideration. There is a delicate balance to be struck between leveraging AI for more accurate diagnoses and treatment plans and ensuring that these technologies support, rather than overshadow, the clinician's expertise. Furthermore, as AI and AR tools become more prevalent, the medical community must grapple with their impact on the clinician-patient relationship. There is a risk that over-reliance on technology could depersonalize care and diminish the role of the clinician's judgment, experience, and personal interaction with the patient.
Future perspectives
As AI and AR technologies continue to advance, their potential to transform urological practice becomes increasingly evident
[24]. The precise diagnostics, tailored treatment protocols, and improved patient care
facilitated by these technologies could usher in a new era of urology defined by enhanced precision and individualized care
[25]. Harnessing AI and AR for the benefit of patients means not only adopting these
technologies but also training clinicians to use them effectively [26]. Education and
training will be critical to maximize the advantages that AI and AR offer, ensuring that clinicians are equipped to incorporate
these tools into practice.
The future will also require a continuous dialogue within the medical community to ensure these technologies are used to complement
human expertise, not replace it. As we look to integrate AI and AR into urology, we must also maintain a commitment to the ethical
use of technology, upholding standards that prioritize patient well-being and trust. Accessibility is a crucial factor in the
widespread adoption of AI and AR in urology. These technologies must be available to a broad range of healthcare settings,
including under-resourced areas, to avoid exacerbating healthcare disparities.
Finally, we must consider the long-term implications of these technologies on the healthcare system. This includes evaluating the
cost-effectiveness of AI and AR in urology, assessing their impact on healthcare delivery, and ensuring they align with the
overarching goals of patient-centered care.
Conclusions
AI and AR technologies herald a new era in urology, characterized by enhanced precision, improved patient outcomes, and more personalized care. However, as we navigate through this technological revolution, it is imperative to address the ethical, social, and practical challenges that accompany the integration of these technologies into clinical practice. By fostering a balanced approach that considers both the potential benefits and the limitations of AI and AR, we can ensure that the future of urology is both innovative and patient centered.
Declarations
Financial support and sponsorship
None.
Conflicts of interest
Riccardo Autorino is a member of the editorial board of Uro-Technology Journal. The authors declare that they have no conflicts and were not involved in the journal's review or decision regarding this manuscript.
References
1. Gómez Rivas J, Toribio Vázquez C, Ballesteros Ruiz C, Taratkin M, Marenco JL, Cacciamani GE, et al. Artificial intelligence and simulation in urology. Actas Urol Esp, 2021, 45(8): 524-529. [Crossref]
2. Hameed BMZ, AVL SD, Raza SZ, Karimi H, Khanuja HS, Shetty DK, et al. Artificial intelligence and its impact on urological diseases and management: a comprehensive review of the literature. J Clin Med, 2021, 10(9): 1864- 1874. [Crossref]
3. Brodie A, Dai N, Teoh JY, Decaestecker K, Dasgupta P, & Vasdev N. Artificial intelligence in urological oncology: An update and future applications. Urol Oncol, 2021, 39(7): 379-399. [Crossref]
4. Salem H, Soria D, Lund JN, & Awwad A. A systematic review of the applications of Expert Systems (ES) and machine learning (ML) in clinical urology. BMC Med Inform Decis Mak, 2021, 21(1): 223-233. [Crossref]
5. Hameed BMZ, Somani S, Keller EX, Balamanigandan R, Mahapatra S, Pietropaolo A, et al. Application of virtual reality, augmented reality, and mixed reality in endourology and urolithiasis: an update by YAU endourology and urolithiasis working group. Front Surg, 2022, 9: 866946. [Crossref]
6. Bouhadana D, Lu XH, Luo JW, Assad A, Deyirmendjian C, Guennoun A, et al. Clinical applications of machine learning for urolithiasis and benign prostatic hyperplasia: a systematic review.J Endourol, 2023, 37(4): 474-494. [Crossref]
7. Checcucci E, Amparore D, Volpi G, Piramide F, De Cillis S, Piana A, et al. Percutaneous puncture during PCNL: new perspective for the future with virtual imaging guidance. World J Urol, 2022, 40(3): 639-650. [Crossref]
8. Shah M, Naik N, Somani BK, & Hameed BMZ. Artificial intelligence (AI) in urology-current use and future directions: an iTRUE study. Turk J Urol, 2020, 46(Supp. 1): S27-S39. [Crossref]
9. Chen J, Remulla D, Nguyen JH, Dua A, Liu Y, Dasgupta P, et al. Current status of artificial intelligence applications in urology and their potential to influence clinical practice. BJU Int, 2019, 124(4): 567-577. [Crossref]
10. Froń A, Semianiuk A, Lazuk U, Ptaszkowski K, Siennicka A, Lemiński A, et al. Artificial intelligence in urooncology: what we have and what we expect. Cancers, 2023, 15(17): 4282-4292. [Crossref]
11. Bianchi L, Chessa F, Angiolini A, Cercenelli L, Lodi S, Bortolani B, et al. The use of augmented reality to guide the intraoperative frozen section during robot-assisted radical prostatectomy. Eur Urol, 2021, 80(4): 480-488. [Crossref]
12. Kowalewski KF, Egen L, Fischetti CE, Puliatti S, Juan GR, Taratkin M, et al. Artificial intelligence for renal cancer: From imaging to histology and beyond. Asian J Urol, 2022, 9(3): 243-252. [Crossref]
13. Checcucci E, Autorino R, Cacciamani GE, Amparore D, De Cillis S, Piana A, et al. Artificial intelligence and neural networks in urology: current clinical applications. Minerva Urol Nefrol, 2020, 72(1): 49-57. [Crossref]
14. Alrishan Alzouebi I, Saad S, Farmer T, & Green S. Is the use of augmented reality-assisted surgery beneficial in urological education? A systematic review. Curr Urol, 2021, 15(3): 148-152. [Crossref]
15. Chahal B, Aydin A, & Ahmed K. Virtual reality vs. physical models in surgical skills training. An update of the evidence. Curr Opin Urol, 2024, 34(1): 32-36. [Crossref]
16. Suresh D, Aydin A, James S, Ahmed K, & Dasgupta P. The role of augmented reality in surgical training: a systematic review. Surg Innov, 2023, 30(3): 366-382. [Crossref]
17. Roberts S, Desai A, Checcucci E, Puliatti S, Taratkin M, Kowalewski KF, et al. “Augmented reality” applications in urology: a systematic review. Minerva Urol Nephrol,2022, 74(5): 528-537. [Crossref]
18. Checcucci E, Amparore D, Fiori C, Manfredi M, Ivano M, Di Dio M, et al. 3D imaging applications for robotic urologic surgery: an ESUT YAUWP review. World J Urol, 2020, 38(4): 869-881. [Crossref]
19. Porpiglia F, Amparore D, Checcucci E, Autorino R, Manfredi M, Iannizzi G, et al. Current use of three-dimensional model technology in urology: a road map for personalised surgical planning. Eur Urol Focus, 2018, 4(5): 652- 656. [Crossref]
20. Pérez-Pachón L, Poyade M, Lowe T, & Gröning F. Image overlay surgery based on augmented reality: a systematic review. Adv Exp Med Biol, 2020, 1260: 175-195. [Crossref]
21. Rodler S, Kidess MA, Westhofen T, Kowalewski KF, Belenchon IR, Taratkin M, et al. A systematic review of new imaging technologies for robotic prostatectomy: from molecular imaging to augmented reality. J Clin Med, 2023, 12(16): 5425-5435. [Crossref]
22. Naik N, Hameed BMZ, Shetty DK, Swain D, Shah M, Paul R, et al. Legal and ethical consideration in artificial intelligence in healthcare: who takes responsibility? Front Surg, 2022, 9: 862322. [Crossref]
23. Gerke S, Minssen T, & Cohen G. Ethical and legal challenges of artificial intelligence-driven healthcare. Artificial intelligence in healthcare, 2020: 295-336.
24. Randazzo G, Reitano G, Carletti F, Iafrate M, Betto G, Novara G, et al. Urology: a trip into metaverse. World J Urol, 2023, 41(10): 2647-2657. [Crossref]
25. Bertolo R, Hung A, Porpiglia F, Bove P, Schleicher M, & Dasgupta P. Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come. World J Urol, 2020, 38(9): 2167-2176. [Crossref]
26. Amparore D, Sica M, Verri P, Piramide F, Checcucci E, De Cillis S, et al. Computer vision and machine-learning techniques for automatic 3D virtual images overlapping during augmented reality guided robotic partial nephrectomy. Technol Cancer Res Treat, 2024, 23: 15330338241229368. [Crossref]