Artificial intelligence and augmented reality: transforming urology? | Licari | Uro-Technology Journal

Open Access | Mini review
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Artificial intelligence and augmented reality: transforming urology?

Leslie Claire Licaria, Eugenio Bolognaa, Francesco Ditonnoa, Riccardo Autorinoa,*

a Department of Urology, Rush University Medical Center, Chicago, IL, USA.

* Corresponding author: Riccardo Autorino
Mailing address: Department of Urology, Rush University Medical Center, Chicago, IL, USA.

Received: 04 March 2024 / Revised: 08 March 2024 / Accepted: 12 March 2024 / Published: 27 March 2024

DOI: 10.31491/UTJ.2024.03.018


The advent of artificial intelligence (AI) and augmented reality (AR) in the medical field has signaled a transformative shift in urological practice. This mini-review encapsulates the current innovations, challenges, and ethical considerations of AI and AR technologies in urology. The potential of AI in urology spans from diagnostic advancements in uro-oncology to predictive modeling in functional urology and urolithiasis, empowering precision medicine with data-driven insights. AR enhances the surgical field with real-time, precision-guided interventions and enriches training with immersive educational experiences. However, the integration of these technologies raises ethical questions about data privacy, potential algorithmic bias, and the impact on the clinician-patient dynamic. Addressing these concerns is essential for a future where AI and AR are not only innovative, but also aligned with patient-centered care.


AI, AR, technology, 3D models, image-guided surgery


The integration of artificial intelligence (AI) and augmented reality (AR) into medical practice heralds a transformative era for healthcare, with urology at the vanguard of this revolution. Historically, urology has frequently been an early adopter of innovative technologies, leveraging advancements to enhance both the precision of surgical interventions and the breadth of clinical research. These emerging technologies offer unprecedented opportunities to refine diagnostic acuity, customize therapeutic interventions, and streamline patient care workflows. In urology, AI and AR are not just futuristic concepts but are already underpinning significant improvements in areas such as surgical planning and surgical training [1]. Concurrently, the role of these technologies in patient management and clinical decision making continues to grow, underlined by the promise to personalize treatment pathways and enhance the prognosis of urological conditions. The evolution of AI from basic diagnostic tools to complex predictive models demonstrates a burgeoning synergy between technology and clinical expertise. However, the path to a fully integrated AI and AR-enabled urological practice is fraught with challenges. Issues around costs, patient privacy, and the potential for algorithmic bias must be carefully navigated. Additionally, there is a need to reconcile the rapid pace of technological innovation with the imperative of maintaining the sanctity of the clinicianpatient relationship.
This mini-review scrutinizes the state-of-the-art AI and AR applications in urology, unpacks the challenges hindering their ubiquitous adoption, and sparks a dialogue on the ongoing debates about ethical implications, potential risks, and the overarching impact on patient care.

Materials and methods

We designed a mini narrative review. A bibliographic search was conducted in January 2024 using PubMed/MEDLINE, Scopus, and Web of Science databases. Special attention was given to articles published in the last 5 years. Additional articles were identified by manually cross-referencing the bibliographies of selected articles. Article selection was conducted by two independent reviewers (L.C.L. and E.B.), and final inclusion was based on a consensus discussion involving a third senior author (R.A.).


Artificial intelligence

AI is commonly defined as the set of computational technologies that emulate intellectual processes typically associated with human cognition, such as reasoning, learning, and problem solving [2]. Human intelligence can be metaphorically described as the union of "software" with its "hardware"; the "software" includes the algorithms and knowledge we use to process information, which can be enhanced throughout life. Conversely, the "hardware" represents the physical structure of the human brain, which is less susceptible to rapid changes compared to software, due to the gradual pace of biological evolution over generations. Interestingly, AI is not restricted by such evolutionary constraints. It has the unique capability to upgrade both its "software" and "hardware" without the need for generational evolution. This reflects the progression of AI development, which has advanced from merely replicating basic human cognitive functions to enhancing and expediting these processes while adeptly handling extensive data, culminating in the creation of sophisticated, advanced systems. When integrated into healthcare, AI aligns with the core principles of scientific research, which involve the meticulous processing of clinical data—the "input"—and the thorough examination of clinical observations—the "output"— to establish and validate interconnected patterns. There are several subfields of AI [3, 4]:
● Machine learning (ML) integrates computer science, statistics, and mathematics to develop algorithms capable of predicting outcomes. These algorithms excel over traditional statistics by focusing on predictions rather than relationships between variables and continuously improve by learning from additional data.
● Deep learning (DL), an advanced branch of ML, utilizes artificial neural networks to analyze data with a complexity that resembles the human brain. DL systems identify patterns and features within medical imagery, often discovering diagnostic indicators that human analysis might miss.
● Big data encompasses vast data sets that traditional software cannot handle. In healthcare, AI processes this data to identify disease patterns, predict disease progression, and aid in drug development. The unlimited potential offered by the evolution of these models has, over the years, found extensive applications in the field of urology, ranging from functional urology to uro-oncology.
In the realm of urolithiasis, the main applications of AI have varied from outcomes prediction to diagnosis and therapy [5]. The prediction of outcomes is the most extensively studied area; stone-free status, the detection of infection, the optimization of kidney stone fragmentation, and the prediction of stone patients' health-related quality of life are some of the outcomes investigated. Predictive applications extend to the differential diagnosis and even to predictions of stone composition. Advances have also been made in the surgical field, such as improving the surgical performance by refining protocols, stone localizations, and patient selection [6, 7]. Urogynecology has also adopted ML systems, particularly for assessing functional outcomes and postoperative results [8]. Predictive models have been employed to evaluate the occurrence of complications such as stress urinary incontinence following prolapse surgery, conditions recurrence, and overall outcomes of surgical interventions. Among benign urological conditions, the heterogenicity of benign prostatic hyperplasia (BPH) has provided a broad scope for the integration of AI. Computer vision-based systems have been evaluated for their diagnostic accuracy in identifying BPH histologically, with results showing a high accuracy level. Neural networks have been applied to support the prediction of complications following BPH-related surgeries, to examine the risk of worsening symptoms, and to analyze the contributing risk factors. Additionally, these networks have been instrumental in developing models to predict patient responses to medical treatments [6]. Urological cancer has reaped the greatest benefits from the development of AI. More data, more connections, and generally more inputs allow the potential of machine learning and its derivatives to be fully realized. Hence, it is clear how urological cancer provides an immense array of factors that are well-suited to the advantages of these new technologies, ranging from pathogenesis and risk factors to diagnosis, treatment decisions, and outcome predictions [9].
In the diagnostic field, major advancements have been achieved in the management of Prostate cancer (PCa), including the use of clinico-pathological data to differentiate between organ-confined and non-organ-confined PCa [10, 11]. Algorithms have been developed to predict Gleason Scores based on MRI studies or to detect PCa on digitized pathology images. Similar concepts have been applied to urothelial and renal cell carcinoma (RCC). Diagnosis and staging of urothelial cancer now potentially rely on imaging and biomarkers, while metabolomic data assist in differentiating between RCC and healthy tissue [12]. These concepts have also been applied to the prediction of oncological outcomes such as disease recurrence, survival analysis, and the guidance of management decision-making and therapy selection [13]. The continual advancement of AI in urology signifies a transformative shift towards data-driven precision medicine (Figure 1). By harnessing sophisticated algorithms and computational analytics, we are not only redefining the paradigms of diagnosis and treatment but also empowering clinicians with predictive insights that could lead to more effective, individualized patient care strategies, ultimately enhancing outcomes and elevating the standard of urological practice.

Figure 1. Integrating artificial intelligence into reality: a schematic overview.

Augmented reality

AI algorithms have also a critical role in extended reality (XR) technologies implementation, that provides clinicians with data-driven insights for decision-making and real-time precision-guided interventions.
XR encompasses a spectrum of immersive technologies that alter and enhance the traditional interface between the user and the computer, each with distinct applications in urology.
● Virtual reality (VR) immerses users completely within a digital environment, isolating them from the physical world.
● Augmented reality (AR) on the other hand, superimposes digital images onto the user's real-world view, typically through digital interfaces that allows for clear visibility of the environment.
● Mixed reality (MR) combines elements of both AR and VR, merging real and virtual worlds to produce new environments where physical and digital objects co-exist and interact in real time.
XR's role in urology extends to education and skill development, where devices like VR headsets and optical headmounted displays (OHMDs) create immersive learning scenarios [14]. VR has progressed from facilitating the practice of fundamental, discrete skills to enabling comprehensive procedural simulations. Likewise, AR offers to urology trainees the opportunity to engage with highly detailed, interactive models, thereby enriching their learning experience and surgical training. The validated fields of application range from uro-oncology to endourology and andrology; however, there is a lack of validated VR simulators for the management of urological emergencies [15].
Despite issues with the quality of studies available and the cost-effectiveness of these advanced training devices compared to the "standard" ones, the early results are largely in favor of the integration of XR simulations in surgical training [16]. Recent advances have led to an even more revolutionary application of AR and MR in the field of Urology. As a matter of fact, the integration of these technologies is actively being explored as a potential game-changer for urologic surgeries [17]. A growing body of evidence suggests that AR and MR can significantly enhance the interactivity of preoperative planning and patient education by integrating standard imaging with patient-specific 3D models [18]. These models can be either printed, visualized through AR, or displayed on 3D/2D computer monitors, offering a more comprehensive and tailored surgical planning and educational process for both surgeons and patients alike. An area where this technology has found an ideal application is in nephron-sparing renal surgery, where the integration of advanced information about tumor position and vascularization and relations with vital structures has been shown to enhance preoperative planning and surgeon's confidence, as well as the patient's understanding of their condition [19]. Moreover, AR can be applied during a surgical intervention to permit augmented visualization directly within the surgical field, thereby integrating seamlessly with the clinician's workflow. By enabling the superimposition of diagnostic preoperative images onto the operative field, AR offers enhanced visual guidance for identification of anatomical structures during complex procedures. Additionally, it can assist in minimizing the risk of complications by indicating the real-time location of fragile structures [20]. Once again, a highly researched field of application has been that of partial nephrectomies, but various uro-oncological and reconstructive procedures have also been evaluated [21].

Challenges and ethical concerns

As we embrace the integration of AI and AR in urology, we confront several challenges and ethical dilemmas that must be addressed to ensure responsible adoption and application of these technologies [22, 23]. One of the primary challenges is data privacy. As healthcare increasingly relies on digital data, ensuring the confidentiality and security of patient information is paramount. AI systems require vast datasets for training and validation, which raises concerns about the potential for data breaches and misuse. Potential biases in AI algorithms also pose significant challenges. AI models can inadvertently perpetuate existing biases present in their training data, leading to disparities in patient care. This is particularly concerning in medicine, where algorithms may influence decisionmaking in patient diagnosis and treatment. The implications of automation on clinical decision-making are also a critical consideration. There is a delicate balance to be struck between leveraging AI for more accurate diagnoses and treatment plans and ensuring that these technologies support, rather than overshadow, the clinician's expertise. Furthermore, as AI and AR tools become more prevalent, the medical community must grapple with their impact on the clinician-patient relationship. There is a risk that over-reliance on technology could depersonalize care and diminish the role of the clinician's judgment, experience, and personal interaction with the patient.

Future perspectives

As AI and AR technologies continue to advance, their potential to transform urological practice becomes increasingly evident [24]. The precise diagnostics, tailored treatment protocols, and improved patient care facilitated by these technologies could usher in a new era of urology defined by enhanced precision and individualized care [25]. Harnessing AI and AR for the benefit of patients means not only adopting these technologies but also training clinicians to use them effectively [26]. Education and training will be critical to maximize the advantages that AI and AR offer, ensuring that clinicians are equipped to incorporate these tools into practice.
The future will also require a continuous dialogue within the medical community to ensure these technologies are used to complement human expertise, not replace it. As we look to integrate AI and AR into urology, we must also maintain a commitment to the ethical use of technology, upholding standards that prioritize patient well-being and trust. Accessibility is a crucial factor in the widespread adoption of AI and AR in urology. These technologies must be available to a broad range of healthcare settings, including under-resourced areas, to avoid exacerbating healthcare disparities.
Finally, we must consider the long-term implications of these technologies on the healthcare system. This includes evaluating the cost-effectiveness of AI and AR in urology, assessing their impact on healthcare delivery, and ensuring they align with the overarching goals of patient-centered care.


AI and AR technologies herald a new era in urology, characterized by enhanced precision, improved patient outcomes, and more personalized care. However, as we navigate through this technological revolution, it is imperative to address the ethical, social, and practical challenges that accompany the integration of these technologies into clinical practice. By fostering a balanced approach that considers both the potential benefits and the limitations of AI and AR, we can ensure that the future of urology is both innovative and patient centered.


Financial support and sponsorship


Conflicts of interest

Riccardo Autorino is a member of the editorial board of Uro-Technology Journal. The authors declare that they have no conflicts and were not involved in the journal's review or decision regarding this manuscript.


1. Gómez Rivas J, Toribio Vázquez C, Ballesteros Ruiz C, Taratkin M, Marenco JL, Cacciamani GE, et al. Artificial intelligence and simulation in urology. Actas Urol Esp, 2021, 45(8): 524-529. [Crossref]

2. Hameed BMZ, AVL SD, Raza SZ, Karimi H, Khanuja HS, Shetty DK, et al. Artificial intelligence and its impact on urological diseases and management: a comprehensive review of the literature. J Clin Med, 2021, 10(9): 1864- 1874. [Crossref]

3. Brodie A, Dai N, Teoh JY, Decaestecker K, Dasgupta P, & Vasdev N. Artificial intelligence in urological oncology: An update and future applications. Urol Oncol, 2021, 39(7): 379-399. [Crossref]

4. Salem H, Soria D, Lund JN, & Awwad A. A systematic review of the applications of Expert Systems (ES) and machine learning (ML) in clinical urology. BMC Med Inform Decis Mak, 2021, 21(1): 223-233. [Crossref]

5. Hameed BMZ, Somani S, Keller EX, Balamanigandan R, Mahapatra S, Pietropaolo A, et al. Application of virtual reality, augmented reality, and mixed reality in endourology and urolithiasis: an update by YAU endourology and urolithiasis working group. Front Surg, 2022, 9: 866946. [Crossref]

6. Bouhadana D, Lu XH, Luo JW, Assad A, Deyirmendjian C, Guennoun A, et al. Clinical applications of machine learning for urolithiasis and benign prostatic hyperplasia: a systematic review.J Endourol, 2023, 37(4): 474-494. [Crossref]

7. Checcucci E, Amparore D, Volpi G, Piramide F, De Cillis S, Piana A, et al. Percutaneous puncture during PCNL: new perspective for the future with virtual imaging guidance. World J Urol, 2022, 40(3): 639-650. [Crossref]

8. Shah M, Naik N, Somani BK, & Hameed BMZ. Artificial intelligence (AI) in urology-current use and future directions: an iTRUE study. Turk J Urol, 2020, 46(Supp. 1): S27-S39. [Crossref]

9. Chen J, Remulla D, Nguyen JH, Dua A, Liu Y, Dasgupta P, et al. Current status of artificial intelligence applications in urology and their potential to influence clinical practice. BJU Int, 2019, 124(4): 567-577. [Crossref]

10. Froń A, Semianiuk A, Lazuk U, Ptaszkowski K, Siennicka A, Lemiński A, et al. Artificial intelligence in urooncology: what we have and what we expect. Cancers, 2023, 15(17): 4282-4292. [Crossref]

11. Bianchi L, Chessa F, Angiolini A, Cercenelli L, Lodi S, Bortolani B, et al. The use of augmented reality to guide the intraoperative frozen section during robot-assisted radical prostatectomy. Eur Urol, 2021, 80(4): 480-488. [Crossref]

12. Kowalewski KF, Egen L, Fischetti CE, Puliatti S, Juan GR, Taratkin M, et al. Artificial intelligence for renal cancer: From imaging to histology and beyond. Asian J Urol, 2022, 9(3): 243-252. [Crossref]

13. Checcucci E, Autorino R, Cacciamani GE, Amparore D, De Cillis S, Piana A, et al. Artificial intelligence and neural networks in urology: current clinical applications. Minerva Urol Nefrol, 2020, 72(1): 49-57. [Crossref]

14. Alrishan Alzouebi I, Saad S, Farmer T, & Green S. Is the use of augmented reality-assisted surgery beneficial in urological education? A systematic review. Curr Urol, 2021, 15(3): 148-152. [Crossref]

15. Chahal B, Aydin A, & Ahmed K. Virtual reality vs. physical models in surgical skills training. An update of the evidence. Curr Opin Urol, 2024, 34(1): 32-36. [Crossref]

16. Suresh D, Aydin A, James S, Ahmed K, & Dasgupta P. The role of augmented reality in surgical training: a systematic review. Surg Innov, 2023, 30(3): 366-382. [Crossref]

17. Roberts S, Desai A, Checcucci E, Puliatti S, Taratkin M, Kowalewski KF, et al. “Augmented reality” applications in urology: a systematic review. Minerva Urol Nephrol,2022, 74(5): 528-537. [Crossref]

18. Checcucci E, Amparore D, Fiori C, Manfredi M, Ivano M, Di Dio M, et al. 3D imaging applications for robotic urologic surgery: an ESUT YAUWP review. World J Urol, 2020, 38(4): 869-881. [Crossref]

19. Porpiglia F, Amparore D, Checcucci E, Autorino R, Manfredi M, Iannizzi G, et al. Current use of three-dimensional model technology in urology: a road map for personalised surgical planning. Eur Urol Focus, 2018, 4(5): 652- 656. [Crossref]

20. Pérez-Pachón L, Poyade M, Lowe T, & Gröning F. Image overlay surgery based on augmented reality: a systematic review. Adv Exp Med Biol, 2020, 1260: 175-195. [Crossref]

21. Rodler S, Kidess MA, Westhofen T, Kowalewski KF, Belenchon IR, Taratkin M, et al. A systematic review of new imaging technologies for robotic prostatectomy: from molecular imaging to augmented reality. J Clin Med, 2023, 12(16): 5425-5435. [Crossref]

22. Naik N, Hameed BMZ, Shetty DK, Swain D, Shah M, Paul R, et al. Legal and ethical consideration in artificial intelligence in healthcare: who takes responsibility? Front Surg, 2022, 9: 862322. [Crossref]

23. Gerke S, Minssen T, & Cohen G. Ethical and legal challenges of artificial intelligence-driven healthcare. Artificial intelligence in healthcare, 2020: 295-336.

24. Randazzo G, Reitano G, Carletti F, Iafrate M, Betto G, Novara G, et al. Urology: a trip into metaverse. World J Urol, 2023, 41(10): 2647-2657. [Crossref]

25. Bertolo R, Hung A, Porpiglia F, Bove P, Schleicher M, & Dasgupta P. Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come. World J Urol, 2020, 38(9): 2167-2176. [Crossref]

26. Amparore D, Sica M, Verri P, Piramide F, Checcucci E, De Cillis S, et al. Computer vision and machine-learning techniques for automatic 3D virtual images overlapping during augmented reality guided robotic partial nephrectomy. Technol Cancer Res Treat, 2024, 23: 15330338241229368. [Crossref]

Subscribe to receive issue release notifications
and newsletters from journals