Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitative Review

Novella Martinello; Hannah Loshak

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitative Review

CADTH Rapid Response Report: Summary with Critical Appraisal

Novella Martinello and Hannah Loshak.

Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; 2020 Mar 6.

Abbreviations

RARP: Robot-assisted radical prostatectomy
RALP: Robotic-assisted laparoscopic prostatectomy
RS: Robotic surgery
RSS: Robotic surgical system
OR: Operating room

Context and Policy Issues

Robotic surgery (RS) involves a surgeon at a console operating remote-controlled robotic arms that facilitate the performance of laparoscopic procedures.¹ The surgeon sits un-scrubbed at a console that provides them with a magnified pseudo 3-dimensional (3D) view of the surgical site. From the console, the surgeon is able to control the robot arms that hold the laparoscopic instruments inserted into the patient.² Over the past 20 years, RS has emerged as an alternative minimally invasive surgical strategy.³ One system of surgical robots, da Vinci (Intuitive Surgical, California, USA), has been widely advertised and adopted by surgeons and hospitals since receiving approval from the United States Food and Drug Administration (FDA) in 2000.⁴ There has been rapid growth in the purchase of da Vinci robots and in the number of RS procedures performed annually in North America and worldwide.² Other competing devices have been introduced to the market more recently (e.g., the Versius System or the Hugo System).

While RS is most commonly used in urology and gynecology, its use is expanding across other surgical specialties, such as ear nose and throat, colorectal, cardiology, pediatrics, and plastic and reconstructive surgery.²^,⁵^,⁶ Surgical advantages have been reported with RS including improved dexterity and intuitive instrument handling, reduction/elimination of tremors, motion scaling, and superior visualization including three-dimensional imaging.²^,⁷^,⁸ Systematic reviews and meta-analyses have found the clinical benefits of RS to include less blood loss compared to conventional laparoscopic surgery,⁹^,¹⁰ shorter hospital stays as compared to open surgery and conventional laparoscopic surgery⁹^,¹¹ and evidence indicates RS holds potential for smaller incisions with minimal scarring and faster recovery than nonrobotic-assisted procedures.¹² The CADTH Rapid Response report further outlines the clinical evidence on its Use in Gynecologic Oncology or Urologic Surgery. Overall, the evidence on RS as the superior surgical option is inconclusive and more trials are needed across surgical specialties.

RS has introduced new challenges and additional responsibilities for surgical teams in an already challenging and multifaceted work environment.¹³^,¹⁴ Challenges with RS relate to the complex, highly technical equipment involved;¹³^,¹⁵ patient positioning; the long duration of the procedure;¹⁶^,¹⁷ and the separation of the primary surgeon from the patient.¹³ These challenges may alter the way that members of the surgical team interact, affecting patient safety and quality of perioperative care related to RS.¹³^,¹⁵^–¹⁷ An understanding of the factors that patients value with regards to robotic surgery and surgical teams’ experiences with these robotic surgical systems is needed. This report summarizes the qualitative evidence of the patients’ and surgical teams’ experiences of robotic surgical systems.

Research Questions

How have people undergoing surgery with the use of robotic surgical systems, and the surgical teams using them, experienced engaging with these systems?
1. What are their (both patients and surgical teams) expectations for, and perspectives of, robotic surgical systems?
2. What are their (both patients and surgical teams) experiences relating to decision-making, surgery, recovery, and long-term impact of robotic surgical systems on their lives or care practices?
3. When deciding whether to engage with robotic surgery, what is involved and what do patients and their surgical teams consider?

Key Findings

A total of 14 publications that investigated how surgical teams and patients perceive and experience robotic surgery (RS) were included in this review. All but three studies focused on the perspectives of surgical teams.

There appears to be positive perspectives and excitement around RS and its potential. Surgical staff perceive RS to impact job roles and workflows by increasing job demands and technical knowledge. The set-up phase of RS is particularly time-consuming but is necessary to ensure patient safety and robot functions.

There is a steep learning curve to RS, and comprehensive and uniform training and education should be required for all surgical staff engaged in RS.

The sensory experience of RS is different from, and perhaps an improvement upon, the sensory experience of laparoscopic or open surgery. Tactile feedback is lost for surgeons in RS, creating a reliance on visual cues. The position in RS for the surgeon may be more physically comfortable than in laparoscopic or open surgery.

Appropriate institutional conditions and support and engagement throughout the organization are required for surgical staff to engage with RS. Surgical staff require on-demand technical support for the RS to address technical issues during surgery and to alleviate stress and concerns among staff.

Excellent surgical team dynamics, which include communication, trust, and positive relationships, are necessary to ensure effectiveness, safety, and efficiency during robotic procedures. A whole team training approach or a dedicated RS team were suggested as strategies to ensure team dynamics.

Findings were mixed around surgical teams’ perspectives related to patient outcomes and recovery. Some participants perceived there to be positive benefits to their patients from RS while some participants did not view RS to be suitable for all patients or procedures. There were no findings related to the long-term impact of RS on patient lives or care practices.

Patients require more information, education, and support for decision-making for RS procedures. Evidence points to a potential lack of understanding about RS and patient perspectives may differ by sex; however, due to limited published literature exploring their experiences, the patient perspective was an overall gap in this review.

Methods

Literature Search Methods

A limited literature search was conducted by an information specialist on key resources including Medline and PsycINFO via OVID, and Scopus. The search strategy was comprised of both controlled vocabulary, such as the National Library of Medicine’s MeSH (Medical Subject Headings), and keywords. The main search concept was robotic surgical systems. Search filters were applied to limit retrieval to qualitative studies or studies relevant to the perspectives and experiences of patients and their caregivers. Where possible, retrieval was limited to the human population. The search was also limited to English language documents published between January 1, 2010 and January 23, 2020.

Selection Criteria and Methods

One reviewer screened citations and selected studies. In the first level of screening, titles and abstracts were reviewed and potentially relevant articles were retrieved and assessed for inclusion. The final selection of full-text articles was based on the selection criteria presented in Table 1.

Table 1

Selection Criteria.

Exclusion Criteria

Articles were excluded if they did not meet the selection criteria outlined in Table 1, they were duplicate publications reporting on the same data or findings, or were published prior to 2010.

Critical Appraisal of Individual Studies

The included studies were critically appraised by one reviewer using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist.¹⁸ Results of the critical appraisal were not used to exclude studies from this review; rather, were used to inform interpretation and transferability of findings.

Data Analysis

One reviewer conducted the analysis using the principles of content analysis.¹⁹^,²⁰ Publications were imported into NVivo-12 software (QSR International) and descriptive codes were applied to help sort codes and data. Included publications were first read through as a whole in order to gain a broad understanding of the data. Then, the studies were read through a second time and coded inductively with an examination for concepts, meanings, and related categories. Categories were compared and contrasted, and organized into higher-level themes to produce an overall set of findings. During the analysis, issues with transferability and the results of the critical appraisal were reflected on to aid with interpretation. Articles focusing on patients’ perspectives were summarized instead of synthesized due to the small amount of data and divergent article topics that could not be integrated.

Summary of Evidence

Quantity of Research Available

A total of 1031 citations were identified in the literature search. Following screening of titles and abstracts, 1013 citations were excluded and 18 potentially relevant reports from the electronic search were retrieved for full-text review. No potentially relevant publications were retrieved from the grey literature search for full text review. Of these potentially relevant articles, four publications were excluded for various reasons, and 14 publications met the inclusion criteria and were included in this report. Appendix 1 presents the PRISMA²¹ flowchart of the study selection. Additional references of potential interest are provided in Appendix 5.

Summary of Study Characteristics

Details regarding the characteristics of included publications and their participants are provided in Appendix 2 and Appendix 3.

Study Design (and Data Collection)

Of the 14 included studies, four reported using a qualitative approach,⁵^,¹⁴^,²²^,²³ three reported using realist evaluation,²^,²⁴^,²⁵ one used convergent mixed-methods,²⁶ and one used hermeneutic phenomenology.²⁷ The remaining five did not report study design but reported using semi-structured interviews as the method of data collection.²⁸^–³²

Country of Origin

Five of the included studies were from the UK,²^,²⁴^,²⁵^,²⁷^,²⁸, four were from the USA,¹⁴^,²⁹^,³⁰^,³², one from Turkey,²² one from Australia,²⁶ one from South Korea,²³ one from Norway⁵, and one study reported occurring across Europe.³¹

Population

Two articles included a patient population,²⁶^,²⁷ one included a general population,²⁸ and the remaining studies included surgical staff. Of the articles focused on surgical staff, three studies included a mixed surgical team,²^,²⁴^,²⁵ two studies included surgeons,³¹^,³² two studies included nurses²²^,²³, two studies included both surgical residents and attendings,²⁹^,³⁰ one study included both anesthesiologists’ and nurse anesthetists,⁵ and one study included both perioperative nurses and nurse anaesthetists.¹⁴

Interventions (and Comparators)

Three studies specified a focus on the da Vinci surgical system²⁹^,³⁰^,³² and the remainder did not specify a RS system.

Three studies focused on colorectal surgery,²^,²⁴^,²⁵ two studies focused on prostatectomy,²⁶^,²⁷ and one study focused on urology,³¹ while the remainder did not specify a surgical specialty or procedure.

Summary of Critical Appraisal

Overall, the included body of evidence was assessed to be of good quality. Details of the critical appraisal can be found in Appendix 4. All studies but one²⁶ clearly stated the research objectives in the introduction or background sections of the paper. In all studies, the qualitative methodology was appropriate to address the aims of the research and to explore the experiences of the participants. The main methodological issue in the body of evidence was a lack of consideration of the relationship between researcher and participants, which was only addressed in two studies.²⁴^,²⁵ Seven studies²^,⁵^,¹⁴^,²³^–²⁵^,²⁷ appropriately discussed and justified the research design. All but two studies²⁶^,³¹ included an adequate discussion of participant selection and recruitment, clearly described the methods of data collection and discussed the concept of saturation, and adequately described the data analysis procedure and provided ample data to support the findings.²⁶^,³¹ Seven studies took into consideration potential ethical issues.²^,⁵^,¹⁴^,²³^–²⁵^,³² Two articles were of poor quality (rating a “no” for almost all criteria)²⁶^,³¹ and contributed little to the interpretation of findings, largely due to unclear descriptions of findings and a lack of relevance to the current review.

Summary of Findings

Surgical staff’s experiences with robotic surgical systems

Participants in three studies expressed positive attitudes towards and perceived usefulness of RS, such as elimination of hand tremor, better visualization, and increased precision.¹⁴^,²⁵^,³² Participants in one study reported that the da Vinci surgical robot is a better tool compared to conventional laparoscopic surgery in that it provides the surgeon with better dexterity and control during operation.¹⁴ Surgical staff described a sense of pride or excitement at the new innovation in their organization, enthusiasm at the opportunity to learn new technologies, and hopefulness for the potential enhanced functions of RS compared to laparoscopic and open surgery.¹⁴^,³²

Surgical staff noted the impact of RS on practice. RS created a change in roles and surgical workflow.⁵^,¹⁴^,²²^,²³^,³² Surgical staff described new demands and challenges in their roles and longer operation durations. The RS set-up phase, which includes robot set-up and docking and patient positioning, was described as time-consuming and as having an impact on overall workflow.⁵^,²³^,³² While time-consuming, nurses described the set-up phase as critical to ensuring patient safety and robot functions.¹⁴^,²³ Surgical nurses described RS as increasing responsibility and demand in their roles because it requires increased technical knowledge and expanded surgical duties.²²^,²³ Nurses in one study described their increased responsibilities coinciding with a lack of clarity in the scope of their role during RS, suggesting a need for role clarity for the RS nurse.²²

Training and education was a key theme underlying the experience of RS among surgical staff. There is a documented steep learning curve to RS¹⁴^,³¹^,³² and a comprehensive training and education program is required to address this. There was an identified need for compulsory, supervised, structured and uniform training and education on RS,¹⁴^,³¹^,³² not only for surgeons,³² but for surgical nurses,¹⁴^,²²^,²³ surgical residents,²⁹ and anesthesia professionals.⁵ Nurses emphasized that on-the-job training is not sufficient.¹⁴^,²²^,²³ Components of training described by participants could include knowledge training, simulations, observations, table assisting, and basic skills.²⁹^–³¹ One nurse described the experience of taking part in a simulation:

“We actually went to [x] center. They have a pig lab. They had multiple DaVinci units set up. We were actually able to mimic the whole surgical procedure from draping to docking, to assisting intraoperatively, to completing the robotic surgery on pigs. It was very interesting. I had never been a part of that kind of thing before.” ¹⁴(participant quotation)

Factors related to the sensory experience of using RS were noted by participants. Surgeons in two studies discussed the lack of tactile feedback in RS.²^,³² While surgeons in one study related the loss of tactile information to the experience of laparoscopic surgery,² surgeons in both studies confirmed that this loss of tactile feedback means it is critical to adapt quickly to relying on visual cues.²^,³² The majority of surgeons in one study agreed that the robot creates a sense of immersion, leading to what they perceived to be reduced situational awareness. This may be because it requires heightened concentration compared to laparoscopic or open surgery.² A strategy to address this described by operating room (OR) teams in one study was positioning the console so that the surgeon has a direct view of the patient and the assistant when they look up from the robot.² Finally, the RS was described by surgeons as ergonomically and physically better than laparoscopic or open surgery because they are sitting as opposed to standing which can reduce fatigue and stress and is more comfortable.²

Factors influencing adoption and use of robotic surgical systems

Surgeons and OR staff described facilitators to use and adoption of RS, including having a dedicated or suitable OR for RS,²⁵ having a surgical assistant experienced in RS,²⁴ and perceived support for RS from surgical colleagues.²⁴ Both surgeons and surgical nurses described the availability of technical support for the robot as a facilitator for use and adoption of a RS system.¹⁴^,²²^,²³^,³² There is the possibility for technical issues to arise, and having a technical support representative available on-site or by phone to troubleshoot is key. The availability of technical support works to alleviate stress and improve confidence levels in case technical issues arise in the OR. One focus group study of perioperative nurses described the lack of a technical support system as creating concern among nurses about unexpected situations related to problems with machine errors.²³

Surgeons and OR surgical staff described barriers to use of RS. One barrier was a low volume of RS patient cases, which hampered surgical experience and skill in RS, especially among surgical residents.²³^,²⁹^,³² Another barrier included the stress of performing RS, which can be dependent on the type of surgery being performed.² OR nurses also described longer turnover time between surgeries as a challenge of using RS:¹⁴

“Our biggest struggle, I am sure like other institutions is staffing especially with ancillary personnel and trying to get them into the room and get the room cleaned and reduce the turned over so the room is ready for the next patient. So, I think that is what we struggle with is the time factor more than anything else…the time between the cases.” ¹⁴(participant quotation)

Two strategies described which could potentially address described barriers included having a team leader or two surgeons partnering on the surgery. Having a team leader to coordinate and define roles, especially in the set-up phase, could improve efficiency, cooperation, and team trust and confidence during RS.⁵^,²⁵^,³² Some surgeon participants stated that they shared the operation with a colleague and this strategy reduced their levels of stress around performing RS.²

Organizational-level factors impacted decision-making about whether to adopt a RS system. Surgical staff discussed the importance of involving and engaging staff at multiple levels of the organization and creating a shared vision around RS systems, as opposed to the implementation of a RS system being surgeon-led,²⁵ in order to create the conditions to accommodate the introduction of the technology. This included ensuring there is board-level and surgical staff support, the availability of comprehensive training, and ensuring the right skill set is available. Surgical staff viewed a RSS as allowing the hospital to be more competitive, in that it attracted patients and surgeons and was perceived as a mark of prestige.²⁵^,³²

Team dynamics was an overarching theme in the findings. Good team communication and team trust were seen as essential parts of robotic surgery.²^,⁵^,¹⁴^,²²^,²⁹ Because the surgeon is seated behind the console separate from the rest of the surgical team, team trust and communication between the team and the surgeon is more important in RS than in laparoscopic or open surgery. The surgeon has to rely on the rest of the team to communicate information outside of their field of vision to avoid complications, reduce distraction, and increase concentration. Surgeons require communication about both the state of the patient and the state of the robot.

“If the surgical tech says, “I see an issue here or maybe we need to go here, or somebody moved the arm this way,” the surgeon is much more open… in these robotic surgery cases than they are in probably any other cases…, it’s just a unique relationship…, there is a lot of mutual trusts. When they’re [surgeons] sitting at the console, they have to trust in what we’re doing and what we should be doing.” ¹⁴(participant quotation)

There needs to be a positive relationship between surgeon and team. That relationship may be impacted by the way in which RS is introduced, and it was suggested that whole team training or having a dedicated RS team could be important strategies to establishing team trust and positive team relationships.²^,⁵^,²⁵ One nurse described their experience of whole team training:

“[During training together] we learned to trust each other. We came back from Strasbourg with that certain knowledge that between us we knew we would each remember something and we would be able to pull it [robot-assisted surgery] off…we seemed to develop a special bond.” ²⁵ (participant quotation)

Some participants suggested the dedicated team could be handpicked based on interest or enthusiasm²⁵ or based on experience and skills such as being prompt and practical, anticipating the next step in the procedure, having no panic, being knowledgeable, and possessing dexterity and foresight.²²

Perspectives on patient outcomes and recovery in RS were mixed among surgical staff. Participants were attracted to the potential positive outcomes provided to their patients by performing surgery robotically, particularly prostatectomy, such as less bleeding, smaller incisions and nerve and tissue sparing.¹⁴^,³² Nurses in one study questioned the suitability of RS for all patients due to the need to be under anesthesia longer because of longer operation times with RS, perceived complications and longer recovery times arising from RS, and did not see the benefits for specific procedures such as robot-assisted hysterectomy or general surgeries such as gallbladder and hernia repairs.¹⁴

“We got surgeons that can do lap-chole in 30 minutes. It is silly to put them through the paces of the robot and spend all that money, and the patient is on the table a little longer…time is money.” ¹⁴ (participant quotation)

Patients’ perspectives and experiences of robot surgical systems

A key theme running throughout the studies was that patients require more information and support when it comes to RS and decision-making around surgery options.²⁶^–²⁸ Among men diagnosed with prostate cancer, there was considerable uncertainty and decision-related distress around treatment options, and participants explained that pre-operative education, information, and support from surgeons, nurses, as well as other patients who have experienced RS is a necessity.²⁶^,²⁷

Among men with prostate cancer undergoing robotic-assisted laparoscopic prostatectomy (RALP), regaining urinary control after catheter removal was the patient’s primary goal. Typically, there was a gradual process of regaining bladder control after the catheter was removed, and all men said they had regained continence by 12 weeks post- RALP.²⁷ The participants viewed incontinence as a trade-off and selected RALP over open and laparoscopic techniques due to their belief that this would be temporary.²⁷

The participants in this study were queried about the psychosocial impact of RALP on personal identity and the individual processes men underwent to reconcile themselves to their new life situation.²⁷ Following RALP, men considered themselves ‘lucky’ to be alive and re-evaluated their lives particularly with regards to their relationships and future goals:

“The cancer itself makes you stop and think a bit about yourself… who you are; what you’re doing; your relationships; other people… what it [cancer] means to you and what place it plays in your life.” ²⁷ (participant quotation)

In an Australian study of patient outcomes after robot-assisted radical prostatectomy (RARP), patients were overall very satisfied with the procedure and would likely recommend it to others.²⁶ The minimally invasive nature of RARP compared to open RP influenced the themes surrounding the entire experience.²⁶

One included study examined male and female perceptions of RS among a general population.²⁸ The majority of female participants expressed concerns in relation with the safety and perception of RS, whereas many male participants appeared to be untroubled by the idea of RS. The lack of acceptance expressed by most female participants appeared to be based on trust. A 26-year-old female stated:

“First of all, it’s the first time I’ve heard that, and as humans, we tend to trust what’s been there traditionally and erm it’s obviously a new method and I am quite sceptical about modern technology and so I would not trust a robot.” ²⁸ (participant quotation)

There were differences in how males and females understood RS. While female participants viewed RS as de-humanizing, males humanized surgical robots and exhibited a sense of anthropomorphism in relation with RS. Although the majority of males had heard of RS, there was a clear lack of understanding as to the surgeon’s role in RS with one 20-year-old male stating:

“It almost calls into question what the point is, if you’ve got a fully qualified surgeon in the room then why leave it up to the robot?” ²⁸(participant quotation)

In this study, the media were described as an important source of information for both male and female participants in relation to attitudes towards the acceptance of RS. For some participants, the media were an effective tool in increasing awareness and understanding and acceptance, but for others, the media resulted in misconceptions about RS.²⁸

Limitations

The key limitation arising from the body of included literature was the lack of patients’ perspectives. All but three included articles²⁶^,²⁸^,³³ focused on the perspectives of surgical staff, and so there is a clear gap in qualitative evidence on the patient experience of RS. Due to the small number of included studies focusing on a patient population, it was not possible to explore differences by patient characteristics. One study explored differences among males and females but sampled a general population,²⁸ and the findings may differ among a sample of patients.

Although no Canadian studies were retrieved and included, there was a wide range of countries in the included studies. There were consistent themes running throughout the studies, suggesting that the key themes related to the experience of RS transfer across contexts and provide implications for the Canadian context as well.

Conclusions and Implications for Decision or Policy Making

This review synthesized the findings of 14 qualitative studies to describe the perspectives and experiences of surgical staff and patients related to RS. This review sheds light on the factors that impact surgical staffs’ engagement with RS and the overall experience of RS in comparison to laparoscopic or open surgery. The patient perspective is lacking in the current review and further qualitative research is needed to understand decision-making, outcomes, and recovery related to RS among a patient population. Ensuring a culturally and economically broad sample of participants would allow for exploration by patient group differences.

The increased work responsibilities, expanded job roles, and impact on workflow suggest that support in this area be established before the introduction of a RS system in an organization. This organizational support could include allowing for longer OR set-up times in scheduling or developing standardized operative forms or check-lists specific to RS.³⁴ These types of measures could improve workflow and efficiency, ensure patient safety, reduce stress among staff due to new technological and knowledge demands, and improve surgical team cooperation and communication.³⁵^–³⁷

The importance of a comprehensive training and education program was identified among surgical staff in multiple countries. Well-structured training programs can contribute to ensuring patient safety and care, and empower staff to feel prepared and confident. Serious complications in robot-assisted laparoscopy, such as lower limb compartment syndrome, seem to be related to suboptimal learning (i.e. inexperienced surgeons and long operating times).³⁸ Programs should be established and required before the implementation of a RS system and organizations may want to consider a whole team training approach to ensure efficiency and positive team dynamics.

The findings also suggest that organizations considering a RS system should explore the type and degree of technical support provided alongside. Reported robotic malfunctions during robotic surgery include malfunction of the setup joint, robot arm, camera and power errors, monocular monitor loss, metal fatigue and malfunction of surgeon’s console hand piece, and software incompatibility.³⁹^–⁴¹ Such problems can result in postponement, delay, or changing the surgery to laparoscopy or open prostatectomy surgery.⁴² The availability of on-demand technical support for the robot was a key facilitator to RS use in the findings and should be an important factor impacting decision-making around the type of system to purchase and adopt.

Overall, it is clear from the findings of this review that the context into which a RS system is introduced is important. This review identified several factors which could influence decision-making around implementation of a RS system in an organization, as well as the facilitators and barriers to the acceptance and use of robots in surgical procedures. These findings can inform decision-making around implementation and assist healthcare organizations to appropriately plan and prepare to employ robotic-assisted surgery in their routine practice.

References

1.: Barbash GI, Glied SA. New technology and health care costs-the case of robot-assisted surgery. N Engl J Med. 2010;363(8):701–704. [PubMed: 20818872]
2.: Randell R, Alvarado N, Honey S, et al. Impact of robotic surgery on decision making: perspectives of surgical teams. AMIA Annu Symp Proc. 2015;2015:1057–1066. [PMC free article: PMC4765621] [PubMed: 26958244]
3.: Wright JD, Burke WM, Wilde ET, et al. Comparative Effectiveness of Robotic Versus Laparoscopic Hysterectomy for Endometrial Cancer. J Clin Oncol. 2012;30(8):783–791. [PMC free article: PMC3295567] [PubMed: 22291073]
4.: Bouquet de Joliniere J, Librino A, Dubuisson J-B, et al. Robotic surgery in gynecology. Front Surg. 2016;3:26–26. [PMC free article: PMC4852174] [PubMed: 27200358]
5.: Myklebust MV, Storheim H, Hartvik M, Dysvik E. Anesthesia professionals’ perspectives of teamwork during robotic-assisted surgery. AORN J. 2020;111(1):87–96. [PubMed: 31886556]
6.: Dobbs TD, Cundy O, Samarendra H, Khan K, Whitaker IS. A systematic review of the role of robotics in plastic and reconstructive surgery—from inception to the future. Front Surg. 2017;4(66). [PMC free article: PMC5694772] [PubMed: 29188219]
7.: Herron DM, Marohn M, The S-MRSCG. A consensus document on robotic surgery. Surg Endosc. 2008;22(2):313–325. [PubMed: 18163170]
8.: Khan MS, Elhage O, Challacombe B, Rimington P, Murphy D, Dasgupta P. Analysis of early complications of robotic-assisted radical cystectomy using a standardized reporting system. Urology. 2011;77(2):357–362. [PubMed: 20828801]
9.: Roh HF, Nam SH, Kim JM. Robot-assisted laparoscopic surgery versus conventional laparoscopic surgery in randomized controlled trials: a systematic review and meta-analysis. PLoS One. 2018;13(1):e0191628. [PMC free article: PMC5779699] [PubMed: 29360840]
10.: Wang Y, Zhao X, Song Y, Cai A, Xi H, Chen L. A systematic review and meta-analysis of robot-assisted versus laparoscopically assisted gastrectomy for gastric cancer. Medicine (Baltimore). 2017;96(48):e8797. [PMC free article: PMC5728759] [PubMed: 29310358]
11.: Gala RB, Margulies R, Steinberg A, et al. Systematic review of robotic surgery in gynecology: robotic techniques compared with laparoscopy and laparotomy. J Minim Invasive Gynecol. 2014;21(3):353–361. [PubMed: 24295923]
12.: Robotic-assisted minimally invasive surgery for gynecologic and urologic oncology: an evidence-based analysis. Ont Health Technol Assess Ser. 2010;10(27):1–118. [PMC free article: PMC3382308] [PubMed: 23074405]
13.: Gill A, Randell R. Robotic surgery and its impact on teamwork in the operating theatre. J Perioper Pract. 2016;26:42–45. [PubMed: 27149832]
14.: Schuessler Z, Scott Stiles A, Mancuso P. Perceptions and experiences of perioperative nurses and nurse anaesthetists in robotic-assisted surgery. J Clin Nurs. 2020;29(1–2):60–74. [PubMed: 31495000]
15.: Harding V, Williams J. Improving efficiency in robotic theatres in the Royal Wolverhampton NHS Trust: a gynaecology theatres perspective. J Perioper Pract. 2016;26(3):46–49. [PubMed: 27149833]
16.: Hsu RL, Kaye AD, Urman RD. Anesthetic challenges in robotic-assisted urologic surgery. Reviews in Urology. 2013;15(4):178–184. [PMC free article: PMC3922322] [PubMed: 24659914]
17.: Kaye AD, Vadivelu N, Ahuja N, Mitra S, Silasi D, Urman RD. Anesthetic considerations in robotic-assisted gynecologic surgery. Ochsner J. 2013;13(4):517–524. [PMC free article: PMC3865830] [PubMed: 24358000]
18.: CASP qualitative checklist. In: Oxford (UK): CASP UK; 2018: https://casp-uk.net/wp-content/uploads/2018/01/CASP-Qualitative-Checklist-2018.pdf. Accessed 2020 Jan 22.
19.: Cavanagh S. Content analysis: concepts, methods and applications. Nurse Res. 1997;4(3):5–16. [PubMed: 27285770]
20.: Krippendorff K. Content analysis: an introduction to its methodology. Thousand Oaks (CA): Sage Publications; 2018.
21.: Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–e34. [PubMed: 19631507]
22.: Uslu Y, Altinbas Y, Ozercan T, van Giersbergen MY. The process of nurse adaptation to robotic surgery: a qualitative study. Int J Med Robot. 2019;15(4):e1996. [PubMed: 30884169]
23.: Kang MJ, De Gagne JC, Kang HS. Perioperative nurses’ work experience with robotic surgery: a focus group study. Comput Inform Nurs. 2016;34(4):152–158. [PubMed: 26848644]
24.: Randell R, Greenhalgh J, Hindmarsh J, et al. How do team experience and relationships shape new divisions of labour in robot-assisted surgery? A realist investigation. Health (Lond). 2019:1363459319874115. [PubMed: 31522572]
25.: Randell R, Honey S, Alvarado N, et al. Factors supporting and constraining the implementation of robot-assisted surgery: a realist interview study. BMJ Open. 2019;9(6):e028635. [PMC free article: PMC6589012] [PubMed: 31203248]
26.: Reynolds BR, Bulsara C, Zeps N, et al. Exploring pathways towards improving patient experience of robot-assisted radical prostatectomy (RARP): assessing patient satisfaction and attitudes. BJU Int. 2018;121: Suppl 3:33–39. [PubMed: 29603580]
27.: Waller J, Pattison N. Men’s experiences of regaining urinary continence following robotic-assisted laparoscopic prostatectomy (RALP) for localised prostate cancer: a qualitative phenomenological study. J Clin Nurs. 2013;22(3–4):368–378. [PubMed: 23301578]
28.: McDermott H, Choudhury N, Lewin-Runacres M, Aemn I, Moss E. Gender differences in understanding and acceptance of robot-assisted surgery. J Robot Surg. 2019;02:02. [PMC free article: PMC7000495] [PubMed: 31049775]
29.: Zhao B, Hollandsworth HM, Lee AM, et al. Making the jump: a qualitative analysis on the transition from bedside assistant to console surgeon in robotic surgery training. J Surg Educ. 2019;23:23. [PMC free article: PMC7036000] [PubMed: 31558428]
30.: Zhao B, Lam J, Hollandsworth HM, et al. General surgery training in the era of robotic surgery: a qualitative analysis of perceptions from resident and attending surgeons. Surg Endosc. 2019;08:08. [PMC free article: PMC6946889] [PubMed: 31286248]
31.: Brinkman WM, Schout BMA, Rietbergen JB, et al. Training robotic surgery in urology: experience and opinions of robot urologists. Int J Med Robot. 2015;11(3):308–318. [PubMed: 25346023]
32.: BenMessaoud C, Kharrazi H, MacDorman KF. Facilitators and barriers to adopting robotic-assisted surgery: contextualizing the unified theory of acceptance and use of technology. PLoS One. 2011;6(1). [PMC free article: PMC3024425] [PubMed: 21283719]
33.: Rebuck DA, Haywood S, McDermott K, Perry KT, Nadler RB. What is the long-term relevance of clinically detected postoperative anastomotic urine leakage after robotic-assisted laparoscopic prostatectomy? BJU Int. 2011;108(5):733–738. [PubMed: 21223475]
34.: Walters L, Eley S. Robotic-assisted surgery and the need for standardized pathways and clinical guidelines. AORN J. 2011;93(4):455–463. [PubMed: 21459183]
35.: Jing J, Honey ML. Using a checklist in robotic-assisted laparoscopic radical prostatectomy procedures. AORN J. 2016;104(2):145–152. [PubMed: 27472974]
36.: Fudickar A, Horle K, Wiltfang J, Bein B. The effect of the WHO Surgical Safety Checklist on complication rate and communication. Dtsch Arztebl Int. 2012;109(42):695–701. [PMC free article: PMC3489074] [PubMed: 23264813]
37.: Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf. 2014;23(4):299–318. [PMC free article: PMC3963558] [PubMed: 23922403]
38.: Pridgeon S, Bishop CV, Adshead J. Lower limb compartment syndrome as a complication of robot-assisted radical prostatectomy: the UK experience. BJU Int. 2013;112(4):485–488. [PubMed: 23879906]
39.: Tugcu V, Mutlu B, Canda AE, Sonmezay E, Tasci AI. Robotic malfunction during live robotic urologic surgery: live surprise in a robotic surgery congress. Arch Ital Urol Androl. 2012;84(4):211–213. [PubMed: 23427746]
40.: Kaushik D, High R, Clark CJ, LaGrange CA. Malfunction of the Da Vinci robotic system during robot-assisted laparoscopic prostatectomy: an international survey. J Endourol. 2010;24(4):571–575. [PubMed: 20192613]
41.: Kim WT, Ham WS, Jeong W, Song HJ, Rha KH, Choi YD. Failure and malfunction of da Vinci Surgical systems during various robotic surgeries: experience from six departments at a single institute. Urology. 2009;74(6):1234–1237. [PubMed: 19716587]
42.: Borden LS, Jr., Kozlowski PM, Porter CR, Corman JM. Mechanical failure rate of da Vinci robotic system. Can J Urol. 2007;14(2):3499–3501. [PubMed: 17466155]

Appendix 1. Selection of Included Studies

Appendix 2. Characteristics of Included Publications

Table 2Characteristics of Included Studies

First Author, Publication Year, Country	Study Design¹	Study Objectives	Sample	Inclusion Criteria	Data Collection
Myklebust, 2020, Norway⁵	Qualitative exploratory study	To explore anesthesiologists’ and nurse anesthetists’ experiences of teamwork during RAS.	3 anesthesiologists and 6 nurse anesthetists	Participants were required to be employed full-time as an anesthesiologist or nurse anesthetist with some experience in RAS	Semi-structured interviews
Schuessler, 2020, USA¹⁴	Qualitative descriptive study	To explore the perceptions and experiences of perioperative nurses and CRNAs in RALS. The objective was to identify the factors that affect nursing care of patients who undergo RALS.	17 nurses	RNs who provided care for patients with RALS.	Semi-structured interviews
McDermott, 2019, UK²⁸	Thematic analysis	To understand male and female perceptions of robot-assisted surgery with the objective of identifying the factors that might inhibit or facilitate the acceptance of robotic surgery.	25 people	Participant selection was made on the basis that they were over the age of 18 and from a variety of ethnic backgrounds.	Semi-structured interviews
Randell, 2019a, UK²⁴	Realist evaluation	To capture stakeholders’ theories concerning how and in what contexts robot-assisted surgery becomes integrated into routine practice.	44 OR staff	OR teams in 10 English hospitals who were using robot-assisted surgery for colorectal surgery, ensuring the OR teams involved in the study varied in their level of experience with robot-assisted surgery.	Semi-structured interviews using the teacher/learner cycle
Randell, 2019b, UK²⁵	Realist evaluation	To investigate how introduction of robot-assisted surgery changes the division of labour within surgical teams and impacts on teamwork and patient safety.	Overall, 32 operations were observed, of which 21 were robot-assisted, constituting 244 hours of data collection, 52 hours of video data, 30 post-operation interviews were undertaken.	Four hospitals were purposively sampled from the nine hospitals that participated in first phase of study.	Data were collected using multiple methods, including video recording of operations, ethnographic observation, and semi-structured interviews
Uslu, 2019, Turkey²²	Qualitative approach	To reveal the experiences of nurses in robotic surgery and their adaptation to this method.	15 nurses	Robotic surgery nurses in the operating rooms of any of four hospitals belonging to a private health group in İstanbul, Turkey.	Focus groups
Zhao, 2019a, USA³⁰	NS	To explore the perceptions of resident and attending surgeons toward robotic surgery education in general surgery residency training.	20 residents and 7 attendings	General surgery residents and attendings in an urban, multi-hospital healthcare system consisting of two university hospitals and a Veterans Affairs (VA) hospital.	Semi-structured interviews
Zhao, 2019b, USA²⁹	NS	To determine barriers associated with the transition from bedside assistant to console surgeon for general surgery residents in the era of robotic surgery in general surgery training.	20 residents and 7 attendings	General surgery residents and attendings in an urban, multi-hospital healthcare system consisting of two university hospitals and a Veterans Affairs hospital.	Semi-structured interviews
Reynolds, 2018, Australia²⁶	Convergent mixed-methods	To determine patient satisfaction and experience after RARP for prostate cancer.	NS	Patients who had undergone RARP (with or without pelvic lymph node dissection) by two high-volume surgeons between a select time period were contacted with regard to potential recruitment to the study.	Focus groups
Kang, 2016, South Korea²³	Qualitative descriptive study	To explore the work experience of perioperative nurses involved in robotic surgery.	15 nurses	Nurses practicing in operating room settings who had been trained for robotic surgery and worked as a member of a robotic surgical team.	Focus groups
Brinkman, 2015, Europe³¹	NS	To understand the training pathways of first generation of robot urologists and their opinions on the ideal training for the future generation.	11 robot urologists	Subscribers to European robot urology section mailing list were invited if they were currently active as robot urologists or if they had previous experience in robot-assisted laparoscopy. Of those who agreed to participate, a cross-European group of robot urologists was selected.	Semi-structured interviews
Randell, 2015, UK²	Realist evaluation	To gather the perspectives of operating room staff on how robotic surgery impacts surgeon decision making	44 operating room staff	All English hospitals participating in a trial were invited to participate in the interview study. English hospitals not participating in the trial but using the robot for colorectal surgery were identified by the trial team and through personal contacts of one of the team members and all were invited to participate in the interview study.	Interviews using the teacher-learner cycle
Waller, 2013, UK²⁷	Hermeneutic phenomenology	To understand how men interpret their experiences of regaining continence following RALP.	7 men	Men > 18 years diagnosed with locally confined, stage pT1, pT2 a, b or c prostate cancer; receiving primary treatment with RALP within the past 12–16 weeks; self-defined as continent of urine; willing and able to articulate their post-RALP experiences of regaining continence in English.	In-depth interviews
BenMessaoud, 2011, USA³²	Content analysis was used in conjunction with the UTAUT model	To help identify the facilitators and barriers to their adoption of RAS among surgeons, this belief-elicitation study contextualized and supplemented constructs of the UTAUT in robotic-assisted surgery.	21 surgeons	Surgeons in Indiana who practice in robotic-assisted surgery subspecialties.	Semi-structured interviews

: NS = not specified; CRNA = Certified Registered Nurse Anesthetist; OR = operating room; RALS = robotic-assisted laparoscopic surgery; RAS = robot assisted surgery; RN = registered nurse; UTAUT = unified theory of acceptance and use of technology; RALP = robotic-assisted laparoscopic prostatectomy; RARP = robot-assisted radical prostatectomy

Appendix 3. Characteristics of Study Participants

Table 3Characteristics of Study Participants

First Author, Publication Year, Country	Sample Size	Age range in years	Other relevant variable(s)
Myklebust, 2020, Norway⁵	9 staff	NS	3 anesthesiologists, 6 nurse anesthetists
Schuessler, 2020, USA¹⁴	17 nurses	30-65	6 preoperative and postoperative nurses, 7 intraoperative nurses, and 4 CRNAs
McDermott, 2019, UK²⁸	25 people	19-26	Sampling to recruit from diverse social and ethnic backgrounds
Randell, 2019a, UK²⁴	44 staff	NS	12 surgeons, 5 trainee surgeons, 1 manager, 6 anesthesiologists 13 OR Nurses, 7 OR Practitioners
Randell, 2019b, UK²⁵	30 staff	NS	NS
Uslu, 2019, Turkey²²	15 nurses	20-37	The duration of the participants’ experience as robotic surgery nurses ranged from 1 year to 10 years.
Zhao, 2019a, USA³⁰	27 staff	NS	20 residents, 7 attendings
Zhao, 2019b, USA²⁹	27 staff	NS	20 residents, 7 attendings
Reynolds, 2018, Australia²⁶	NS	NS	NS
Kang, 2016, South Korea²³	15 staff	25-41	NS
Brinkman, 2015, Europe³¹	11 staff	NS	NS
Randell, 2015, UK²	44 staff	NS	12 surgeons, 5 trainee surgeons, 1 manager, 6 anesthesiologists 13 OR Nurses, 7 OR Practitioners
Waller, 2013, UK²⁷	7 patients	57–71	All had recently undergone RALP and defined themselves as continent of urine
BenMessaoud, 2011, USA³²	21 staff	NS	4 of 7 OB/GYN and 4 of 7 urology surgeons were robot users. 1 of 4 surgeons in the cardiovascular specialty and 1 of 3 in general surgery were robot users.

: NS = not specified; CRNA = Certified Registered Nurse Anesthetist; RALP = robotic-assisted laparoscopic prostatectomy; OR = operating room

Appendix 4. Critical Appraisal of Included Publications

Table 4Critical Appraisal of Included Publications Using CASP Qualitative Checklist¹⁸

First Author, Year	Clear statement of the aims of the research?	Qualitative methodology appropriate?	Research design appropriate to address the aims of the research?	Recruitment strategy appropriate to the aims of the research?	Data collected in a way that addressed the research issue?	Relationship between researcher and participants been adequately considered?	Ethical issues been taken into consideration?	Data analysis sufficiently rigorous?	Clear statement of findings?	Relevant to the current review?
Myklebust, 2020⁵	+	+	+	+	+	−	+	+	+	+
Schuessler, 2020¹⁴	+	+	+	+	+	−	+	+	+	+
McDermott, 2019²⁸	+	+	−	+	+	−	−	+	+	+
Randell, 2019a²⁴	+	+	+	+	+	+	+	+	+	+
Randell, 2019b²⁵	+	+	+	+	+	+	+	+	+	+
Uslu, 2019²²	+	+	−	+	+	−	+	+	+	+
Zhao, 2019a³⁰	+	+	−	+	+	−	−	+	+	+
Zhao, 2019b²⁹	+	+	−	+	+	−	−	+	+	+
Reynolds, 2018²⁶	−	+	−	−	−	−	−	−	−	−
Kang, 2016²³	+	+	+	+	+	−	+	+	+	+
Brinkman, 2015³¹	+	+	−	−	−	−	−	−	−	−
Randell, 2015²	+	+	+	+	+	−	+	+	+	+
Waller, 2013²⁷	+	+	+	+	+	−	−	+	+	−
BenMessaoud, 2011³²	+	+	−	+	+	−	+	+	−	−

: + = yes; −= no

Appendix 5. Additional References of Potential Interest

The following references were not included in the current review because their study designs did not meet inclusion criteria, however, may be of interest as they focus on patient perspective and experience.

Barbosa JA, Barayan G, Gridley CM, et al. Parent and patient perceptions of robotic vs open urological surgery scars in children. J Urol. 2013;190(1):244–250. [PubMed: 23276511]
Chu CM, Agrawal A, Mazloomdoost D, et al. Patients’ knowledge of and attitude toward robotic surgery for pelvic organ prolapse. Female Pelvic Med Reconstr Surg. 2019;25(4):279–283. [PubMed: 29369840]
Collingwood SA, McBride RB, Leapman M, et al. Decisional regret after robotic-assisted laparoscopic prostatectomy is higher in African American men. Urol Oncol. 2014;32(4):419–425. [PubMed: 24411791]
Freilich DA, Penna FJ, Nelson CP, Retik AB, Nguyen HT. Parental satisfaction after open versus robot assisted laparoscopic pyeloplasty: results from modified Glasgow Children’s Benefit Inventory Survey. J Urol. 2010;183(2):704–708. [PubMed: 20022046]
Irani M, Prabakar C, Nematian S, Julka N, Bhatt D, Bral P. Patient perceptions of open, laparoscopic, and robotic gynecological surgeries. Biomed Res Int. 2016;2016:4284093. [PMC free article: PMC5093233] [PubMed: 27840826]
Long E, Kew F. Patient satisfaction with robotic surgery. J Robot Surg. 2018;12(3):493–499. [PubMed: 29288373]
McNanley AR, Duecy EE, Flynn MK, Buchsbaum GM. Recovery after robot-assisted laparoscopic sacrocolpopexy: the patients’ perspective. J Robot Surg. 2010;4(1):1–5. [PubMed: 27638564]

About the Series

CADTH Rapid Response Report: Summary with Critical Appraisal

ISSN: 1922-8147

Version: 1.0

Funding: CADTH receives funding from Canada’s federal, provincial, and territorial governments, with the exception of Quebec.

Suggested citation:

Experiences with and expectations of robotic surgical systems: a rapid qualitative review. Ottawa: CADTH; 2020 Mar (CADTH rapid response report: summary with critical appraisal).

Disclaimer: The information in this document is intended to help Canadian health care decision-makers, health care professionals, health systems leaders, and policy-makers make well-informed decisions and thereby improve the quality of health care services. While patients and others may access this document, the document is made available for informational purposes only and no representations or warranties are made with respect to its fitness for any particular purpose. The information in this document should not be used as a substitute for professional medical advice or as a substitute for the application of clinical judgment in respect of the care of a particular patient or other professional judgment in any decision-making process. The Canadian Agency for Drugs and Technologies in Health (CADTH) does not endorse any information, drugs, therapies, treatments, products, processes, or services.

While care has been taken to ensure that the information prepared by CADTH in this document is accurate, complete, and up-to-date as at the applicable date the material was first published by CADTH, CADTH does not make any guarantees to that effect. CADTH does not guarantee and is not responsible for the quality, currency, propriety, accuracy, or reasonableness of any statements, information, or conclusions contained in any third-party materials used in preparing this document. The views and opinions of third parties published in this document do not necessarily state or reflect those of CADTH.

CADTH is not responsible for any errors, omissions, injury, loss, or damage arising from or relating to the use (or misuse) of any information, statements, or conclusions contained in or implied by the contents of this document or any of the source materials.

This document may contain links to third-party websites. CADTH does not have control over the content of such sites. Use of third-party sites is governed by the third-party website owners’ own terms and conditions set out for such sites. CADTH does not make any guarantee with respect to any information contained on such third-party sites and CADTH is not responsible for any injury, loss, or damage suffered as a result of using such third-party sites. CADTH has no responsibility for the collection, use, and disclosure of personal information by third-party sites.

Subject to the aforementioned limitations, the views expressed herein do not necessarily reflect the views of Health Canada, Canada’s provincial or territorial governments, other CADTH funders, or any third-party supplier of information.

This document is prepared and intended for use in the context of the Canadian health care system. The use of this document outside of Canada is done so at the user’s own risk.

This disclaimer and any questions or matters of any nature arising from or relating to the content or use (or misuse) of this document will be governed by and interpreted in accordance with the laws of the Province of Ontario and the laws of Canada applicable therein, and all proceedings shall be subject to the exclusive jurisdiction of the courts of the Province of Ontario, Canada.

The copyright and other intellectual property rights in this document are owned by CADTH and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to CADTH and its licensors.

Except where otherwise noted, this work is distributed under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence (CC BY-NC-ND), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/

Bookshelf ID: NBK562938PMID: 33074607

PubReader
Print View
Cite this Page
Martinello N, Loshak H. Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitative Review [Internet]. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; 2020 Mar 6.
PDF version of this title (657K)

In this Page

Context and Policy Issues
Research Questions
Key Findings
Methods
Summary of Evidence
Conclusions and Implications for Decision or Policy Making
References
Selection of Included Studies
Characteristics of Included Publications
Characteristics of Study Participants
Critical Appraisal of Included Publications
Additional References of Potential Interest

Other titles in this collection

CADTH Rapid Response Reports

Related information

NLM Catalog
Related NLM Catalog Entries
PMC
PubMed Central citations
PubMed
Links to PubMed

Recent Activity

Clear Turn Off Turn On

Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitati...
Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitative Review

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Bookshelf

Experiences with and Expectations of Robotic Surgical Systems: A Rapid Qualitative Review

Abbreviations

Context and Policy Issues

Research Questions

Key Findings

Methods

Literature Search Methods

Selection Criteria and Methods

Table 1

Exclusion Criteria

Critical Appraisal of Individual Studies

Data Analysis

Summary of Evidence

Quantity of Research Available

Summary of Study Characteristics

Study Design (and Data Collection)

Country of Origin

Population

Interventions (and Comparators)

Summary of Critical Appraisal

Summary of Findings

Limitations

Conclusions and Implications for Decision or Policy Making

References

Appendix 1. Selection of Included Studies

Appendix 2. Characteristics of Included Publications

Table 2Characteristics of Included Studies

Appendix 3. Characteristics of Study Participants

Table 3Characteristics of Study Participants

Appendix 4. Critical Appraisal of Included Publications

Table 4Critical Appraisal of Included Publications Using CASP Qualitative Checklist18

Appendix 5. Additional References of Potential Interest

About the Series

Suggested citation:

Views

In this Page

Other titles in this collection

Related information

Similar articles in PubMed

Recent Activity

Table 4Critical Appraisal of Included Publications Using CASP Qualitative Checklist¹⁸