Learning Surgical Skills Through Video-Based Education: A Systematic Review

Background Educational videos are a potent resource for the learning of surgical skills among different study cohorts. However, there is limited evidence on the effectiveness of different educational video interventions and their features. 
Methods
 A systematic search of MEDLINE (via PubMed), Embase (via OVID), Cochrane libraries and Clinicaltrials.gov was performed from inception to 28/02/21. Studies included were not limited by date of publication, studies aiming to assess the impact of video-based interventions in the direct acquisition of surgical skill were included. Eligible studies were analysed based on study type, type of video intervention, method of assessment and period of education. The educational impact of the studies was also assessed as per Messick’s framework for testing validity of evaluation methods and McGhagie’s model for analysing translational outcomes. 
Results
 22 studies were deemed suitable for inclusion, of which 14/22 (63.6%) demonstrated a significant improvement in knowledge/skills following the video-based teaching interventions, 3/22 (13.6%) studies demonstrated an improvement in trainee satisfaction scores. A recurrent limitation of the included studies was the lack of validation of selected assessment methods. None of the included studies scored on all 5 parameters of validity as defined by Messicks validity framework. Furthermore, none of the included trials were conducted for long enough to indicate direct changes to patient outcomes resultant from educational methods. Conclusion Video-based surgical education is effective in learning surgical skills within different levels of surgical training; however, superior study quality and follow-up is required to determine which aspects of video-based interventions are most impactful.


Background
To date, the current model by which surgeons are trained relies heavily on apprenticeship-based Halstedian frameworks, 1 through observation trainee surgeons acquire the knowledge, technical skills and critical judgement to successfully operate on patients. Although surgical expertise is integral to healthcare, challenges exist in the training of competent surgeons such as the implementation of work time directives, 2 reduced opportunities for performing core procedures 3,4 and budget cuts in healthcare organisations limiting investments into surgical training. 5 With the advent of new video recording technologies, and improved computer systems, opportunity stands for innovation in surgical education. The use of educational videos offers many significant advantages including remote learning on demand, cost effectiveness, saved time and transport fees for healthcare professionals and students. Furthermore, with the development of validated checklists in the assessment of surgical skills, the opportunity lies for the extension of video-based learning to assessment and further. 6 The utility of videos in learning surgical skills became increasingly apparent during the COVID-19 pandemic. 7 The aim of this study is to compare the effectiveness of video-based teaching interventions to alternative teaching methods, such as text and live workshop demonstrations. Moreover, it aims to evaluate the manner in which implementation of video-based teaching interventions is most effective in the acquisition of surgical skills by healthcare professionals.

Methods
This review was conducted according to PRISMA (Preferred Reporting Items for Systematic Reviews and Metaanalyses) guidelines 8 and registered on "PROSPERO" the international prospective register of systematic reviews prior to the study commencement (Registration ID: CRD42021233836).

Eligibility
Inclusion for research analysed in this systematic review was limited to primary research. All studies with original data relating to surgical skill acquisition through the utilisation of video technology were included. Inclusion criteria for study participants were medical students, surgical trainees, surgeons of all grades and specialities, healthcare students and healthcare professionals. The control group was the cohort receiving alternative methods of training in the acquisition of surgical skills. Papers were included unrestricted by publication date however study inclusion was limited to those which could be obtained in the English language if the mentioned criteria were met. Studies that did not utilise video in the direct demonstration and teaching of a surgical skill or procedural steps were not included. Studies regarding the improvement of surgical skill following video-based feedback, patient education or live streaming the surgical teaching were not included, as this review is assessing the use of video in the initial learning and acquisition of surgical skill in healthcare professionals/students.

Search Methods
3 Databases were searched for this systematic review: MEDLINE (Via PubMed), Embase (via OVID) and Cochrane libraries, Clinicaltrials.gov was also searched for eligible ongoing studies. The search was conducted using the terms: Video Ã AND surgical AND teaching OR skill OR education. The listed databases were searched from inception date to 28/02/21. Effort was undertaken in analysing reference lists from articles meeting the inclusion criteria to identify additional eligible studies.

Study Selection
Study selection was performed by two independent authors and with a third more senior reviewer to settle disagreements during study selection. Three levels of review were completed, firstly title screening and abstract screening to eliminate ineligible articles and lastly a full analysis of the manuscript, disagreements following discussion regarding eligibility were settled by the third reviewer. Articles which met the eligibility criteria also had their reference lists screened for additional eligible articles. Table 1 Data Extraction The extraction of data from the eligible studies was completed and checked by both authors, disagreements following discussion were resolved by the third reviewer. Information extracted from the studies included: study type, type of educational video, participant information, duration of education, method of assessment and outcome data analysed for methodological validity and educational effectiveness using Messicks and McGhagie's frameworks. For data extraction, a standard template was decided on prior to analysis of studies (Table 2).

Data Analysis
Eligible studies were analysed according to Messicks criterion (Table 3) 9,10 testing for validity of the study assessments on five different facets: content, response processes, internal structure, relation to other variables and consequences. Each of these parameters were graded according to the Beckman et al. rating scale ( Table 4). The level of validity in each domain ranged from N, where there was no discussion of the source of validity, to two whereby data strongly supported the source of validity. 11 Validity scoring was performed by two independent authors, a senior author was present to reconcile any disagreements in scoring. Scores were subsequently reviewed by all co-authors prior to completion of this manuscript. The level of effectiveness (LoE) of the educational models in the respective studies were quantified using an adapted version of McGhagie's translational outcomes, to serve video-based educational models as opposed to simulation based educational models. In the present review McGhagie's initial translational outcomes have been extended to five outcomes (Table 5). 12,13 Risk of Bias Study quality was assessed and quantified using the validated medical education research study quality instrument (MERSQI), 14 the MERSQI tool consists of six scoring domains created to assess different aspects of The impact of using the assessment Determining the pass-fail score and considerations for the subject on obtaining a pass or fail Table 4. Rating scale by Beckman et al.

Score Definition
N N = no discussion of source of validity evidence presented 0 0 = discussion of source of validity but no data 1 1 = data weakly supports source of validity or is limited 2 2 = data strongly supports source of validity medical education research. Domains include study design, sampling, type of data, validity of assessment instruments, data analysis, and study outcomes. The maximum score per study is contingent on the study purpose, hence for this systematic review scores were standardised over a common denominator of 18 enabling comparison. Due to the nature of the studies and the heterogeneity of the study designs and outcome measures, qualitative analysis of the articles was performed.

Study Selection
The search produced 2201 results, following duplicate removal 1688 studies remained for screening. The 1688 studies underwent title and abstract screening, 34 remaining studies underwent a full text screening, after a thorough analysis of the 34 articles, a further 12 articles were excluded, 22 articles remained for inclusion ( Figure 1).

Study Characteristics
Of the included studies 20/22 were randomised controlled trials and 2/22 were prospective observational studies. The number of intervention groups also differed between studies. The total number of participants across the included studies within this systematic review was 797.

Features of Video-Based Interventions
Multiple methods were used in the delivery of video-based interventions, the most common simply included 2dimensional video with narrations describing procedural steps which 11/22 (50%) mentioned to have utilised either through audio or on-screen narration. Animation technology was implemented in 2/22 studies (9.1%), 18,24 a further 2/22 (9.1%) also used still images throughout the procedural video to augment certain instructional steps. 31,32

Level of Effectiveness
Of the 22 papers analysed according to McGhagie's translational outcomes, 13 3/22 (13.6%), studies scored an LOE of 1 indicating positive trainee satisfaction relating to the intervention, typically recorded via feedback forms.14/22 (63.6%) papers scored an LOE of 2 suggesting a statistically significant improvement in surgical skill as quantified by a form of examination, either practical or knowledge based. 1/22 (4.5%) of studies scored an LOE of 3, indicating an improvement of performance in the clinical context. Crawshaw et al. 26 assessed the effectiveness of incorporating a comprehensive instructional video prior to the performance of a right laparoscopic colectomy, which improved overall surgical performance and operative independence within surgical residents. Lastly, 4/22 (18.2%) of the included studies scored N/A, this was allocated when no statistically significant positive outcome was reported relating to the administration of the educational intervention. None of the articles included in this study reported a LOE of 4 or 5 whereby patient outcomes were quantifiably affected or changes witnessed on a systemic level. This LOE requires an additional level of analysis, through widescale long term implementation of the educational interventions, which was not conducted in the included studies. Rather studies assessed educational interventions on a more immediate level, such as in trainee satisfaction and improvement of knowledge and skills. The majority of studies within this systematic review assessed outcomes through validated practical examination checklists, completed by blinded expert examiners testing multiple aspects of good surgical practice such as proper motion handling, knowledge of instruments, and flow of operation. 37 Subjective student experience surveys, designed to assess the allocated intervention on factors such as: student satisfaction, clarity of teaching materials and whether the participant deems the educational method effective 38 were also incorporated in 7/22 studies. Of the 7 studies which incorporated student surveys, 6 (85.7%) demonstrated favourable responses towards the video-based teaching intervention.

Video Based Interventions vs. Conventional Methods
Conventional teaching methods were used in 9/22 (40.9%) studies within this systematic review, defined in this context as: live workshops, textbook use and instructor led teaching sessions/lectures. In 7/9 (77.8%) of the studies comparing video-based interventions to conventional methods, a LOE of 1 or 2 was recorded, favouring video-based interventions. Overlying themes which made video-based interventions a superior modality, include the ability of the participant able to control replay speed, pause and advance to focus on certain aspects. 20,21 Further benefits of the use of educational video were in the post recording edits, educational video can be enhanced through the use of narrated diagrams, physical demonstration and the ability to highlight specific aspects augmenting understanding among participants. 28 Video Based Learning vs. Virtual reality Two of the included studies within this systematic review compared typical video-based education to virtual reality, 23,36 Lohre et al. 23 assessed orthopaedic residents in performing a reverse shoulder arthroplasty, results revealed that the immersive virtual reality had a significantly higher mean OSATS scores than the video group (15.9 [2.5] vs 9.4 [3.2]; difference, 6.9; 95% CI, 3.3-9.7; P < .001). Yoganathan et al. 36 demonstrated similar findings, although the study focussed on acquisition of basic surgical skills as opposed to a complete procedure. Scores were significantly superior in the virtual reality (VR) group (median knot score 5.0 vs 4.0 P = .04) than the video-based intervention group, with 25% more of the cohort being able to construct a completed reef knot.

MERSQI Scores
All the included studies above were graded according to the MERSQI tool which has a maximum possible score of 18, although for certain studies adjustments were made as not all assessment fields were applicable and a common denominator of 18 was implemented. The mean MERSQI score across all studies was 11.66 (64.7%) (range = 9.5 -14.5 (52.7% -80.6%), Standard deviation = 1.55 (8.6%)).

Validity of Evaluation Methods
Of the analysed studies within this systematic review, 14/ 22 (63.6%) studies reported with data strongly supporting the validity of the evaluation instrument (Table 4) on at least one of Messicks parameters (Table 3). 9/22 (40.9%) reported grade 2 validity on one parameter, 3/22 (13.6%) studies reported grade 2 validity on at least 2 parameters, and only 2/22 (9.1%) studies reported a grade 2 of validity on 3 domains. Validity was quantified through an analysis of the eligible papers and where specific information relating to the features of an evaluation instrument were not included, more information was obtained regarding the evaluation tool through the relevant paper obtained from reference reviews.

Discussion
This systematic review has demonstrated that video-based teaching methods led to improved surgical skills/trainee satisfaction in 17/22 of the studies. Previous systematic reviews investigating the use of video in surgical education conducted by Green et al. 39 and Ahmet et al. 40 suggest a positive impact, however the eligibility criteria for both studies was limited to medical students and residents, conversely this study analysed research relating to any healthcare profession. The analysis of students from a range of different academic backgrounds, was considered more reflective of educational effectiveness of the video-based methods in surgical skill acquisition, due to the variability in the study cohort.

Effective Implementation of Educational Video
An evident theme from this study was that video-based interventions which utilised virtual reality and simulationbased education proved more effective than standard videos. The immersive nature of virtual reality may render superior visuospatial awareness of technique, studies utilising virtual reality training had outcomes significantly superior to basic teaching video. 23,36 Sakamoto et al. comparing simulation-based teaching interventions for microsurgical skills found video-based interventions to be inferior, improvement in the simulation group may have been due to the opportunity for hands on rehearsal. 27

Video Based Learning vs. Conventional Methods
Several features of video were markedly advantageous to conventional methods, themes identified upon analysis included: the ability to control replay speed and to navigate through videos directly to specific parts of focus, 20,21 the highlighting of different anatomical structures, the imposition of schematic overlays and the use of audio/text narrations. 28 Furthermore, through video editing attention can be directed towards the most important aspects of the teaching via auditory and visual annotations, enabling learners to achieve a superior understanding of taught concepts. Video editing also enables the removal of nonessential information reducing extraneous load, which by the cognitive load theory, would increase learning potential. 41,42 Advantages of Using Videos Video-based teaching enables standardisation of educational materials, enabling not only a consistent foundation for future clinical trials but also consistently high-quality educational content for healthcare students/professionals. An example is the New England Journal of Medicine (NEJM) video series for learning surgical procedures. Videos are designed by multiple experts, they then undergo peer review prior to dissemination. Through the use of this video series, Saun et al. proved the superiority of standardised video-based teaching methods over didactic teaching for chest tube insertion, authors speculating the basis being due to the possible variability in the teaching approaches by different instructors, being overcome by standardised video. 28 With advances in computer technology, trainees have access to high quality digital content at the click of a finger, creating opportunity for improvement of access to surgical education. Traditional learning of surgical techniques relies heavily on workshop-based interventions and the apprenticeship model, 43 however research suggests significant advantages of augmenting surgical skills training with educational video through the ability to widely disseminate surgical video via the internet, study remotely and control aspects of material review. 39 Video is also documented to enhance retention in the learner to a greater degree, a person typically retains 10-15% of that which is read, 10-20% of what is listened to, and 20-30% of what is viewed, however when audio and video materials are used in conjunction the retention of knowledge increases to 40-50%. 44

Disadvantages of Educational Videos
Although the employment of video-based teaching methods poses significant advantages, surgical video to date may not be able to replace traditional interactive teaching methods. Interactive teaching sessions have the additional element of live feedback and the ability to ask questions which solely relying on a teaching video would not provide. 28 Further studies, assessing video-based interventions with additional features such as expert feedback during practicing of surgical skills and questions and answers, may enable a more comprehensive comparison of teaching methods.

Recommendations
Most studies included in this systematic review (17/22) demonstrated improved examination scores and positive trainee perceptions resulting from video-based training. However, whether video-based education had advantages which translated to improved performance in the clinical context was undocumented, except in a study by Crawshaw et al. 26 The extension of studies to assess surgical performance beyond a simulated environment can support the investment into video-based educational resources and curricula for trainees. Conduction of such research could be in retrospective analysis of complications and operative time prior to video-based education and after. Alternatively, patient outcomes in a surgical setting could be investigated through a randomised controlled trial where video-based education is employed.
In addressing low validity scores, the design of checklists and study protocols (with validity criterion accounted for) would have improved external validity of included studies. The advantages of the sources of validity mentioned are evident, an example of this is in comparing a study by Chien et al. 16 which scored high on content validity, with Dantas et al. 17 which showed no evidence of content validity. Chien et al. 16 used a scoring checklist with domains based upon predefined guidelines in published literature accounting for expert opinion, whereas Dantas et al. 17 did not clearly define where test items were derived.
The establishment of a study protocol checklist, which accounts for validity and investigation of similar outcomes, would increase study quality and homogeneity allowing a more rigorous evaluation of video-based surgical education. The literature reports the existence of standardised educational video libraries, for example the NEJM video series. The uniformity in composition video features, if applied across multiple study groups, would further prove the utility of video-based education.

Limitations
Numerous recurring weaknesses were identified within the studies, firstly the low study quality as assessed by the MERSQI tool with the average quality quoted at 64.7%, Secondly, the use of validated evaluation instruments was lacking, none of the 22 studies reported validity on all five sources. Thirdly, none of the studies in this systematic review achieved higher than an LOE of 3, suggesting the lack of long term follow up. It may beneficial if future clinical trials could attempt to assess translational outcomes in terms of clinical practice. Also, multiple studies failed to specify the type of video used, as well as adjunctive features which may have been causative in the witnessed change/lack of change of assessment scores, thereby preventing a holistic analysis of video features which may have contributed to outcomes.
Due to the heterogeneity in study methodology of included papers, quantitative assessment of effectiveness of video-based interventions was not feasible and qualitative analysis of studies was performed. Studies differed drastically on multiple measures such as outcomes assessed limiting comparability. This study limited inclusion to papers which were in the English language.

Conclusion
Video based interventions proved to have a positive effect in (17/22) 77.3% of the studies analysed in this systematic review. With the advent of new video technologies, and the ease of accessibility of video-based teaching, based on the findings of this review, the creation of standardised video programmes for the teaching of surgical skills is likely to enhance current teaching methods. It is evident from this study that video-based education overcomes many limitations in acquiring procedural knowledge and was favoured by trial participants. Although video may not be able to replace the interactive aspect of learning surgical skills, an analysis of the literature proved favourable outcomes in practical and knowledge examinations. Future research in the field should ensure evaluation tools are validated, ensuring reliability of any results, and aim to assess outcomes on a wider scale outside of the immediate study setting.

Author Contributions
Abdullatif Aydin and Samy Cheikh Youssef contributed to the conception and design of this study, Samy Cheikh Youssef and Alexander Canning carried out study selection and assessed data from studies with Abdullatif Aydin as senior reviewer. This article was written with input from Prokar Dasgupta, Abdullatif Aydin, Nawal Khan, Samy Cheikh Youssef and Alexander Canning, and critically appraised by Prokar Dasgupta, Abdullatif Aydin and Nawal Khan. All authors read and approved the final manuscript.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.

Data Availability
All sources of data analysed during this study are cited in the bibliography.