![]() | ![]() |
Formats:
|
||||||||||||
Application of Haptic Feedback to Robotic Surgery 1Division of Cardiac Surgery, The Johns Hopkins and 2Department of Mechanical Engineering, Johns Hopkins University, Baltimore, Maryland. Address reprint requests to: David D. Yuh, 600 North Wolfe Street, Blalock 618, Baltimore, MD 21287E-mail:dyuh/at/csurg.jhmi.jhu.edu The publisher's final edited version of this article is available at J Laparoendosc Adv Surg Tech A. See other articles in PMC that cite the published article.Abstract Robotic surgical systems have greatly contributed to the advancement of minimally invasive endoscopic surgery. However, current robotic systems do not provide tactile or haptic feedback to the operating surgeon. Under certain circumstances, particularly with the manipulation of delicate tissues and suture materials, this may prove to be a significant irritation. We hypothesize that haptic feedback, in the form of sensory substitution, facilitates the performance of surgical knot tying. This preliminary study describes evidence that visual sensory substitution permits the surgeon to apply more consistent, precise, and greater tensions to fine suture materials without breakage during robot-assisted knot tying. INTRODUCTION Technological advancements in computer-assisted surgical systems and increasing health care demands have expanded the field of minimally invasive surgery to new specialties. Robotic surgical systems have several advantages over conventional endoscopic surgery, including three-dimensional vision, increased range of motion, tremor filtration, and motion scaling, permitting surgical tasks to be performed in confined spaces.1,2 Furthermore, patients and the health care industry are aggressively pursuing procedures with smaller incisions, less pain, and shorter recovery times.3 Computer-assisted robotic surgical systems have made total endoscopic cardiac surgery a reality. However, current robotic surgical systems are limited by the lack of tactile or haptic feedback critical to performing complex, delicate surgical tasks.4 Clinical successes with robot-assisted minimally invasive cardiac surgery have lagged behind those achieved with robot-assisted laparoscopic general surgery due, in large part, to the lack of feedback. This deficiency with current robotic systems is a significant handicap in performing the technically more intricate and delicate surgical tasks inherent in cardiac surgery. For example, suturing a coronary arterial anastomosis with fine polypropylene suture is a highly dexterous task, in which the surgeon typically uses his or her sense of touch to puncture tissue with a fine needle, pull the suture through, and tie and tighten knots. In our own observations of experienced and talented cardiac surgeons training with the da Vinci surgical system, fine polypropylene sutures are often broken, and delicate tissues torn, due to the application of excessive forces conventionally attenuated with haptic feedback. The consequences of such surgical errors or delays in cardiac surgery (e.g., coronary microvascular or great vessel trauma, prolonged cardiopulmonary bypass) present much greater potentials for irreversible injury, excessive hemorrhage, or even death for the patient. “Sensory substitution” is a means by which applied forces are conveyed to the operating surgeon with visual or auditory representations. In this preliminary study, we assess the feasibility and potential benefits of sensory substitution in providing haptic feedback in the context of robotic-assisted knot tying. METHODS Subjects The human subjects of this study included surgeons within the Division of Cardiac Surgery at the Johns Hopkins Hospital; the study intended to provide basic guidelines on the level and resolution (temporal and magnitude) of force-sensing required for knot-tying. Da Vinci surgical robotic system The da Vinci computer-enhanced instrumentation system (Intuitive Surgical Inc., Mountain View, CA) consists of three major components, an input device (surgeon’s console), a digital interface, and an output device (manipulator). The console houses the display system, the input handles, the digital interface, and the electronic controller. The tool handles are serial link manipulators that serve both as high-resolution input devices reading thepposition, orientation, and grip commands from the surgeon, and as haptic displays. The image of the surgical site is transmitted to the surgeon through a high-resolution stereo display. The system projects the image of the surgical site atop the surgeon’s hands (via mirrored overlay optics), while the controller transforms the spatial motion of the tools into the camera frame of reference. In this arrangement, the system restores hand-eye coordination and provides a natural scaled-down correspondence in motions. The user interface permits the surgeon to control camera positioning while keeping the slave instrument tips in the operator’s view, to reposition the masters in their work space, and to focus the endoscopic camera. Orientational alignment is always provided, and positional alignment can be adjusted to permit repositioning of the master handles independent of the instrument tips. Motion scaling and tremor filtering by the digital interface further augment precision. The electronic controller is capable of fully interconnected controls of 48 degrees of freedom at update rates over 1000 cycles per second. A second subsystem consists of a side cart, made up of fully sterilizable endoscopic instruments, the instrument and camera manipulators, the endoscope, and the assistant’s user interface. The end-effectors are fully sterilizable instruments that attach interchangeably to the two manipulators (automated instrument recognition) and provide a total of 6 degrees of freedom (plus tool function) intracorporeally. Three manipulators drive the two endoscopic instruments and camera. In turn, these manipulators are positioned around the body by three passive multiple-link arms mounted to a fixed base. As an emergency fail-safe, the arms can be removed from the patient in seconds. Characterization of applied suture forces In conjunction with the Johns Hopkins University Department of Mechanical Engineering, a tension measuring device (TMD) was constructed, allowing force-sensing on both the left and right hands (Fig. 1
Sensory substitution trials Five surgeons first performed hand-ties using various suture materials commonly used in cardiac operations: 2–0 Ti-Cron, 5–0, 6–0, and 7–0 polypropylene (US Surgical Corporation, Norwalk, CT). Each surgeon performed five ties with each suture and the mean tensions were recorded. A sensory substitution mechanism, in the form of a visual color bar scale, was placed in-line with the TMD to relay the mean tension applied to the sutures to the operating surgeon during the knot-tying task. The visual color bar scale was calibrated using the mean suture tensions obtained during manually tied knots. The color bar progresses from yellow to green to red as tension increases (Fig. 2
The five surgeons then performed robotic-assisted knot tying as the control technique, followed by performing the same tying procedure with the aid of the color bar sensory substitution. Each of the surgeons performed five surgical knots with each type of suture with and without sensory feedback. The surgeons were instructed to either increase or decrease the amount of tension applied to the suture to maintain the color bar in the green zone. The amount of tension applied to the sutures was determined using the tension-measuring device. Statistical analysis The paired Student’s t-test was used to compare the suture tensions between robot-assisted knot tying with and without haptic sensory substitution. RESULTS The mean tensions applied to the various suture materials with and without haptic/sensory feedback during robotic assisted knot tying were measured. In the Ti-Cron group, the mean tension applied to the suture was lower with the aid of the color bar sensory substitution. Further, the amount of applied tension to the suture was significantly more consistent with haptic feedback as evidenced by the narrower standard deviation (Table 2).
During robotic-assisted knot tying, mean tensions applied to fine polypropylene sutures were greater with each type of suture when haptic feedback was applied. As the sutures became progressively smaller, increased tensions applied to the suture neared statistical significance in the 6–0 polypropylene group (P = 053) and achieved statistical significance in the 7–0 polypropylene group (P < 0.001). As with the Ti-Cron group, the standard deviations were narrower when knots were tied with the aid of sensory/haptic feedback in each polypropylene group. DISCUSSION Traditionally, cardiac surgery is performed through a median sternotomy, providing optimal access to all cardiac structures and great vessels. The disfigurement and pain associated with this extensive incision has been a longstanding but heretofore acceptable consequence of cardiac surgery. Fueled largely by the advent of less invasive and traumatic laparoscopic and thoracoscopic techniques for noncardiac operations and the rise of catheter-based interventions for myocardial revascularization, minimally invasive approaches to cardiac surgery are being developed. Computer-augmented, robot-assisted surgery has enhanced the ability of surgeons to perform minimally invasive procedures by placing a digital interface between the surgeon and the instrument tips. The digital interface enhances the surgeon’s ability to perform minimally invasive cardiac surgery in several ways. First, by filtering high-frequency signals, surgical tremor is eliminated. Second, the digital interface allows for motion scaling in which easy-to-perform macroscopic movements at the surgeon console are scaled down to a microscopic scale inside the patient, enhancing dexterity. Finally, the computer interface permits the accurate translation of the surgeon’s hand motions to an endoscopic wrist placed within the chest cavity, conferring much higher degrees of freedom than traditional manually actuated endoscopic instruments. The da Vinci system has been used successfully to perform endoscopic coronary artery bypass grafting5,6 and mitral valve repairs.7 These early operations have been performed in carefully selected patients by a select few surgeons who were intimately involved with the development of these systems in the laboratory and the operating room. Despite these early procedural successes, improvements and adjunct technologies are needed before widespread use of the current systems can occur.8 For instance, the lack of tactile feedback has led to prolonged operative times and difficulty in performing some forcesensitive surgical tasks required in cardiac operations.4 Experienced robotic and endoscopic surgeons use visual cues (e.g., tissue deformation) to approximate how much force is being applied: however, this type of sensory substitution is applicable only if the surgeon has a good internal model of tissue consistency to correlate applied forces and tissue deflection. Ideally, haptic feedback in robotic surgical systems would be relayed directly to the surgeon’s control actuators, rendering a true tactile feedback. This capability, however, will require the application of force-sensors on the endoscopic instrumentation. Our laboratories are actively pursuing this goal. As an interim alternative, we have employed sensory substitution to display haptic information via a visual display. Previous work in using audio feedback of force levels has been a success in microsurgery.9 However, additional noise, added to the sounds already associated with cardiac operations (e.g., cardiopulmonary bypass pump, hemodynamic/pulse oximetery monitors), may prove undesirable. A visual force-feedback display in the operative field display may prove to be more ergonomic and have already found application. Image-guided virtual environments in the form of advanced video game systems and surgical simulators utilize surface image deformation, sometimes in conjunction with haptic instrumentation, to provide realtime simulations of operative procedures to include spine surgery, cholecystectomy, and hysteroscopy, and colonoscopy.10,11 Future developments in this area includes visco-elastic modeling of the surface image deformations as well as higher fidelity force computations and system validation. In these virtual simulators, the image itself serves as a sensory substitution modality. The impetus of these early experiments was to apply the virtual environmental concept of visual force feedback to the real world surgical task of suture knot tying. Our results demonstrate significantly greater and more consistent tensions applied to suture materials, without breakage, during robotic knot tying enhanced with haptic feedback compared to knots tied without feedback. During the execution of these experiments, several interesting observations were noted. Surgeons broke the finer polypropylene sutures on several occasions both with and without sensory substitution. Interestingly, surgeons were able to partially compensate for the lack of haptic feedback by visually observing the deformation of tissue when the knot was pulled away from the tissue. We also noted that different users employed different tying techniques; some surgeons used equal force with both hands, while others favored increased control with one hand. In this preliminary study, we have shown that haptic feedback, in the form of a visual color bar scale, permits surgeons to be more consistent, precise, and apply significantly greater tensions to fine sutures without breakage during robotic knot tying. Our visual sensory substitution aid represents an initial step towards providing surgeons with tactile feedback during robotic surgery. This early work with computer-enhanced robotic-assisted cardiac surgery represents the field in its infancy. The associated costs of the robotic systems are indeed high, the learning curves extremely steep, and the applications limited in the context of cardiac surgery. Nevertheless the substantial potential benefits in terms of the already proven clinical advantages of minimally invasive surgery (e.g., shorter hospital stays, less incision pain, fewer wound complications) warrant the continued safe development of this technology. Clearly, the ability to confer haptic feedback to present surgical robotic systems would contribute significantly to the safe performance of cardiac surgical procedures with these complex systems. Acknowledgments This research was supported by grants from the National Institutes of Health (R01 EB002004) and the Whitaker Foundation (RG-02-911). References 1. Czibik G, D’Ancona G, Donias HW, Karamanóukian HL. Robotic cardiac surgery: Present and future applications. J Cardiothorac Vasc Anesth. 2002;16:502–507. [PubMed] 2. Schneeberger E, Michler R. An overview of the Intuitive system: The surgeon’s perspective. Op Tech Thorac Cardiovasc Surg. 2001;6:170–176. 3. Reger T, Jahnke M. Robotic cardiac surgery. AORN J. 2003;1:182–186. [PubMed] 4. Shennib H, Bastawisy A, Mack MJ, Moll FH. Computer-assisted telemanipulation: an enabling technology for endoscopic coronary artery bypass. Ann Thorac Surg. 1998;66:1060–1063. [PubMed] 5. Kappert U, Cichon R, Schneider J, et al. Closed-chest coronary artery surgery on the beating heart with the use of a robotic system. J Thorac Cardiovasc Surg. 2000;120:809–811. [PubMed] 6. Mohr FW, Falk V, Diegeler A, et al. Computer-enhanced robotic cardiac surgery: Experience in 149 patients. J Thorac Cardiovasc Surg. 2001;121:842–853. [PubMed] 7. Chitwood WR, Nabong LW, Elbeery JE, et al. Robotic mitral valve repair: Trapezoidal resection and prosthetic annuloplasty with the da Vinci surgical system. J Thorac Cardiovasc Surg. 2000;120:1171–1172. [PubMed] 8. Awad H, Wolf R, Gravlee G. The future of robotic cardiac surgery. J Cardiothorac Vasc Anesth. 2002;16:395–396. [PubMed] 9. Okamura A, Smaby N, Cutkosky M. An overview of dexterous manipulation. ICRA. 2000;1:255–262. 10. Downes M, Cavusoglu M, Gantert W, et al. Virtual environments for training critical skills in laparoscopic surgery. Proceedings of Medicine Meets Virtual Reality Conference (MMVR)1998, IOS Press, pp. 316–322. 11. Rolfsson G, Nordgren A, Bindzau S, et al. Training and assessment of laparoscopic skills using a haptic simulator. Proceedings of Medicine Meets Virtual Reality Conference (MMVR) 2002, pp. 409–411. |
PubMed related articles
Your browsing activity is empty. Activity recording is turned off. |
|||||||||||
J Cardiothorac Vasc Anesth. 2002 Aug; 16(4):502-7.
[J Cardiothorac Vasc Anesth. 2002]AORN J. 2003 Jan; 77(1):182-6.
[AORN J. 2003]Ann Thorac Surg. 1998 Sep; 66(3):1060-3.
[Ann Thorac Surg. 1998]J Thorac Cardiovasc Surg. 2000 Oct; 120(4):809-11.
[J Thorac Cardiovasc Surg. 2000]J Thorac Cardiovasc Surg. 2001 May; 121(5):842-53.
[J Thorac Cardiovasc Surg. 2001]J Thorac Cardiovasc Surg. 2000 Dec; 120(6):1171-2.
[J Thorac Cardiovasc Surg. 2000]J Cardiothorac Vasc Anesth. 2002 Aug; 16(4):395-6.
[J Cardiothorac Vasc Anesth. 2002]Ann Thorac Surg. 1998 Sep; 66(3):1060-3.
[Ann Thorac Surg. 1998]