Current route-planning practice. The physician interactively scrolls through a high-resolution 3D MDCT image, which consists of a contiguous stack of 2D sections, to mentally determine a complex 3D airway route that reaches a diagnostically important ROI. Later, during live bronchoscopy, the physician draws upon this mental impression to navigate the bronchoscope through the airways. Even though the physician typically has the 3D MDCT scan available for perusal in the bronchoscopy laboratory during the procedure, his/her accuracy at reaching peripheral ROIs is only on the order of 40% [8].

Example of the impact of Airway-Tree Editing for case 20349.3.24 (Figures 8 and 9). Part (a) shows a resulting endoluminal rendering derived from the raw segmented tree I_S, while part (b) shows a similar result after a small peripheral generation-12 child branch is added via Airway-Tree Editing. The blue line is the preplanned route leading to the ROI rendered transparently in red. The added branch is necessary, as verified by the video for this case recorded during the live guided bronchoscopy — note the small airway in the upper-left portions of the views for parts (b-c).

Examples of endoluminal surfaces produced by the proposed surface-definition method and the previous method relying on fixed-HU values. The selected bifurcation is for a peripheral airway near the medial segment of the right upper lobe. A Siemens Sensation 40 scanner generated the MDCT image (image size: 512 × 512 × 557; voxel resolution: Δx = Δy = Δz = 0.50 mm; patient 20349.3.26). (a) Raw local oblique MDCT cross section in the vicinity of the bifurcation and a corresponding bronchoscopic video frame depicting the bifurcation. (b) Surfaces (red) produced by the proposed method overlaid on the cross-sectional image and an endoluminal rendering corresponding to the pose of the real bronchoscopic video of (a). Parts (c-f) depict corresponding results using the fixed-HU method for various HU values. The isosurfaces can be simultaneously too conservatively and too incompletely defined. For all thresholds, the surfaces are too narrow — the bifurcation that is apparent in the video frame (a) is not completely visible in any of the endoluminal renderings (c-f). With the exception of the -850HU isosurface, all isosurfaces also contain noticeable holes (the black regions appearing in the endoluminal renderings).

An endoluminal view produced near the end of a route. This view indicates a few of the extra attributes provided by our proposed planning system. The blue line indicates the preplanned route, while a 4-mm blue arrow points toward the center of the green extraluminal ROI. Even though ROI is located outside the airway, the transparent rendering and arrow allow the physician to unambiguously locate the lesion.

Video frames and the corresponding virtual bronchoscopic views along the remainder of the route depicted in Figure 8.

Another comparison a bronchoscopic video frame (a) with endoluminal renderings (b) and (c). This example is from patient 20349.3.24. The rendering (b) was generated after running through the methods incorporated into the proposed system — the rendering is visually accurate. Part (c) depicts an unacceptable rendering, produced by a previously-proposed surface definition method at the same location as parts (a-b) [10,11,41].

Illustrative outputs from the procedure planning system. A Siemens Sensation 40 scanner generated the input 3D MDCT image used for this example (image size: 512 × 512 × 796; voxel resolution: Δx = Δy = 0.86 mm, Δz = 0.50 mm; patient 20349.25). (a) Step 2 (ROI Definition): the use of live wire to define a previously identified lesion on a 2D slice of input 3D MDCT scan I. (b) Step 3 (Airway-Tree Segmentation): segmented airway tree I_S (green) and the fully-defined peripheral lesion (red). (c) Step 5 (Centerline Analysis): Extracted centerlines (red) appear with the airway tree and ROI. (d) Step 6 (Route Planning): Blue path indicates the selected route for the previously defined ROI. (e) Step 7 (Airway-Tree Editing): the technician identifies a weakly-defined 8^th generation airway, indicated by arrows in the local 2D cross-sectional slice on the left and the surface view on the right.

Idealized gray-scale profiles of a thick-walled airway (a) and thin-walled airway (b). The dotted line represents the underlying idealized HU profile of the airway-wall interface, with small values corresponding to air and large values corresponding to soft tissue. The solid line represents the actual observed blurred HU profile in I. The local maximum of the intensity gradient is marked by a dot, while the location of the average between air and soft-tissue HU values is marked by a short horizontal line. For thick-walled airway (a), both the average-value isosurface and the maximum of the intensity gradient properly localize the surface interface separating air and wall. For the thin-walled airway (b), however, neither quantity accurately localizes this surface. The average-HU isosurface value is never reached, which causes holes in an endoluminal rendering of this surface. Conversely, the maximum of the intensity gradient appears well within the airway, which results in an overly constricted endoluminal rendering.

Various endoluminal surface representations produced for a location containing parallel peripheral branches at generations 6, 7, 8, and 9 in the right middle lobe medial segment. MDCT chest scan generated by a Siemens Sensation 16 scanner (image size: 512×512×544; voxel resolution: Δx = Δy = 0.68 mm, Δz = 0.50 mm; patient 20349.3.7): Part (a) shows the raw oblique cross-sectional data. Part (b) illustrates a pair of airways that are separated by a large topological distance, indicated by the dashed line, but a small Euclidean distance, shown by the solid line. Part (c) shows the same data with the surfaces (red) generated by an isotropic dilation of the airway-tree segmentation by 1.0 mm; the two airways singled out in part (b) have nearly merged together. Part (d) shows the surfaces produced by the proposed method — the two airways are now distinct. Part (e) shows an oblique cross section of the final likelihood image I_L.

A bronchoscopic route to a peripheral ROI in patient 20349.3.28. The Route Overview shows a global view of the airway tree, centerlines (red), route (blue) and ROI (green, left upper lobe). The views to the right show video frames taken along the route to the ROI (left) and registered endoluminal renderings of airway surfaces derived by the method of Section 2.3 (right). The endoluminal views are taken along the route at generations 1, the trachea, and 2, the left main bronchus.

A comparison a bronchoscopic video frame (a) with endoluminal renderings (b) and (c) of patient 20349.3.28 (also used for Figures 8 and 9). The visually-accurate rendering (b) was generated after running through the proposed method of Section 2.3. Part (c) depicts an unacceptable rendering, produced by a previously-proposed surface definition method at the same location as parts (a-b) [10,11,41].

A screen capture of the guidance system during a live procedure (case 20349.3.28). The left portion shows a global view consisting of an extraluminal rendering of the airway tree, centerlines, and three ROIs (red, green, and blue). The blue centerline depicts the computed route for the green ROI. The yellow cylinder and green protruding needle represent the tip of the virtual bronchoscope at the current viewing site of the procedure. The right portion of the screen capture illustrates endoluminal video (left views) and renderings (right views). The top-left endoluminal rendering represents the current viewing site of the system, as indicated by the virtual bronchoscope, with the blue line indicating the route to follow to the ROI. The top-right view shows the live bronchoscopic video feed. The lower left (endoluminal rendering with blue route) and lower right (bronchoscopic video) views represent a “frozen look” of the previously encountered bifurcation passed while the physician maneuvered the bronchoscope along the route. The physician follows the blue line to reach the ROI. Currently, the physician is 49.8 mm from the ROI, as indicated in the bottom right footer. When the physician reaches the vicinity of the ROI, a view similar to that shown in Figure 7 appears.