pmc logo image
Logo of nihpaNIHPA bannerabout author manuscriptssubmit a manuscript
Journal List > NIHPA Author Manuscripts

Formats:
Lect Notes Comput Sci. Author manuscript; available in PMC 2008 September 26.
Published in final edited form as:
Lect Notes Comput Sci. 2007; 4522 LNCS: 421–431.
PMCID: PMC2553685
NIHMSID: NIHMS20373
Copyright notice and Disclaimer
Accurate Spatial Neighborhood Relationships for Arbitrarily-shaped Objects using Hamilton-Jacobi GVD
Sumit K. Nath, Kannappan Palaniappan, and Filiz Bunyak
MCVL, Department of Computer Science, University of Missouri-Columbia, MO, USA
* {naths, palaniappank, bunyak}@missouri.edu
Abstract
Many image segmentation approaches rely upon or are enhanced by using spatial relationship information between image regions and their object correspondences. Spatial relationships are usually captured in terms of relative neighborhood graphs such as the Delaunay graph. Neighborhood graphs capture information about which objects are close to each other in the plane or in space but may not capture complete spatial relationships such as containment or holes. Additionally, the typical approach used to compute the Delaunay graph (or its dual, the Voronoi polytopes) is based on using only the point-based (i.e., centroid) representation of each object. This can lead to incorrect spatial neighborhood graphs for sized objects with complex topology, eventually resulting in poor segmentation. This paper proposes a new algorithm for efficiently, and accurately extracting accurate neighborhood graphs in linear time by computing the Hamilton-Jacobi generalized Voronoi diagram (GVD) using the exact Euclidean-distance transform with Laplacian-of-Gaussian, and morphological operators. The algorithm is validated using synthetic, and real biological imagery of epithelial cells.
Spatial neighborhood relationships among objects is an important characteristic in many image analysis, computer vision and robotics applications. One common approach is to compute Delaunay graphs from an ordinary Voronoi diagram (OVD), using information from centroids of objects [1]. In the context of biological image analysis, the OVD has been used for accurate segmentation and analysis of confluent migrating cells [2, 3], tissue architecture characterization [4], or endothelial cell classification [5]. Our application is primarily focused on accurate segmentation and tracking of cells in biomedical video sequences that undergo complex shape changes like mitosis and apoptosis.
An OVD using points is insensitive to object properties like size, shape, orientation or containment. Thus, neighborhood graphs derived from point-based centroid representations of arbitrarily-shaped objects often lead to incorrect neighborhood relationships as shown in Figs. 1(b) and (e)Fig. 1. Applications that depend on accurate spatial neighborhood relationships would consequently fail or lead to unpredictable behavior. For example, incorrect neighborhood relationships may lead to false merges of neighboring cells in the segmentation algorithm described in [2]. In other applications, such as robot path planning, inaccurate neighborhoods obtained from OVD’s may impede the movement of the robot or lead to weak navigation performance [6]. An alternative to the OVD is to compute the generalized Voronoi diagram (GVD) that takes into account the size, shape, orientation, and placement of objects when computing neighborhood relationships. As seen from Figs. 1(c) and 1(f)Fig. 1 the GVD accurately identifies the neighborhoods of complex-shaped objects (e.g., the thin long non-convex worm-like object).
Fig. 1
Fig. 1
Fig. 1
[a and d]: Synthetic images showing arbitrarily-shaped objects. [b and e]: It is evident that an ordinary Voronoi diagram (OVD), computed from centroid points of objects (shown as white squares) leads to incorrect neighborhood relationships. [c and f (more ...)
The GVD in any dimension can be precisely defined using point to object distance measures [1, p 280]. Let A = {A1, A2,…, AN} be a set of arbitrarily shaped objects in a d–dimensional space Rd. Now, for any point p [set membership] Rd, let D(p, Ai) denote a distance measure representing how far the point p is from the object Ai which is typically the minimum distance from p to any point in object Ai. The dominance region (also known as influence-zone) of Ai, is then defined as
equation M1
(1)
A generalized Voronoi boundary, between Ai and Aj, can then be defined as the loci of equidistant points between both objects, equation M2, where
equation M3
(2)
and the corresponding influence zone for Ai, V(Ai), is the set intersection
equation M4
(3)
Hence, the generalized Voronoi diagram of A, GVD(A), is given by the union of of such generalized Voronoi regions, as
equation M5
(4)
When A is a collection of points rather than sized objects, GVD(A) reduces to an ordinary Voronoi diagram, OVD(A). Note that OVD boundaries are always straight lines or hyperplanes, whereas GVD boundaries can be complex curves or surfaces. Fig. 1Fig. 1 shows examples of the OVD and GVD for objects in a plane (i.e., d = 2). For those interested in properties of the OVD for point objects, we direct them to the book by Okabe et al. [1] and the survey paper by Aurenhammer [7].
The GVD representation of a set of objects has a number of useful properties: (i) it is a thin set that partitions a space into connected regions (ii) it is homotopic to the number of objects, (iii) it is invariant under transformations applied to all objects, and (iv) each region of the GVD is guaranteed to contain the entire object.
Sugihara presents an algorithm to construct an approximate GVD by reducing an object to a collection of points [8]. A different class of algorithms to construct GVD’s is based on morphological operators and label propagation. This consists of labeling connected components (objects) in an image, and simultaneously growing them using dilation operators. The loci of points at which these regions stop growing determine the influence zone of each object. In the literature, this algorithm is referred to as skeletons by influence zone (SKIZ) and is described in detail by Vincent [5,9]. Lu and Tan have presented a variation of SKIZ by approximating connected components as polygons and expanding the regions using Freeman codes for document image analysis [10]. Hoff et al. have reported a fast algorithm for GVD construction using graphics hardware [11].
Recently, Siddiqi et al. proposed a new class of algorithms to compute object skeletons using the average outward flux of the gradient of a distance transform [12]. Homotopy preserving properties of this algorithm makes it a strong alternative to other algorithms that use the Euclidean distance transform (EDT) to compute object skeletons. A Hamilton-Jacobi formulation for shock tracking, combined with homotopy preserving thinning leads to a robust and low-complexity implementation. As an original contribution, we propose using the Hamilton-Jacobi formulation to compute GVD’s. The focus of this paper is on efficiently extracting exact neighborhood relationships of arbitrarily shaped objects (e.g., biological cells) using the GVD as the basic underlying framework, based on a fast EDT. It should be noted that even though our algorithm aims at solving a problem in biological image analysis, it can be applied to other applications in computer vision such as robot navigation, remote sensing of urban areas or content-based image retrieval.
The paper is organized as follows. In Sec. 2, we summarize our proposed algorithm and explain its key features. Comparative results of using OVD versus GVD for computing cell neighborhood relationships are shown in Sec. 3, and conclusions in Sec. 4.
The proposed algorithm to compute a neighborhood adjacency graph equation M6 for an image P containing N–arbitrarily shaped objects, using GVD in R2 is shown in Algorithm 1, and described in detail in the following paragraphs.
Algorithm 1
Algorithm 1
Compute a 2D Neighborhood Adj. Graph
In order to compute reliable GVD boundaries touching objects need to be separated by at least a one-pixel gap. In Step 1, labeled pixels are (temporarily) removed from the image if they are adjacent to one or more different labeled pixels, without any gap. In Step 2, we convert the modified multi-labeled mask into a binary image with non-zero pixels representing N distinct connected components.
Siddiqi et al. have reported using a Borgefors distance transform (BDT) in their skeletonization algorithm [12]. However, the BDT is an approximation of the Euclidean distance transform (EDT). Hence, in Step 3, we compute the exact EDT using a “separable algorithm” proposed by Felsenzwalb and Huttenlocher (FH-EDT) that is fast (linear time), and efficient to implement [13].
Let equation M7 be a 1D grid, and equation M8 an arbitrary function on the grid. The one-dimensional FH-EDT of f is defined as
equation M9
(5)
with the added constraint that for each point equation M10, the distance transform of f is bounded by a parabola rooted at (q, f(q)). The distance transform at point p is the height of the lower envelope of all such parabolas [13, Fig. 1Fig. 1]. The FH-EDT algorithm computes the distance transform in O(n) time. The efficiency of this algorithm is evident by considering a two-dimensional grid equation M11, and equation M12 an arbitrary function on the grid. The two-dimensional distance transform of f is given by
equation M13
(6)
where equation M14 is the 1D distance transform of f restricted to the column indexed by x′. Hence, the 2D distance transform can be computed separably in linear time.
In order to detect points of singularities (or shock points), Siddiqi et al. propose to compute the average outward flux at every point in a vector field q (derived from the distance transform) using a Hamilton-Jacobi formulation [12]. Using the divergence theorem, a relationship between the divergence of the vector field div(q), and the average outward flux is given by [12]
equation M15
(7)
where δR is the bounding contour of the region R, [mathematical script N] is the outward normal at each point of the contour, and ds is the element of integration. The divergence div(q) can be equivalently written as the sum of partial derivatives with respect to each of the vector field’s component directions. However, the vector field (i.e., distance field) is differentiable at all points except at singular or shock points. This is the justification provided by Siddiqi et al. for using Eq. 7, and a limit approximation, to locate singularities in q. As an alternative, in Step 4, we propose using a 2D Laplacian-of-Gaussian [nabla]2Gσ [asterisk in circle] D operator on the distance transform, D , in order to detect regions of local maxima (or minima, depending on how the Laplacian operator is applied), i.e., ridge points. The Gaussian operator Gσ smooths the distance transform prior to applying the Laplacian operator insuring differentiability at shock points. Smoothing, however, does not guarantee homotopy preservation of GVD boundary points. Hence, to satisfy both constraints, the regularization parameter σ is set to a small value.
In Step 5, we threshold E = [nabla]2Gσ [asterisk in circle] D ) to obtain the binary image, Ethr,
equation M20
before computing the GVD, A suitable choice of the threshold value, TLD, is critical in homotopy preservation of GVD boundaries. A low threshold value results in larger number of spurious features (such as branches and associated holes), while a larger threshold significantly reduces these features at the cost of breaking real object boundaries. We set TLD = 0 by default.
After binarization of E, the background should normally be segmented into N connected generalized Voronoi regions, corresponding to N input objects. However, when computing the Laplacian of the EDT, regions of local maxima, i.e., ridges, may appear very close to each other and interact to produce “holes” that are small connected background components (shown in Fig. 2(b) and (c)Fig. 2). In our algorithm, each influence zone (i.e., V (Ai)) corresponds to a unique object (Ai) in the image. Hence, in Step 6, such holes are removed using a threshold parameter THS, prior to computing the GVD. Non-removal of such holes prevents further removal of ridges that are attached to such holes, termed as branches. Hole removal is effected by size-constrained connected component analysis. The binarized image, obtained in Step 5 of Algorithm 1 is inverted followed by a connected component analysis. All connected components below a certain size are classified as part of the background which results in “hole-filling”.
Fig. 2
Fig. 2
Fig. 2
Examples of isolated holes and checker-board pattern holes that are formed when a large number of single pixel width ridges appear very close to each other. Non-pruned branches may affect the performance of such applications as robot navigation. A relatively (more ...)
In Step 8, a thinning algorithm (c.f. [15]) is applied to the hole-filled, binarized image in order to reduce ridge boundaries to single pixel thickness. This step is necessary in order to simplify the search for neighborhood adjacency relationships along boundaries. A key component of any thinning algorithm is the preservation of end points. Thus, after thinning, spurious ridges, without holes, remain attached to actual GVD boundaries. We term such ridges as spurs (see Fig. 5(f)Fig. 5 for example). Hence, in Step 9, we remove such spurs by applying a pruning algorithm having the same features as standard thinning algorithms (e.g., [15]) but enforcing the constraint of non-preservation of end points. Let this thinned (and optionally pruned) image be represented as equation M21. After obtaining one-pixel thick GVD boundaries, we invert equation M22 in Step 9 as equation M23. This is followed, in Step 10, by a connected component analysis on P and assigning unique labels to each GVD influence zone, i.e., equation M25, where equation M26 is the ith connected component formed from the corresponding generalized Voronoi influence zone. Finally, in Step 11, a 3×3 window positioned at each boundary pixel (i.e., pixels not part of any connected component) is analyzed, from which a neighborhood relationship map equation M27 is constructed (see Fig. 4Fig. 4 for some examples). To complement the discussion in previous paragraphs, key steps of our algorithm are shown in Fig. 3Fig. 3.
Fig. 5
Fig. 5
Fig. 5
(a) A representative region from the original image (Frame 46). (b) OVD boundaries superimposed with four unique colors. The centroids are represented as small squares. (c) GVD boundaries superimposed with four unique colors. (d) Neighborhood relationships (more ...)
Fig. 4
Fig. 4
Fig. 4
Examples of neighborhood relationships between connected components when centering a 3×3 neighborhood on a boundary pixel (marked with a X). Shaded regions indicate one pixel thick boundaries of connected components in Q. Valid generalized (more ...)
Fig. 3
Fig. 3
Fig. 3
A flow diagram that describes Algorithm 1. (a) A section of the original mask with four unique foreground colors obtained from Nath et al. algorithm [2]. Cells that are touching each other are marked with arrows. (b) A binary image B is obtained (more ...)
The Hamilton-Jacobi GVD algorithm for determining accurate neighborhood graphs was applied to a biomedical application involving cell segmentation and tracking [2]. Time-lapse phase contrast microscopy of epithelial cells moving in a monolayer sheet are imaged at 0.13μm resolution, and appear as a clustering of dark colored nuclei with indistinct boundaries (Fig. 5(a)Fig. 5) [2, 3, 17].
The OVD regions, and associated Delaunay graph based on centroids of cell nuclei in Fig. 5(a)Fig. 5 are shown Figs. 5(b) and (d)Fig. 5, respectively. Edges E1, E2, and E3 show incorrect object adjacency relationships based on OVD regions A, and B. The object colors are based on graph-vertex coloring and used to implement a fast 4-color level set-based cell segmentation algorithm incorporating spatial coupling constraints [2]. The main feature of the 4-color level set algorithm is to assign different colors to neighboring cells, in order to prevent false merges. From Fig. 5(b)Fig. 5 it can be observed from region A that the two green-colored cells are neighbors of each other, yet they are not marked as neighbors when using an OVD. However, in Fig. 5(c) and 5(e)Fig. 5, these cells are correctly classified as neighbors when using our proposed GVD algorithm (the cells have been recolored).
The neighborhood adjacency graphs for the synthetic images shown in Figs. 1(a) and 1(d)Fig. 1 using the OVD and GVD are shown in Tables 1 and 2, respectively. It is clearly evident that the Hamilton-Jacobi GVD algorithm correctly identifies neighbors of objects in both images, while errors are evident when using OVD to compute the spatial adjacencies of objects. For example, the long thing worm-like object, B6, is adjacent to smaller elliptical objects B1, B3, B4, B5, B7, and, B8. It does not overlap any other object and has a worm-like influence zone based on the GVD, as seen in Fig. 1(f)Fig. 1.
Table 1
Table 1
Neighborhood map of Fig. 1(a)Fig. 1
Table 2
Table 2
Neighborhood map of Fig. 1(d)Fig. 1
We compare the robustness of the Hamilton-Jacobi GVD algorithm with a watershed-based, fast implementation of SKIZ [18, pg. 170–173] in MATLAB. Figs. 6 (c) and (d)Fig. 6, and Figs. 6 (e) and (f)Fig. 6 show that the SKIZ-based GVD, and the Hamilton-Jacobi GVD are both relatively insensitive to perturbations in object boundaries as indicated by the arrows. However, Hamilton-Jacobi GVD boundaries are more accurate (same arrows), since the exact EDT is used.
Fig. 6
Fig. 6
Fig. 6
[a]: Subset of objects from Fig. 1(a)Fig. 1 showing objects A4 and A5. [b]: Same objects with perturbed boundaries. [c and d]: SKIZ-based implementation of GVD with 8-connected label propagation. [e and f]: Hamilton-Jacobi GVD.
In this paper, we have presented a novel algorithm for computing Hamilton-Jacobi based GVD’s to build accurate spatial neighborhood adjacency graphs for arbitrarily-shaped objects. Our algorithm extends the Hamilton-Jacobi skeletonization algorithm of Siddiqi et al. [12], and is coupled with morphological-based operators to remove spurious regions from the initial GVD boundaries. A fast Laplacian-of-Gaussian (LoG) filter is used to detect potential GVD boundary locations (i.e., shock points). Useful features of the LoG filter, like the guarantee of closed contours, continuity of ridges, and non-formation of new ridges with an increase in scale (smoothing) makes it appealing for our algorithm. We compare the performance of our Hamilton-Jacobi GVD algorithm, with a previously developed OVD framework for cell segmentation in [2] on real biological, as well as synthetic images. In all instances, we demonstrate the superiority of our GVD algorithm.
As a future work, we would like to present a comparison of our algorithm with other state-of-the-art algorithms described in the literature. Due to the separable nature of the FH-EDT algorithm [13], we can obtain neighborhood relationships between objects in higher dimensions. Hence, we would like to extend our algorithm to Rd, d > 2.
Acknowledgments
This work was supported by a U.S National Institute of Health NIBIB award R33 EB00573.
References
1. Okabe A, Boots B, Sugihara K, Chiu SN. Spatial Tesselations: Concepts and Applications of Voronoi Diagrams. 2. John Wiley & Sons Ltd; West Sussex, UK: 2000.
2. Nath S, Palaniappan K, Bunyak F. Cell segmentation using coupled level sets and graph-vertex coloring. In: Larsen R, Nielsen M, Sporring J, editors. LNCS - Proc MICCAI 2006. Vol. 4190. Springer-Verlag; 2006. pp. 101–108.
3. Nath S, Bunyak F, Palaniappan K. Robust tracking of migrating cells using four-color level set segmentation. In: Blanc-Talon J, Popescu D, Philips W, Scheunders P, editors. LNCS -Proc ACIVS. 4179-0920. Springer-Verlag; 2006. 2006. pp. 920–932.
4. Keenan S, Diamond J, McCluggage W, Bharucha H, Thompson D, Bartels P, Hamilton P. An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (CIN). J Pathol. 2000;192:351–362. [PubMed]
5. Vincent L, Masters B. Proc SPIE - Image Algebra Morph Image Proc III. Vol. 1769. San Diego, CA: 1992. Morphological image processing and network analysis of cornea endothelial cell images; pp. 212–226.
6. Choset H, Walker S, Ard K, Burdick J. Sensor-based exploration: Incremental construction of the hierarchical generalized voronoi graph. I. Journ Robotics Res. 2000;19:126–148.
7. Aurenhammer F. Voronoi diagrams - A survey of a fundamental geometric data structure. ACM Comp Surveys. 1991;23:345–405.
8. Sugihara K. Approximation of generalized Voronoi diagrams by ordinary Voronoi diagrams. Com Vis Graph Image Process. 1993;55:522–531.
9. Vincent L. Graphs and mathematical morphology. Sig Proc. 1989;16:365–388.
10. Lu Y, Tan C. Constructing area Voronoi diagram in document images. IEEE Comp Soc; Proc. 8th IEEE Int. Conf. Doc. Anal. Recog.; 2005. pp. 342–346.
11. Hoff K, III, Keyser J, Lin M, Manocha D, Culver T. Fast computation of generalized voronoi diagrams using graphics hardware. SIGGRAPH-99: Proc. 26th Ann. Conf. Comp. Graphics Inter. Tech; ACM Press/Addison-Wesley Publications. 1999. pp. 277–286.
12. Siddiqi K, Bouix S, Tannenbaum A, Zucker S. Hamilton-Jacobi skeletons. Int J Comput Vis. 2002;48:215–231.
13. Felzenswalb P, Huttenlocher D. Technical Report TR2004-1963. Dept. of Comp. Sci., Cornell University; Ithaca, NY: 2004. Distance transforms of sampled functions.
14. Rosenfeld A. A characterization of parallel thinning algorithms. Inform Control. 1975;29:286–291.
15. Cychosz J. Graphics gems IV. Academic Press Professional, Inc; San Diego, CA, USA: 1994. Efficient binary image thinning using neighborhood maps; pp. 465–473.
16. Lam L, Lee S, Suen CY. Thinning methodologies-A comprehensive survey. IEEE Trans Patt Anal Machine Intel. 1992;14:869–885.
17. Bunyak F, Palaniappan K, Nath SK, Baskin TI, Dong G. Proc 3rd IEEE Int Symp Biomed Imaging (ISBI). IEEE Comp. Soc; Arlington, VA: 2006. Quantitive cell motility for in vitro wound healing using level set-based active contour tracking; pp. 1040–1043.
18. Soille P. Morphological Image Analysis. 2. Springer; New York, USA: 2004.

PubMed articles by these authors

Links
See more articles cited in this paragraph