|
|
Lect Notes Comput Sci. Author manuscript; available in PMC 2008 June 10. Published in final edited form as: Lect Notes Comput Sci. 2005; 3804: 445–453. doi: 10.1007/11595755_54. | PMCID: PMC2423937 NIHMSID: NIHMS20368 |
Adaptive Robust Structure Tensors for Orientation Estimation and Image Segmentation Sumit K. Nath and Kannappan Palaniappan MCVL, Department of Computer Science, University of Missouri-Columbia, Columbia MO 65211, USA Recently, Van Den Boomgaard and Van De Weijer have presented an algorithm for texture analysis using robust tensor-based estimation of orientation. Structure tensors are a useful tool for reliably estimating oriented structures within a neighborhood and in the presence of noise. In this paper, we extend their work by using the Geman-McClure robust error function and, developing a novel iterative scheme that adaptively and simultaneously, changes the size, orientation and weighting of the neighborhood used to estimate the local structure tensor. The iterative neighborhood adaptation is initialized using the total least-squares solution for the gradient using a relatively large isotropic neighborhood. Combining our novel region adaptation algorithm, with a robust tensor formulation leads to better localization of low-level edge and junction image structures in the presence of noise. Preliminary results, using synthetic and biological images are presented. Structure tensors have been widely used for local structure estimation [ 1, 2, 3], optic-flow estimation [ 4, 5] and non-rigid motion estimation [ 6]. Robust statistical estimators have been shown to provide better results when compared with traditional least-squares based approaches [ 7]. In our work on motion analysis using biological image sequences [ 6, 8], we have reported the advantages of using structure tensors for segmentation. This is due to the fact that smoothing is minimized in the direction of the orientation vector, resulting in features that are less blurred at object discontinuities. Combining robust estimators with structure tensor-based orientation estimation is a recent development that holds promising potential to improve localization accuracy in the presence of noise. Boomgaard and Weijer [ 2] apply robust tensor-based estimation for texture analysis and boundary detection while demonstrating the limitations of a total least-squares based approach. Robust estimators are computationally more expensive than their least-squares counterparts. An iterative approach is required to solve a robust structure-tensor matrix, as it becomes non-linearly dependent on the orientation of the patch [ 2]. However, robust structure tensors significantly improve orientation estimates. Instead of using a fixed local neighborhood, an adaptive area for integration has been shown to be beneficial for optic-flow estimation [ 5]. A spatially varying Gaussian kernel that adjusts to local image structure, size and shape is presented in this paper. We also show how this kernel can be efficiently embedded in the fixed-point iteration scheme proposed by Boomgaard and Weijer [ 2]. In addition, we also investigate the use of the Geman-McClure robust error function which is experimentally shown to yield improvements in localization of low-level image structures. The paper is organized as follows. In Section 2, we discuss mathematical concepts associated with 2D structure tensor estimation. Section 3 describes our proposed adaptive spatially varying Gaussian kernel algorithm. Section 4 presents some results and discussion when using our algorithm on synthetic and biological images. A conclusion is provided in Section 5. 2 2D Structure Tensor Based Orientation Estimation Let v(x) be the true gradient of an image patch Ω(y), centered at x. The norm of the error vector between the estimated gradient g(y) at location y and v(x) is given by e(x, y) as This can also be seen in . For clarity, we omit the positional arguments in some instances. In order to estimate v, we will minimize an error functional ρ(||e(x, y)||2) integrated over the image patch Ω, subject to the condition that ||v|| = 1 and ||g|| = 1 (as these are direction vectors). The least-squares error functional is ρ(||e(x, y)||) = ||e(x, y)||2 and the error over the image patch eLS can be written as, On simplifying this expression, we obtain Here, W (x, y) is a spatially invariant weighting function (e.g., Gaussian) that emphasizes the gradient at the central pixel within a small neighborhood, when evaluating the structure tensor. Minimizing eLS with respect to v, subject to the condition that || v|| = 1, is equivalent to maximizing the second term of Eq. 3. Using Lagrange multipliers, we can write this criterion as Differentiating εLS ( x, y) to find the extremum leads to the standard eigenvalue problem for solving for the best estimate of v, given by  . For clarity, we replace with v in the remaining part of the paper. is the least-squares structure tensor at position x using the weighting kernel W. The maximum eigenvector solution of Eq. 5 gives the least-squares estimate for the gradient at pixel x using the surrounding gradient information. Although v![[perpendicular]](/corehtml/pmc/pmcents/x22A5.gif) ( x) could be determined using the minimum eigenvector, it should be noted that for an ideal edge, the smaller eigenvalue will be zero. Hence it is numerically more reliable to estimate the maximum eigenvector. Unlike the least-squares (or quadratic) error measure, robust error measures are noise tolerant by imposing smaller penalties on outliers [ 7]. In this paper, we use the Geman-McClure robust error function [ 7], instead of the Gaussian robust error function used in [ 2]. The Geman-McClure robust error function is defined as, where, m is a parameter that determines the amount of penalty imposed on large errors. The Gaussian robust error function is a special case of the Leclerc robust error function [7, p. 87, Fig. 29], with η2 = 2. shows that both robust error measures ‘clamp’ the influence of large outliers to a maximum of one, whereas the quadratic measure is unbounded. The Geman-McClure function clamps the error norm more gradually, when compared with the Leclerc function. Moreover, we experimentally obtained improved results when using the Geman-McClure function than with the Leclerc function. Using Eq. 6, the error function to be minimized can be written as Minimization of εGM, subject to the constraint that || v|| = 1, is equivalent to maximizing the second term of Eq. 7 within the region Ω. Using Lagrange multipliers, this can be written as follows, Differentiating εGM (x, y), with respect to v, and setting it to zero gives is the Geman-McClure robust structure tensor. The following iterative equation, is a fixed-point functional iteration scheme for numerically solving (λ, v) in Eq. 9 that usually converges to a local minimum [ 2]. Several convergence criterion can be used. Some of them include || vi+1 − vi|| < ε, Tr( J( x, vi, W) < ktrace (a trace threshold), and the size of W (for which we refer the reader to the next section). The total least-squares solution is used to initialize the iterative process in Eq. 11. 3 Spatially Varying Gaussian Kernel Adaptation The structure tensor estimates in the neighborhood Ω can be weighted to increase the influence of gradients close to the central pixel and less influence from the surrounding region. A soft Gaussian convolution function was used in [ 2]. In this work, we propose a spatially varying kernel, W ( x, y), that is a Gaussian function with adaptive size and orientation within Ω. The neighborhood Ω is initialized as a circular region and subsequently adapted to be an oriented elliptical region. Spatially varying adaptation of the kernel (local neighborhood shape and coefficients) is beneficial for improving the estimation of oriented image structures. When computing the structure tensor at a pixel located on an edge, it would be beneficial to accumulate local gradient information along a thin and parallel region to the edge. At the same time, influence of local gradients parallel to the gradient at the pixel should be minimized. Such a strategy would lead to an improved estimate of the gradient. A neighborhood where two or more edges meet is referred to as corners. For localizing such regions, it would be beneficial to select a region that is very small. The proposed adaptive structure tensor algorithm describes the approach by which appropriate small regions can be derived. shows the adaptive algorithm at an ideal edge. In this figure, the dashed-line elliptical region is oriented along the gradient while the solid-line elliptical region (that is scaled and rotated by 90°) is oriented along the edge. The spatially varying kernel Wi ( x, y) that is used with Eq. 10 is defined as where K is a scaling factor associated with the Gaussian function. We initialize the kernel W0( x, y) as an isotropic Gaussian with
 . A fairly large number is chosen (typically R 0 = 8), in order to reduce the influence of noise when evaluating the structure tensor. The columns of Ui are the eigenvectors
 , with the columns of U−1 initialized as the co-ordinate axes. Let
 and
 (with
 ) be the eigenvalues of the structure tensor J( x, vi–1, Wi) at the ith iteration. Scaled versions of these eigenvalues are used to update the semi-major and semi-minor axes for the ( i + 1) th iteration as The eigenvectors obtained from the current iteration, along with
 are used to update the kernel as follows This kernel is used to compute a new structure tensor J( x, vi, Wi+1) as per Eq. 10. To account for the spatially varying Gaussian kernel, Eq. 11 is modified to the following form We experimentally determined that two or three iterations were sufficient to achieve the convergence criteria presented in the previous section. We demonstrate the performance of our algorithm using synthetic and biological images. Edges and junctions at which two or more edges meet (i.e., corners) are typical low-level image structures. depicts a synthetic image having different types of edges (i.e., horizontal, vertical and slanted) and corners. shows the least-squares estimate for the structure tensor, using a circular region for W (x, y). Smeared edges and corners, that result from this process, are shown in the intensity maps of confidence measures (. The proposed (spatially varying) adaptive robust tensor method produces better localization of edges and corners, as shown in . Along ideal edges, one of the eigenvalues is nearly equal to zero. Consequently, there would be no adaptation in the size of the kernel ( Eq. 12). Thus, the improved localization of edges in is due to the robust component of our algorithm. With noisy edges, however, both eigenvalues will be non-zero. Hence, both kernel adaptation and robust estimation contribute to the improved localization of noisy edges as shown in . At junctions, both eigenvalues are greater than zero and can be nearly equal to each other for 90° corners [9, Ch. 10]. Hence, there is a nearly isotropic decrease in the kernel ( Eq. 12) which leads to improved localization of corners as seen in both . We also show the effect of using different robust error measures with real biological images. shows a Lilium longiflorum pollen tube, imaged using high resolution Nomarski optics (diameter of pollen tube is 20 microns). These images are used to study the movement of the tip and small interior vesicles that actively contribute to the growth dynamics of pollen tubes [ 10]. shows a section of the Arabidopsis thaliana root from the meristem region, with root hairs and internal cellular structures (diameter of the root is approximately 100 microns). Temporal stacks of such root images were used to automatically compute the most spatiotemporally accurate growth profile along the medial axis of the root, for several plant species [ 8]. As seen from , the Geman-McClure function does a better job at detecting more salient image features, such as vesicles in the pollen tubes and internal cellular structures in the root, that are important in characterizing the physiology of biological motions. In a previous paper, we have presented growth characterization results using a least-squares based robust tensor algorithm for computing velocity profiles of root growth in Arabidopisis thaliana [ 8]. The accurate localization and segmentation feature of the proposed adaptive robust structure tensor algorithm can be suitably extended for computing such velocity profiles, or growth in other biological organisms. 5 Conclusion and Scope for Future Work An adaptive, robust, structure tensor algorithm has been presented in this paper that extends the robust orientation estimation algorithm by Boomgaard and Weijer [ 2]. The adaptation procedure for local orientation estimation uses a new, spatially varying, adaptive Gaussian kernel that is initialized using the total least-squares structure tensor solution. We adapt the size, orientation and weights of the Gaussian kernel simultaneously at each iteration step. This leads to improved detection of edge and junction features, even in the presence of noise. In a future work, we intend to explore the relationship between the proposed adaptive robust-tensor algorithm and anisotropic diffusion [ 11, 12]. This research was supported in part by the U.S National Institutes of Health, NIBIB award R33 EB00573. The biological images were provided by Dr. Tobias Baskin at the University of Massachusetts, Amherst. 1. Granlund GH, Knutsson H. Signal Processing for Computer Vision. Kluwer Academic Publishers; Dordrecht, Netherlands: 1995. 2. van den Boomgaard R, van de Weijer J. 2nd Intl Work Text Anal Synt. 2. Intl Work Text Anal Synt; Copenhagen, Dennmark: 2002. Robust estimation of orientation for texture analysis; pp. 135–138. 3. Köthe U. Proc IEEE Int Conf Computer Vision. Vol. 1. Nice; France: 2003. Integrated edge and junction detection with the boundary tensor; pp. 424–431. 4. Jähne B, Haußecker H, Scharr H, Spies H, Schmundt D, Schurr U. LNCS-1407: Proc 5th ECCV. Vol. 2. Freiburg, Germany: Springer-Verlag; 1998. Study of dynamical processes with tensor-based spatiotemporal image processing techniques; pp. 322–336. 5. Nagel HH, Gehrke A. LNCS-1407: Proc 5th ECCV. Vol. 2. Freiburg, Germany: Springer-Verlag; 1998. Spatiotemporally adaptive estimation and segmentation of OF-Fields; pp. 86–102. 6. Palaniappan K, Jiang HS, Baskin TI. CVPR-IEEE Workshop on Articulated and Nonrigid Motion. Vol. 1. Washington, DC: 2004. Non-rigid motion estimation using the robust tensor method; pp. 25–33. 7. Black MJ, Rangarajan A. On the unification of line processes, outlier rejection, and robust statistics. Intern J Comput Vis. 1996;19:57–91. 8. Weele C, Jiang H, Palaniappan KK, Ivanov VB, Palaniappan K, Baskin TI. A new algorithm for computational image analysis of deformable motion at high spatial and temporal resolution applied to root growth: Roughly uniform elongation in the meristem and also, after an abrupt acceleration, in the elongation zone. Plant Phys. 2003;132:1138–1148. 9. Jähne B, Haußecker H, Geißler P. Handbook of Computer Vision and Applications - (Signal Processing and Pattern Recognition). Vol. 2. Academic Press; San Diego, CA: 1999. 10. H-Clarke TL, Weddle NM, Kim S, Robi A, Parris C, Kunkel JG, Hepler PK. Effect of extracellular calcium, pH and borate on growth oscillations in Lilium formosanum pollen tubes. J Exp Bot. 2003;54:65–72. [PubMed] 11. Weickert J. Anisotropic Diffusion in Image Processing. Teubner-Verlag; Stuttgart, Germany: 1998. 12. Brox T, van den Boomgaard R, Lauze F, van de Weijer J, Weickert J, Mrázek P, Kornprobst P. Adaptive structure tensors and their applications. In: Weickert J, Hagen H, editors. Visualization and Image Processing of Tensor Fields. Springer-Verlag; Berlin, Germany: 2005.
|
See other articles in PMC that 
