- We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

# Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching

^{†}To whom correspondence should be addressed. E-mail: li.ca.noinhcet.sc@nor.

## Abstract

An efficient algorithm for isometry-invariant matching of surfaces is presented. The key idea is computing the minimum-distortion mapping between two surfaces. For this purpose, we introduce the generalized multidimensional scaling, a computationally efficient continuous optimization algorithm for finding the least distortion embedding of one surface into another. The generalized multidimensional scaling algorithm allows for both full and partial surface matching. As an example, it is applied to the problem of expression-invariant three-dimensional face recognition.

**Keywords:**Gromov–Hausdorff distance, isometric embedding, iterative-closest-point, partial embedding

The problem of comparison between deformable surfaces has recently gained a lot of interest in pattern recognition, especially in the fields of three-dimensional (3D) data analysis and synthesis. It arises, for example, in medical imaging and 3D face recognition (1). The fundamental question is how to efficiently yet accurately quantify the similarity between a given reference surface (“model”) and some other surface (“probe”), usually a bent version of the model. We would like to capture the distinction of the model and the probe intrinsic properties associated with the metric structure of the surface, while ignoring the extrinsic properties that describe the way the surface is immersed into the ambient space and that often change while the surface bends. A deformation that preserves the intrinsic structure of the surface is called an “isometry.” Our goal is thus to define a computable isometry-invariant measure of similarity between surfaces. Often, the problem is even more complicated, when having the probe only partially available (see Fig. 1). We refer to the latter setting as “partial surface matching” and to the case where the whole probe is available as “full surface matching.”

One of the earliest attempts of isometry-invariant surface matching is the classical “iterative closest point” (ICP) algorithm (2). It addresses a particular case of the partial matching problem where only rigid (Euclidean) isometries are allowed. An efficient method for the construction of near-isometry-invariant representations of surfaces [called “canonical forms” (CFs)] based on Euclidean embeddings was presented in ref. 3, as a generalization of ref. 4. This approach used a multidimensional scaling (MDS) algorithm (5). MDS is closely related to dimensionality reduction (6, 7) and can be performed in a computationally efficient manner. Euclidean embeddings are used in theoretical computer science for representing metric spaces usually arising from geometry of graphs (8).

Recently, Mémoli and Sapiro (9) used a discrete probabilistic approximation of the Gromov–Hausdorff (GH) distance (10) to compare surfaces up to isometries. Their method is based on evaluating all of the permutations between two sets of surface samples, which is computationally expensive. Moreover, the GH distance is inappropriate for partial matching.

One of the main contributions of this work is the “partial embedding (PE) distance,” motivated by Gromov's theory. For its efficient computation, we introduce an extension of MDS, which we call generalized MDS (GMDS). The key idea is to measure the minimum possible distortion when trying to isometrically embed one surface into another. By using GMDS, we can handle full as well as partial surface matching; this ability is one advantage over a straightforward use of the GH distance.

The paper includes six sections. *Theoretical Foundations* reviews the theoretical background of isometry-invariant surface matching. We define our PE distance and show its relation to the GH distance. The next section, *GMDS*, introduces the GMDS and addresses implementation and optimization considerations. The following section focuses on partial matching of surfaces. In *Results*, we show some experimental results. Finally, in *Conclusions*, we summarize the main results.

## Theoretical Foundations

Let be a smooth connected and compact Riemannian surface (2-manifold) with minimal geodesics between every two points *s, s*′ on . We define the geodesic distance function as . For simplicity, we say “surface ,” implying the underlying metric space (, ) induced by the Riemannian geometry of . In our context, surfaces are perceived from a point of view of metric rather than Riemannian geometry; in other words, we do not consider the Riemannian metric tensor, but the geodesic distances that it induces.

In practice, we work with sampled surfaces represented by finite metric spaces. We use the following definitions: a subset is called an “*r*-covering” (or “*r*-net”) of if , where is a ball of radius *r* around in . A finite *r*-covering of consisting of *N* points is denoted by . The metric on is assumed to be the restricted metric for all *s, s*′ in . We refer to *r* as the “sampling radius.” An arbitrary finite sampling of consisting of *N* points is denoted by , where is called “continuous” surface and a “discrete” one.

Given two surfaces and , a transformation is said to have “distortion” ε if

If in addition ψ is “ε-surjective” (i.e., for all ), it is called an “ε-isometry,” and and are called “ε-isometric.” A transformation with disψ = 0 is called an “isometry,” and and admitting such a transformation are called “isometric.”

**Desiderata.** Let denote the set of all compact surfaces, and denote the space of the equivalence classes of all isometries in , i.e., a space in which a point represents all of the isometries of a compact surface. The similarity between two surfaces can be measured by a nonnegative function . Slightly abusing common terminology, we call *d* a “distance,” even if it is not a metric (often, the terms distance and metric are used as synonyms). Ideally, *d* should have the following properties: **Desideratum 1: Metric properties.** *d* should be a metric on , i.e., satisfy the following conditions: (*i*) ; (*ii*) if and only if and are isometric; (*iii*) ; and (*iv*) .

** Desideratum 2: Good similarity measure.** (

*i*) If , then and are

*c*ε-isometric, where

*c*> 0; (

*ii*) if and are ε-isometric, then . Note that this property is an extension of the second metric axiom to ε-isometries.

** Desideratum 3: Partial matching.** Given a patch of .

** Desideratum 4: Sampling consistency.** Given a finite

*r*-covering of , lim

_{r→0}.

*Desideratum 5: Efficiency*.*d* should be efficiently computable, or at least efficiently approximated. Practically, given and , should be computable in polynomial time of max{*N, N*′}.

Unfortunately, *Desiderata 1*–*3* cannot be satisfied simultaneously. In practical applications, it appears that the ability to perform partial matching (*Desideratum 3*) is the most important. In the following, we show a distance that satisfies *Desiderata 3*–*5* and partially satisfies *Desiderata 1* and *2*.

It is also important to note the following potential danger of partial matching. Consider an extreme case of a smooth curve on . Obviously, if *Desideratum 3* holds, then , but because a curve can be isometrically embedded into any surface, we are liable to obtain for an arbitrary which is not necessarily an isometry of . In other words, when the patch is too small, partial matching can be meaningless.

**GH Distance.** Our starting point is the GH distance introduced by Gromov (10). For two surfaces and , it is defined by

where and are isometric embeddings into the metric space , and is the Hausdorff distance between two subsets and of . For bounded metric spaces and , the GH distance can be defined alternatively (11) as

where .

**Proposition 1.** (*i*) *d*_{GH} *is a finite metric on* ; (*iia*) , *then* *and* *are 2*ε*-isometric*; (*iib*) *if* *and* *are ε-isometric, then* ; (*iii*) .

For proof, see ref. 11. The GH distance is a good candidate for our “ideal” similarity measure between surfaces; from *Proposition 1*, it follows that it satisfies *Desiderata 1, 2, and 4*. However, there are two main disadvantages. First, the definition in Eq. **2** is hard to compute (*Desideratum 5*). Secondly, *d*_{GH} does not allow partial matching between surfaces (*Desideratum 3*). Given a surface and a patch such that (i.e., is an *R*-covering of ), we can have , although we would have liked the distance between the surface and its patch to be zero.

**PE Distance.** Our assumption was that we have a model surface and a probe surface , which can be an isometrically bent patch of . We have shown the disadvantage of the GH distance in such a setup. To allow for partial matching, we define the PE distance

Intuitively, *d*_{PE} measures the metric distortion obtained trying to embed into in the “most isometric” way. It can be considered as only part of the GH distance, hence the name “partial embedding.”

**Proposition 2.** *Given compact surfaces **, **, and **, d*_{PE} *satisfies the following*: (*i*) ; (*ii*) *if and only if* *is isometrically embeddable into* ; *and* (*iii*) .

The proof closely follows the proof of theorem 7.3.30 in ref. 11. Still, *d _{PE}* is not symmetric, and furthermore, not a metric on . Yet, it satisfies some important metric properties, namely: isometry invariance (

*ii*) and a nonsymmetric version of the triangle inequality (

*iii*). Moreover, because , it trivially follows that if and are ε-isometric, then . The converse appears to be true under some more restrictive assumptions.

Given an *r*′-covering of and an *r*-covering of , the distance *d*_{PE} satisfies the following important properties.

**Proposition 3.** (*i*) ; (*ii*) .

Note that a particular case of *Proposition 3i* is . By virtue of this property [unlike the corresponding property of the GH distance] *d*_{PE} can be used for partial matching. From *Proposition 3* it also follows that *d*_{PE} is consistent with sampling, namely, given and , two finite coverings of and , respectively, . In other words, if we have a sufficiently dense finite sampling of and , is a sufficiently good approximation of the continuous .

## GMDS

We now address a practical question of computing the PE distance *d*_{PE}, starting from the CF approach (3). As an input, we assume to be given two sampled surfaces and (representing the continuous surfaces and ) and the geodesic distances between the samples , represented by an *N* × *N* matrix and an *N*′ × *N*′ matrix , respectively.

**CF and MDS.** In ref. 3, a three-stage algorithm for near-isometry-invariant surface matching was proposed. First, each of the sampled surfaces is mapped into an *m*-dimensional Euclidean space by a near-isometric embedding (“flattening”), obtained by minimization of the stress function

with respect to **X**. Here **X** denotes an *N* × *m* matrix of coordinates in and denotes the Euclidean metric. The stress can be thought of as an *L*_{2} measure of the distance distortion caused by such an embedding. Algorithms minimizing the stress σ are known as MDS (5). was called the CF of and the “embedding error.”

Note that both CFs and are defined up to a Euclidean isometry (translation, rotation, and reflection). The second stage of the CF algorithm is an alignment of the CFs in . It is obtained by diagonalizing the matrix of the second-order moments of each CF and reordering the axes such that the diagonal values are decreasing. At the third stage, a distance based on high-dimensional moments (12) is used to compare between and after alignment. We denote the distance computed as rigid matching between the CFs of and by .

Strictly speaking, the CF algorithm satisfies only *Desideratum 5*: it is very efficient computationally. Unlike alternative combinatorial optimization, MDS is a continuous optimization problem that can be solved by using standard optimization methods. Recently, a very fast multigrid implementation of the CF algorithm was presented (13). However, the CF algorithm lacks true isometry invariance, because, in general, a surface cannot be isometrically embedded into (8). Therefore, this “flattening” introduces a distortion. Partial matching is also complicated using this method.

**GMDS.** The choice of a Euclidean embedding space was followed by other choices, like spherical (14, 15) and hyperbolic (16) spaces, which proved to be advantageous for certain types of surfaces. The leitmotif of these approaches was choosing an embedding space in which geodesic distances can be expressed analytically. In practice, this possibility exists only in very few spaces.

So far, we have a good distance *d*_{PE}, which satisfies our Desiderata on one hand and an efficient MDS framework for computing Euclidean embeddings on the other. Next, we unite theory and practice. We consider a generalization of the CF algorithm, where surface is embedded into another surface . We call this procedure GMDS.

Similarly to the Euclidean case, we define the “generalized stress” as

for 1 ≤ *p* < ∞, and

for *p* = ∞, where the matrix **U** represents the positions of *N*′ points on in some local or global parametric coordinates **u*** _{i}*, and

**W**= (

*w*) is a symmetric matrix of nonnegative weights. Note two major differences between the generalized stress and the stress defined in Eq.

_{ij}**4**. First, the metric in the embedding space is not given analytically and is approximated numerically. Second, we replaced the

*L*

_{2}norm with a general

*L*norm. Furthermore, the embedding space is 2D, whereas in the CF algorithm the embedding space is usually at least 3D.

_{p}Choosing *p* = ∞ and **W** = **1**_{N}_{′×}_{N}_{′} we easily obtain

In other words, minimization of the generalized stress allows computing *d*_{PE}. We discuss numerical optimization algorithms in the next section. Moreover, it is possible to use a similar procedure to compute *d*_{GH} (A.M.B., M.M.B., and R.K., unpublished results). In practice, it may appear that using other norms (e.g., *p* = 1 or 2) is advantageous. In *Results*, it is demonstrated that we can use σ* _{p}* instead of σ

_{∞}. We denote the distances computed in this way by .

An important issue is the choice of the parametric coordinate system. For objects acquired by a range sensor, a global parameterization (e.g., *U* = [0, 1]^{2}), is usually readily available. Global parameterization for objects with simple topology (e.g., homeomorphic to a disk or a sphere) is also very natural and can be computed from a general mesh by a variety of efficient algorithms (17). For general triangulated meshes with more sophisticated topology, a single global coordinate system may not suffice. In this case, we propose to resort to a local coordinate system, in which each point on the polyhedral surface is represented by an index of the triangle to which it belongs and a vector in barycentric coordinates of that triangle.

**Computational Aspects.** For 1 ≤ *p* < ∞, the generalized stress (5) is minimized by iterative gradient-type optimization methods. Here, for simplicity, we show the projected gradient descent algorithm for a global parameterization ; in practice, more efficient optimization methods such as conjugate gradients or quasi-Newton can be used (18). Note that because the problem is nonconvex, such methods do not necessarily converge to the global minimum. Local convergence is a common property of MDS algorithms in general (5). There are standard techniques to overcome this problem, one of which is multiscale or multigrid optimization (13, 19).

For globally parameterized surfaces, the basic projected gradient descent step has the form

where *k* is the iteration number. α^{(}^{k}^{)} is the step size on iteration *k*; it is computed by using line search (18). *P _{U}* is a projection operator onto the parameterization domain

*U*.

*σ*

_{U}*denotes the gradient of σ*

_{p}*with respect to*

_{p}*U*; its computation involves the distance function and its derivatives. When local parameterization is used, the optimization algorithm becomes more elaborate because it requires unfolding of triangular faces of the mesh and transformations from barycentric coordinate systems of adjacent triangles to compute the stress along a polylinear path.

For *p* =∞, we rewrite the problem as constrained optimization with 2*N*′ + 1 variables, a linear objective, and *N*′(*N*′ – 1) nonlinear inequality constraints using an artificial variable τ,

for all *j* > *i*. The new problem can be solved by using standard constrained optimization methods, e.g., penalty-barrier or augmented Lagrangian (18).

A crucial part of the GMDS problem is the computation of geodesic distances. We use the “fast marching method” (FMM), which computes geodesic distances on surfaces by solving the eikonal equation on general triangulated meshes (20). For surfaces with global parameterization, an efficient parametric version of FMM can be used (21).

The geodesic distances between the fixed sample points of can be precomputed by using FMM. However, the **u*** _{i}* coordinates representing ψ(

*q*) on change during the iterations of the numerical minimization algorithm. Thus, the distances have to be reevaluated at every iteration. This computation is critical for the GMDS. For that goal, we developed a numerical procedure we termed “three-point geodesic distance approximation” (detailed in A.M.B., M.M.B., and R.K., unpublished results). The idea is to produce a for and its derivatives, interpolating their values from the matrix .

_{i}## Partial Matching of Surfaces

Let be a patch of the probe surface and be a finite sampling of . We consider the distance . By our assumption, the metric on is given by restricting to , i.e., . However, is computed numerically on and can be inconsistent with . Specifically, geodesics that pass through points not in give rise to inconsistent distances, for which (see Fig. 2 *Left*). In pathological cases, the inconsistency can be arbitrarily large, such that the partial matching makes little sense (see example in Fig. 2 *Right*). To overcome this problem, the inconsistent distances are excluded defining the corresponding weights *w _{i}*

_{′}

_{j}_{′}= 0 in the generalized stress (5).

*Left*) Correct use of weights (shown in solid red is an inconsistent distance that must be excluded from the stress computation and in dashed blue is the corresponding correct distance). (

*Right*) The embedding

**...**

A practical problem is how to locate the inconsistent distances. Here, we address two possible scenarios: (*i*) the original surface or its sampled version is available, and (*ii*) only the patch is available. The first scenario occurs when one intentionally wishes to exclude parts of . In this case, the FMM is used first to compute the distances on and then to compute the distances on . The weights are defined by

for all *i, j* = 1,... *N*′, where δ stands for the FMM accuracy.

The second scenario is more general, yet, also a more challenging one. When the original surface is not given, we need to rely on the geodesics of [in practice, the geodesics are found by backtracking (22)]. By using the above argument, we define the weights by

for all *i, j* = 1,... *N*′, i.e., excluding the geodesics that touch the boundary .

## Results

**Embedding of Spherical Surfaces.** In the first experiment, we measured the distance between a unit 2D sphere sampled at 3,200 points, and spheres with radii in the range 0.5 ÷ 2.5 sampled at a smaller number of points according to the “farthest point sampling” strategy (23) with random seed. Ten random samplings were used for each radius. Two distance measures were compared as follows: (obtained by the minimization of σ_{2}) with 100 points and *d*_{CF} with 100, 250, and 500 points. was computed by using local representation of points in barycentric coordinates.

Fig. 3 presents the normalized distances as a function of the sphere radius. The PE distance appears to be extremely sensitive to the geometry of the surfaces. A change as small as 0.1% in the spherical patch radius (from 1.000 to 0.999) increases by a value exceeding the variance due to sampling. Similar results are achieved by using and slightly inferior with *d*_{PE}. For comparison, with the same number of points (100), *d*_{CF} is unable to discern between spheres differing in radius by >10%. Increasing the number of points to 500 makes *d*_{CF} sensitive to radius differences of ≈2.64%, which is still one order of magnitude below the sensitivity of .

*d*

_{CF}(three bottom lines). The numbers on the plots stand for the number of samples in the embedded surface.

**Face Recognition Experiment.** In ref. 1, we argued that human facial expressions can be approximated as isometric deformations of the face. According to this model, an isometry-invariant surface matching results in nearly expression-invariant face recognition.

In the second experiment, we applied our GMDS approach to a toy 3D face recognition problem of 20 faces taken from the Notre Dame database (24, 25). This experiment is a proof of concept rather than a real face recognition system, in which tune-up plays a crucial role. As models, four different faces were used. Each model contained ≈4,000 points and was represented using global parameterization. As probes, faces of the same subjects with different facial expressions were used (four probes per subject). The probes were cropped by using geodesic mask (1) (leaving the nose and the forehead region) and subsampled by using farthest point sampling to 53 points. The distances were weighted as described in *Partial Matching of Surfaces*. All distances were computed numerically by a version of parametric FMM (21). We tested two PE distances: *d*_{PE} and . The generalized stress function σ* _{p}* and its gradient were implemented in C. For 53 points, the computation time of σ

*and σ*

_{p}*is ≈20 msec on a Pentium IV PC.*

_{p}Fig. 4 presents the similarity matrix using *d*_{PE} (second row) and (third row). Note that in both cases a perfect separation between different subjects is possible; the recognition error (measured as “equal error rate,” or EER; see ref. 26 for definition) in this experiment is zero. It is worthwhile noting the small number of points used in our approach, compared with the thousands of points required in the CF algorithm (1, 3) for similar matching accuracy.

*Top*) Model facial surfaces. (

*Middle*and

*Bottom*) Similarity matrices obtained using

*d*

_{PE}(

*Middle*) and (

*Bottom*), respectively. All values are shown in logarithmic scale.

An example of embedding probe surfaces of two different subjects into a model surface are shown in Fig. 5. The figure depicts the “local stress” value at each point *k*, i.e., the contribution of the *k*th point to σ_{∞}. We observe that the stress is significantly higher when a probe of a different subject is embedded into the model surface.

## Conclusions

We introduced a previously undescribed framework for isometry-invariant surface matching. The core of our computational framework is GMDS, a generalization of standard MDS algorithms. We exemplified the use of GMDS on the problem of expression-invariant 3D face recognition.

Our approach favorably compares with previous attempts to perform isometry-invariant surface matching. First, our PE distance naturally allows for isometry-invariant matching of partially missing surfaces. Secondly, the properties of our distance and its computation are completely deterministic. Thirdly, GMDS used for the PE distance computation is a continuous optimization problem and can be solved very efficiently by using standard optimization methods. In light of recent results (13), we expect a significant performance boosting by using multigrid methods.

Finally, we note that the same numerical framework can be applied for computing the exact GH distance between surfaces (A.M.B., M.M.B., and R.K., unpublished results).

## Acknowledgments

We thank Alexander Brook and Roman Goldenberg for thorough reading of the manuscript, Facundo Mémoli and Vladimir Lin for fruitful discussions, and our reviewers for valuable comments. This work was supported in part by United States-Israel Science Foundation Grant 2004274, Israeli Science Foundation Grant 738/04, and the Charles Krown Research Fund.

## Notes

Author contributions: A.M.B., M.M.B., and R.K. designed research, performed research, contributed new reagents/analytic tools, analyzed data, and wrote the paper.

Conflict of interest statement: No conflicts declared.

This paper was submitted directly (Track II) to the PNAS office.

Abbreviations: CF, canonical form; FMM, fast marching method; GH, Gromov–Hausdorff; MDS, multidimensional scaling; GMDS, generalized MDS; PE, partial embedding.

## References

**,**5–30.

**,**119–152.

**,**1285–1295.

**,**1005–1008.

**,**2323–2326. [PubMed]

**,**5591–5596. [PMC free article] [PubMed]

**,**333–344.

**,**622–631.

**,**99–116.

**,**8431–8435. [PMC free article] [PubMed]

**,**315–327.

**,**1305–1315. [PubMed]

**,**619–624. [PubMed]

**National Academy of Sciences**

## Formats:

- Article |
- PubReader |
- ePub (beta) |
- PDF (336K)

- Diffeomorphism invariant Riemannian framework for ensemble average propagator computing.[Med Image Comput Comput Assist Interv. 2011...]
*Cheng J, Ghosh A, Jiang T, Deriche R.**Med Image Comput Comput Assist Interv. 2011; 14(Pt 2):98-106.* - Calculus of nonrigid surfaces for geometry and texture manipulation.[IEEE Trans Vis Comput Graph. 2007]
*Bronstein A, Bronstein M, Kimmel R.**IEEE Trans Vis Comput Graph. 2007 Sep-Oct; 13(5):902-13.* - Conformal metric optimization on surface (CMOS) for deformation and mapping in Laplace-Beltrami embedding space.[Med Image Comput Comput Assist Interv. 2011...]
*Shi Y, Lai R, Gill R, Pelletier D, Mohr D, Sicotte N, Toga AW.**Med Image Comput Comput Assist Interv. 2011; 14(Pt 2):327-34.* - Face recognition using face-ARG matching.[IEEE Trans Pattern Anal Mach Intell. 2005]
*Park BG, Lee KM, Lee SU.**IEEE Trans Pattern Anal Mach Intell. 2005 Dec; 27(12):1982-8.* - Conformal geometry and its applications on 3D shape matching, recognition, and stitching.[IEEE Trans Pattern Anal Mach Intell. 2007]
*Wang S, Wang Y, Jin M, Gu XD, Samaras D.**IEEE Trans Pattern Anal Mach Intell. 2007 Jul; 29(7):1209-20.*

- Correlation and Interaction Visualization of Altmetric Indicators Extracted From Scholarly Social Network Activities: Dimensions and Structure[Journal of Medical Internet Research. ]
*Liu CL, Xu YQ, Wu H, Chen SS, Guo JJ.**Journal of Medical Internet Research. 15(11)e259* - Algorithms to automatically quantify the geometric similarity of anatomical surfaces[Proceedings of the National Academy of Scie...]
*Boyer DM, Lipman Y, St. Clair E, Puente J, Patel BA, Funkhouser T, Jernvall J, Daubechies I.**Proceedings of the National Academy of Sciences of the United States of America. 2011 Nov 8; 108(45)18221-18226* - Molecular Evolution of Phosphoprotein Phosphatases in Drosophila[PLoS ONE. ]
*Miskei M, Ádám C, Kovács L, Karányi Z, Dombrádi V.**PLoS ONE. 6(7)e22218* - Near-isometric flattening of brain surfaces[NeuroImage. 2010]
*Balasubramanian M, Polimeni JR, Schwartz EL.**NeuroImage. 2010 Jun; 51(2)694-703* - Applying unmixing to gene expression data for tumor phylogeny inference[BMC Bioinformatics. ]
*Schwartz R, Shackney SE.**BMC Bioinformatics. 1142*

- Generalized multidimensional scaling: A framework for isometry-invariant partial...Generalized multidimensional scaling: A framework for isometry-invariant partial surface matchingProceedings of the National Academy of Sciences of the United States of America. Jan 31, 2006; 103(5)1168PMC

Your browsing activity is empty.

Activity recording is turned off.

See more...