Send to

Choose Destination
IEEE Trans Pattern Anal Mach Intell. 2010 Feb;32(2):304-20. doi: 10.1109/TPAMI.2008.301.

Linear sequence-to-sequence alignment.

Author information

Department of Computer Engineering, Centro Federal de Educação Tecnológica de Minas Gerais, Av. Amazonas 7675, CEP 30510-000 Belo Horizonte, MG, Brazil.


In this paper, we consider the problem of estimating the spatiotemporal alignment between N unsynchronized video sequences of the same dynamic 3D scene, captured from distinct viewpoints. Unlike most existing methods, which work for N = 2 and rely on a computationally intensive search in the space of temporal alignments, we present a novel approach that reduces the problem for general N to the robust estimation of a single line in IR(N). This line captures all temporal relations between the sequences and can be computed without any prior knowledge of these relations. Considering that the spatial alignment is captured by the parameters of fundamental matrices, an iterative algorithm is used to refine simultaneously the parameters representing the temporal and spatial relations between the sequences. Experimental results with real-world and synthetic sequences show that our method can accurately align the videos even when they have large misalignments (e.g., hundreds of frames), when the problem is seemingly ambiguous (e.g., scenes with roughly periodic motion), and when accurate manual alignment is difficult (e.g., due to slow-moving objects).


Supplemental Content

Full text links

Icon for IEEE Engineering in Medicine and Biology Society
Loading ...
Support Center