Visual Correspondence Using Energy Minimization and Mutual Information

Junhwan Kim, Vladimir Kolmogorov and Ramin Zabih.

In IEEE International Conference on Computer Vision (ICCV), October 2003.


We address visual correspondence problems without assuming that scene points have similar intensities in different views. This situation is common, usually due to non-lambertian scenes or to differences between cameras. We use maximization of mutual information, which is a powerful technique for registering images that requires no a priori model of the relationship between scene intensities in different views. However, it has proven difficult to use mutual information to compute dense visual correspondence. Using mutual information to compare fixed-size windows suffers from the well-known problems of fixed windows, namely poor performance at discontinuities and in low-texture regions. In this paper, we show how to compute visual correspondence using mutual information without suffering from these problems. Using a simple approximation, we show how to incorporate mutual information into the standard energy minimization framework used in early vision. The energy can then be efficiently minimized using graph cuts, which preserves discontinuities and handles low-texture regions. Experiments demonstrate that our algorithm is effective for many situations, including real scenes that have had their intensities artificially altered.