Combinatorial optimization for higher-order segmentation functionals: Entropy, Color Consistency, Curvature, etc

This talk discusses recent advances in image segmentation in the context of higher-order appearance and smoothness functionals. The most basic segmentation energies combine unary terms enforcing certain color appearance models and pair-wise terms enforcing boundary smoothness. If color models are known, the corresponding binary optimization problem can be globally minimized. Estimation of color models leads to NP-hard mixed optimization problems that are typically solved with iterative block-coordinate descent (Zhu-Yuille, Chan-Vese, GrabCut, etc.) sensitive to initialization. This talk motivates higher-order appearance functionals (e.g. entropy and color-consistency) that do not require explicit estimation of segment appearance models. We show that in many cases such energies can be minimized globally. For example, our approach allows replacing iterative “grabcut” technique with a one cut method finding a global minimum. We also discuss a general Trust Region approach for approximate minimization of other high-order appearance terms. Time permitting, we will also motivate higher-order boundary smoothness terms (e.g. curvature) and describe the corresponding state-of-the-art combinatorial optimization techniques.