Pattern Mining in Computer Vision

Visual pattern minig is a useful tool to construct mid-level visual patterns that can be used for image classification, image retrieval, and image browsing. In this project page we present some of our work in visual pattern mining.

Effective Use of Frequent Itemset Mining for Image Classification

ECCV 2012

Basura Fernando*, Elisa Fromont** and Tinne Tuytelaars*
*KU Leuven ESAT-PSI IBBT Belgium
**UMR CNRS 5516 Laboratoire Hubert Curien University of Saint-Etienne France

Abstract

In this paper we propose a new and effective scheme for applying frequent itemset mining to image classification tasks. We refer to the new set of obtained patterns as Frequent Local Histograms or FLHs. During the construction of the FLHs, we pay special attention to keep all the local histogram information during the mining process and to select the most relevant reduced set of FLH patterns for classification. The careful choice of the visual primitives and some proposed extensions to exploit other visual cues such as colour or global spatial information allow us to build powerful bag-of-FLH-based image representations. We show that these bag-of-FLHs are more discriminative than traditional bag-of-words and yield state-of-the art results on various image classification benchmarks.

READ THE FULL PAPER

Project









Mining Multiple Queries for Image Retrieval: On-the-fly learning of an Object-specific Mid-level Representation

ICCV 2013

Basura Fernando* and Tinne Tuytelaars*
*KU Leuven ESAT-PSI IBBT Belgium

Abstract

In this paper we present a new method for object retrieval starting from multiple query images. The use of multiple queries allows for a more expressive formulation of the query object including, e.g., different viewpoints and/or viewing conditions. This, in turn, leads to more diverse and more accurate retrieval results. When no query images are available to the user, they can easily be retrieved from the internet using a standard image search engine. In particular, we propose a new method based on pattern mining. Using the minimal description length principle, we derive the most suitable set of patterns to describe the query object, with patterns corresponding to local feature configurations. This results in a powerful object-specific mid-level image representation. The archive can then be searched efficiently for similar images based on this representation, using a combination of two inverted file systems. Since the patterns already encode local spatial information, good results on several standard image retrieval datasets are obtained even without costly re-ranking based on geometric verification.

READ THE FULL PAPER

Project









Mining Mid-level Features for Image Classification

IJCV 2014

Basura Fernando*, Elisa Fromont** and Tinne Tuytelaars*
*KU Leuven ESAT-PSI IBBT Belgium
**UMR CNRS 5516 Laboratoire Hubert Curien University of Saint-Etienne France

Abstract

Mid-level or semi-local features learnt using class-level information are potentially more distinctive than the traditional low-level local features constructed in a purely bottom-up fashion. At the same time they preserve some of the robustness properties with respect to occlusions and image clutter. In this paper we propose a new and effective scheme for extracting mid-level features for image classification, based on relevant pattern mining. In particular, we mine relevant patterns of local compositions of densely sampled low-level features. We refer to the new set of obtained patterns as Frequent Local Histograms or FLHs. During this process, we pay special attention to keeping all the local histogram information and to selecting the most relevant reduced set of FLH patterns for classification. The careful choice of the visual primitives and an extension to exploit both local and global spatial information allow us to build powerful bag-of-FLH-based image representations. We show that these bag-of-FLHs are more discriminative than traditional bag-of-words and yield state-of-the-art results on various image classification benchmarks, including Pascal VOC.

READ THE FULL PAPER

Project









Dataset Fingerprints: Exploring Image Collections Through Data Mining

CVPR 2015

Konstantinos Rematas, Basura Fernando, Frank Dellaert, and Tinne Tuytelaars

Abstract

As the amount of visual data increases, so does the need for summarization tools that can be used to explore large image collections and to quickly get familiar with their content. In this paper, we propose \emph{dataset fingerprints}, a new and powerful method based on data mining that extracts meaningful patterns from a set of images. The discovered patterns are compositions of discriminative mid-level features that co-occur in several images. Compared to earlier work, ours stands out because i) it's fully unsupervised, ii) discovered patterns cover large parts of the images,often corresponding to full objects or meaningful parts thereof, and iii) different patterns are connected based on co-occurrence, allowing a user to ``browse'' the images from one pattern to the next and to group patterns in a semantically meaningful manner.

READ THE FULL PAPER

Project









Acknowledgements

The authors acknowledge the support of the IBBT Impact project Beeldcanon and the FP7 ERC Starting Grant 240530 COGNIMUND **.