Geoffroy Fouquier | Home | Publications

Travaux de thèse | Phd thesis

Optimisation de séquences de segmentation combinant
modèle structurel et focalisation de l'attention visuelle.
Application à la reconnaissance de structures cérébrales dans des images IRM 3D.

Sequential segmentation optimization using a structural model and focus of visual attention.
Application to the recognition of internal brain structures in 3D magnetic resonance images (MRI).

Abstract

We aim at recognizing a 3D scene described by a 3D image and a structural model, i.e., a model that describes the spatial arrangement of the objects. The sequential segmentation framework is considered. This allows us to segment and recognize objects in a sequential way, using at each step the previously recognized object to guide the segmentation of the next ones. We propose to use the spatial information included in the model to optimize the segmentation sequence from a reference object to a selected target. This sequence is viewed as a path in a graph where vertices represents objects and edges represents spatial relations.

Two approaches are proposed and both approaches are used for segmentation and recognition of internal brain structures in 3D magnetic resonance images. We also propose an adaptation of these methods to cope with pathological cases (e.g., brain tumors).

First approach: offline evaluation of the relevance of a path in a spatial graph

This approach has been introduced in this publication: GBR 2007

It proposes to evaluate the relevance of a path according to the generic available knowledge. This estimation is realized either on each spatial relation independently or directly on a fuzzy subset that represents the whole path at once. The best path according to a criterion is then selected and the objects may be segmented.

Figure 1: The first approach consists in an evaluation of path relevance in order to select the best path. Two different methods are proposed: the first consists in an independent evaluation of each spatial relation then a graph optimization to choose the best path. The second computes a representation of the whole graph as a fuzzy subset and then evaluates the relevance.

Second approach: sequential segmentation guided by a pre-attentive mechanism

This approach has been introduced in this publication: ECAI 2008. The learning procedure for spatial relation parameters is presented in this publication: IJCAI 2007. A journal paper is currently submitted.

This approach proposes to integrate the segmentation sequence optimization directly into a sequential segmentation framework proposed in [1]. The latter work is defined as a knowledge-based object recognition approach where objects are segmented in a predefined order, starting from the simplest object to segment to the most difficult one. The choice of the most appropriate sequence of segmentation is one of the difficulties raised by this approach. It also lacks a step which could evaluate a segmentation of a particular object and detect errors to avoid their propagation.

Our contribution is to extend the sequential segmentation framework by introducing a pre-attentional mechanism (a saliency map [2]) in the optimization of the segmentation order on the one hand, and to introduce criteria and a data structure which allow us to detect errors and control the ordering strategy on the other hand.

Figure 2: General scheme of the sequential segmentation framework guided by a saliency map. The process starts with a model graph with generic knowledge and is sequentially specialized to a specific image.

Figure 3: The segmentation framework used the spatial knowledge represented in the image space as defined in [3] in order to reduce the search domain and for the selection of the next structure to segment.

Figure 4: Spatial consistency criterion used for the evaluation of the segmentation of a specific structure.

Figure 5: Segmentation result with our approach guided by pre-attentive features (a slice from a 3D segmented volume). 6 structures are segmented. The segmentation of each structure is achieved with a deformable model described in [1]. The tail of the putamen is not segmented due to the radiometry and its shape.

[1] Olivier Colliot, Oscar Camara and Isabelle Bloch. Integration of fuzzy spatial relations in deformable models. Application to brain MRI segmentation. Pattern Recognition, Volume 39, Issue 8, August 2006, Pages 1401-1414.

[2] L. Itti, C. Koch, E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, Nov. 1998. Vol. 20, Issue 11, pages 1254-1259

[3] I. Bloch. Fuzzy Spatial Relationships for Image Processing and Interpretation: A Review. Image and Vision Computing, 2005, Vol. 23, Nb. 2, pages 89-110

Résumé des travaux

Nos travaux portent sur l'interprétation d'une scène dont nous possédons un modèle, représentant l'agencement spatial des objets contenus dans cette scène. Dans le cadre d'une segmentation séquentielle permettant de reconnaître les objets les uns après les autres en fonction des étapes antérieures, nous utilisons la connaissance spatiale du modèle pour optimiser la séquence de segmentation à effectuer à partir d'un objet de référence vers un objectif à segmenter. Nous proposons pour cela d'optimiser un chemin dans un graphe représentant les objets de la scène (noeuds) et leurs relations spatiales (arcs). Deux approches sont proposées.

La première approche effectue une optimisation à partir de l'information spatiale du modèle uniquement, en évaluant un critère de pertinence de chaque chemin. L'évaluation est effectuée de manière indépendante sur chaque arc dans un premier temps, puis nous proposons une manière de représenter un chemin entier, permettant d'évaluer la pertinence du chemin à partir de cette représentation.

La deuxième approche s'intègre dans un processus de segmentation séquentielle, vu comme l'exploration progressive d'une image à partir d'un objet de référence. Nous utilisons une modélisation d'une technique pré-attentionnelle, une carte de saillance, afin de guider le processus de segmentation séquentielle, en intégrant à l'approche structurelle des informations de saillance extraites de l'image à interpréter.

Le domaine d'application de ces approches est la segmentation des structures sous-corticales du cerveau dans des images IRM 3D dont certaines présentent des pathologies.