Medical Keypoint Transfer Segmentation

Abstract

We present an image segmentation method that transfers label maps of entire organs from the training images to the novel image to be segmented. The transfer is based on sparse correspondences between keypoints that represent automatically identified distinctive image locations. Our segmentation algorithm consists of three steps: (i) key point matching, (ii) voting-based keypoint labeling, and (iii) keypointbased probabilistic transfer of organ label maps. The accuracy of our method compares favorably to common multi-atlas segmentation while offering a speed-up of about three orders of magnitude. Furthermore, keypoint transfer requires no training phase or registration to an atlas. The algorithm’s robustness enables the segmentation of scans with highly variable field-of-view.

Keywords

algorithm   atlas   image   keypoint   transfer   scan   Segmentation

1. Introduction

Image segmentation refers “to the partition of an image into a sets of region that cover it. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze” [1]. In medical imaging, these segments often correspond to different tissue classes, organs, pathologies, or other biologically relevant structures [2]. Atlas-based segmentation is a medical imaging technique to do image segmentation. Training images are manualy labeled by a clinical expert to segment unseen images from these training images. Image registration is a process that searches to correct alignement of images “to align the atlas image or images to a new, unseen image” [3].

2. Problem Statement

Is atlas-based segmentation without dense correspondences possible? Typical registration- and patch-based segmentation methods [4, 5, 6. 7] compute correspondences for each location in the novel image to be segmented to the training images. These correspondences are either obtained from dense deformation fields or from the retrieval of similar patches. For scans with a large field-of-view, such approaches become computationally intense.

Fig.1: Flowchart of the atlas-based segmentation approach.

3. Segmentation Method as Solution Proposed

We propose a segmentation method based on distinctive locations in the image – keypoints. In contrast to manually selected landmarks [8], keypoints are automatically extracted as local optima of a saliency function [9]. Matches between keypoints in test and training images provide correspondences for a sparse set of image locations, which we use to transfer entire organ segmentations. Working with sparse correspondences and transferring entire organ maps makes our method computationally efficient. The probabilistic fusion of organ maps across all matches and training subjects yields a segmentation accuracy comparable to that of state-of-the-art methods, while offering orders of magnitude of speed-up.

Keypoint matching offers the additional advantage of robustness in establishing correspondences between images with varying field-of-view. This is important when using manually annotated whole-body scans to segment clinical scans with a limited field-of-view. In clinical practice, the diagnostic focus is commonly on a specific anatomical region. In order to minimize radiation dose to the patient and scanning time, only the region of interest is scanned. For instance, CT scans of the kidneys are acquired to assess the presence of tumors, kidney stones or abscesses. The alignment of scans with a limited field-of-view to full abdominal scans is challenging with intensity-based registration, especially when the initial transformation does not roughly align corresponding anatomical structures. The efficient and robust segmentation through keypoint transfer offers a practical tool to handle the growing number of clinical scans.

Fig.2: Illustration of keypoint transfer segmentation. First, keypoints (white circles) in training and test images are matched (arrow). Second, voting assigns an organ label to the test keypoint (r.Kidney). Third, matches from the training images with r.Kidney as labels are transferred to the test image, creating a probabilistic segmentation. We show the manual segmentation for comparison.

Fig. 2 illustrates the keypoint transfer segmentation. Keypoints are identified at salient image regions invariant to scale. Each keypoint is characterized by its geometry and a descriptor based on a local gradient histogram. After keypoint extraction, we obtain the segmentation in three steps. First, keypoints in the test image are matched to keypoints in the training images based on the geometry and the descriptor. Second, reliable matches vote on the organ label of the keypoint in the test image. In the example, two matches vote for right kidney and one for liver, resulting in a majority for right kidney. Third, we transfer the segmentation mask from the entire organ for each match that is consistent with the majority label vote; this potentially transfers the organ map from one training image multiple times if more than one match is identified for this training image. The algorithm also considers the confidence of the match in the keypoint label voting and evaluates the similarity of the image regions. Keypoint transfer does not require a training stage and its ability to approximate the organ shape can further improve with the growing number of manually labeled images.

4. Method Evaluation

We evaluate our method on the publicly available Visceral dataset [10, 11]. Multi-atlas segmentation on the Visceral data was proposed by [12, 13], which we use as a baseline method in our experiments. Our work builds on the identification of keypoints, defined as a 3D extension [14] of the popular scale invariant feature transform (SIFT) [15]. In addition to image alignment, 3D SIFT features were also applied to study questions related to neuroimaging [16, 17]. In contrast to previous uses of the 3D SIFT descriptor, we use it to transfer information across images.

Fig.3 : 3D SIFT VIEW – Lung CT.

5. Conclusion

We introduced an image segmentation method based on keypoints that transfers label maps of entire organs. Relying on sparse correspondences between keypoints in the test and training images increases the effi ciency of the method. Keypoint matches are further robust to variations in the field-of-view of the images, which enables segmentation of partial scans. Our algorithms for the keypoint voting and the segmentation transfer were derived from generative models, where latent random variables were marginalized out. The accuracy of our segmentation compares favorably to multi-atlas segmentation, while requiring about three orders of magnitude less computation time.

Related Research Articles

To get more information on this subject, we invite you to read the following article: