Active index for content-based medical image retrieval

A review of intelligent content-based indexing and browsing of medical images

Physicians are beginning to be able to gain access, through the Internet, to the world’s collections of multimedia medical information such as MRI (magnetic resonance imaging) and CT (computer tomography) image archives, videos of surgical operations and medical lectures, textual patient records and media annotations. New techniques and tools are needed to represent, index, store and retrieve digital content efficiently across large collections. In this review, we trace the development of visual information systems for healthcare and medicine from Picture Archiving and Communications Systems (PACS) to the recent advances in content-based image retrieval, whereby images are retrieved based on their visual content similarity - that is, colour, texture, and shape. Medical images, unlike consumer-oriented images, pose additional challenges to content-based image retrieval, in that visual features of normal and pathological images are typically separated by only subtle differences in visual appearance. Intelligent image retrieval and browsing therefore requires a combination of prior knowledge of the medical domain, image content and image annotation analysis. To this end, we also overview the I-Browseproject, conducted jointly by the Clinical School of the University of Cambridge and the City University of Hong Kong, which aims to develop techniques which enable a physician to search over image archives through a combination of semantic and iconic contents.

A review of content-based image retrieval systems in medical applications-clinical benefits and future directions

Content-based visual information retrieval (CBVIR) or content-based image retrieval (CBIR) has been one on the most vivid research areas in the field of computer vision over the last 10 years. The availability of large and steadily growing amounts of visual and multimedia data, and the development of the Internet underline the need to create thematic access methods that offer more than simple text-based queries or requests based on matching exact database fields. Many programs and tools have been developed to formulate and execute queries based on the visual or audio content and to help browsing large multimedia repositories. Still, no general breakthrough has been achieved with respect to large varied databases with documents of differing sorts and with varying characteristics. Answers to many questions with respect to speed, semantic descriptors or objective image interpretations are still unanswered. In the medical field, images, and especially digital images, are produced in ever-increasing quantities and used for diagnostics and therapy. The Radiology Department of the University Hospital of Geneva alone produced more than 12,000 images a day in 2002. The cardiology is currently the second largest producer of digital images, especially with videos of cardiac catheterization ( approximately 1800 exams per year containing almost 2000 images each). The total amount of cardiologic image data produced in the Geneva University Hospital was around 1 TB in 2002. Endoscopic videos can equally produce enormous amounts of data. With digital imaging and communications in medicine (DICOM), a standard for image communication has been set and patient information can be stored with the actual image(s), although still a few problems prevail with respect to the standardization. In several articles, content-based access to medical images for supporting clinical decision-making has been proposed that would ease the management of clinical data and scenarios for the integration of content-based access methods into picture archiving and communication systems (PACS) have been created. This article gives an overview of available literature in the field of content-based access to medical image data and on the technologies used in the field. Section 1 gives an introduction into generic content-based image retrieval and the technologies used. Section 2 explains the propositions for the use of image retrieval in medical practice and the various approaches. Example systems and application areas are described. Section 3 describes the techniques used in the implemented systems, their datasets and evaluations. Section 4 identifies possible clinical benefits of image retrieval systems in clinical practice as well as in research and education. New research directions are being defined that can prove to be useful. This article also identifies explanations to some of the outlined problems in the field as it looks like many propositions for systems are made from the medical domain and research prototypes are developed in computer science departments using medical datasets. Still, there are very few systems that seem to be used in clinical practice. It needs to be stated as well that the goal is not, in general, to replace text-based retrieval methods as they exist at the moment but to complement them with visual search tools.

Content-based retrieval in picture archiving and communication systems

A COntent-Based Retrieval Architecture (COBRA) for picture archiving and communication systems (PACS) is introduced. COBRA improves the diagnosis, research, and training capabilities of PACS systems by adding retrieval by content features to those systems. COBRA is an open architecture based on widely used health care and technology standards. In addition to regular PACS components, COBRA includes additional components to handle representation, storage, and content-based similarity retrieval. Within COBRA, an anatomy classification algorithm is introduced to automatically classify PACS studies based on their anatomy. Such a classification allows the use of different segmentation and image-processing algorithms for different anatomies. COBRA uses primitive retrieval criteria such as color, texture, shape, and more complex criteria including object-based spatial relations and regions of interest. A prototype content-based retrieval system for MR brain images was developed to illustrate the concepts introduced in COBRA.