Photobook: Content-based manipulation of image databases

Neural network feature learning based on image self-encoding

With the rapid development of information technology and the arrival of the era of big data, people’s access to information is increasingly relying on information such as images. Today, image data are showing an increasing trend in the form of an index. How to use deep learning models to extract valuable information from massive data is very important. In the face of such a situation, people cannot accurately and timely find out the information they need. Therefore, the research on image retrieval technology is very important. Image retrieval is an important technology in the field of computer vision image processing. It realizes fast and accurate query of similar images in image database. The excellent feature representation not only can represent the category information of the image but also capture the relevant semantic information of the image. If the neural network feature learning expression is combined with the image retrieval field, it will definitely improve the application of image retrieval technology. To solve the above problems, this article studies the problems encountered in deep learning neural network feature learning based on image self-encoding and discusses its feature expression in the field of image retrieval. By adding the spatial relationship information obtained by image self-encoding in the neural network training process, the feature expression ability of the selected neural network is improved, and the neural network feature learning based on image coding is successfully applied to the popular field of image retrieval.

ML2S-SVM: multi-label least-squares support vector machine classifiers

Purpose

Image classification is becoming a supporting technology in several image-processing tasks. Due to rich semantic information contained in the images, it is very popular for an image to have several labels or tags. This paper aims to develop a novel multi-label classification approach with superior performance.

Design/methodology/approach

Many multi-label classification problems share two main characteristics: label correlations and label imbalance. However, most of current methods are devoted to either model label relationship or to only deal with unbalanced problem with traditional single-label methods. In this paper, multi-label classification problem is regarded as an unbalanced multi-task learning problem. Multi-task least-squares support vector machine (MTLS-SVM) is generalized for this problem, renamed as multi-label LS-SVM (ML2S-SVM).

Findings

Experimental results on the emotions, scene, yeast and bibtex data sets indicate that the ML2S-SVM is competitive with respect to the state-of-the-art methods in terms of Hamming loss and instance-based F1 score. The values of resulting parameters largely influence the performance of ML2S-SVM, so it is necessary for users to identify proper parameters in advance.

Originality/value

On the basis of MTLS-SVM, a novel multi-label classification approach, ML2S-SVM, is put forward. This method can overcome the unbalanced problem but also explicitly models arbitrary order correlations among labels by allowing multiple labels to share a subspace. In addition, the multi-label classification approach has a wider range of applications. That is to say, it is not limited to the field of image classification.

Robust Adaptive Median Binary Pattern for Noisy Texture Classification and Retrieval

Texture is an important characteristic for different computer vision tasks and applications. Local binary pattern (LBP) is considered one of the most efficient texture descriptors yet. However, LBP has some notable limitations, in particular its sensitivity to noise. In this paper, we address these criteria by introducing a novel texture descriptor, robust adaptive median binary pattern (RAMBP). RAMBP is based on a process involving classification of noisy pixels, adaptive analysis window, scale analysis, and a comparison of image medians. The proposed method handles images with highly noisy textures and increases the discriminative properties by capturing microstructure and macrostructure texture information. The method was evaluated on popular texture datasets for classification and retrieval tasks and under different high noise conditions. Without any training or prior knowledge of the noise type, RAMBP achieved the best classification compared to state-of-the-art techniques. It scored more than 90% under 50% impulse noise densities, more than 95% under Gaussian noised textures with a standard deviation σ = 5 , more than 99% under Gaussian blurred textures with a standard deviation σ = 1.25 , and more than 90% for mixed noise. The proposed method yielded competitive results and proved to be one of the best descriptors in noise-free texture classification. Furthermore, RAMBP showed high performance for the problem of noisy texture retrieval providing high scores of recall and precision measures for textures with high noise levels. Finally, compared with the state-of-the-art methods, RAMBP achieves a good running time with low feature dimensionality.

A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine

The advancements in the multimedia technologies result in the growth of the image databases. To retrieve images from such image databases using visual attributes of the images is a challenging task due to the close visual appearance among the visual attributes of these images, which also introduces the issue of the semantic gap. In this article, we recommend a novel method established on the bag-of-words (BoW) model, which perform visual words integration of the local intensity order pattern (LIOP) feature and local binary pattern variance (LBPV) feature to reduce the issue of the semantic gap and enhance the performance of the content-based image retrieval (CBIR). The recommended method uses LIOP and LBPV features to build two smaller size visual vocabularies (one from each feature), which are integrated together to build a larger size of the visual vocabulary, which also contains complementary features of both descriptors. Because for efficient CBIR, the smaller size of the visual vocabulary improves the recall, while the bigger size of the visual vocabulary improves the precision or accuracy of the CBIR. The comparative analysis of the recommended method is performed on three image databases, namely, WANG-1K, WANG-1.5K and Holidays. The experimental analysis of the recommended method on these image databases proves its robust performance as compared with the recent CBIR methods.

Generic and Specific Impressions Estimation and Their Application to KANSEI-Based Clothing Fabric Image Retrieval

Current image retrieval techniques are mainly based on text or visual contents. However, both text-based and contents-based methods lack the capability of utilizing human intuition and KANSEI (impression). In this paper, we proposed an impression-based image retrieval method in order to realize the image retrieval according to our impression presented by impression keywords. We first propose a generic and specific impressions estimation method based on machine learning and then apply it to impression-based clothing fabric image retrieval. We use a semantic differential (SD) method to measure the user’s impressions such as brightness and warmth while they view a cloth fabric image. We also extract both global and local features of cloth fabric images such as color and texture using computer vision techniques. Then we use support vector regression to model the mapping functions between the generic impression (or specific impression) and image features. The learnt mapping functions are used to estimate the generic and specific impressions of cloth fabric images. The retrieval is done by comparing the query impression with the estimated impression of images in the database.

Building and Querying RDF/OWL Database of Semantically Annotated Nuclear Medicine Images

As the use of positron emission tomography-computed tomography (PET-CT) has increased rapidly, there is a need to retrieve relevant medical images that can assist image interpretation. However, the images themselves lack the explicit information needed for query. We constructed a semantically structured database of nuclear medicine images using the Annotation and Image Markup (AIM) format and evaluated the ability the AIM annotations to improve image search. We created AIM annotation templates specific to the nuclear medicine domain and used them to annotate 100 nuclear medicine PET-CT studies in AIM format using controlled vocabulary. We evaluated image retrieval from 20 specific clinical queries. As the gold standard, two nuclear medicine physicians manually retrieved the relevant images from the image database using free text search of radiology reports for the same queries. We compared query results with the manually retrieved results obtained by the physicians. The query performance indicated a 98 % recall for simple queries and a 89 % recall for complex queries. In total, the queries provided 95 % (75 of 79 images) recall, 100 % precision, and an F1 score of 0.97 for the 20 clinical queries. Three of the four images missed by the queries required reasoning for successful retrieval. Nuclear medicine images augmented using semantic annotations in AIM enabled high recall and precision for simple queries, helping physicians to retrieve the relevant images. Further study using a larger data set and the implementation of an inference engine may improve query results for more complex queries.

A faceted approach to reachability analysis of graph modelled collections

Nowadays, there is a proliferation of available information sources from different modalities-text, images, audio, video and more. Information objects are not isolated anymore. They are frequently connected via metadata, semantic links, etc. This leads to various challenges in graph-based information retrieval. This paper is concerned with the reachability analysis of multimodal graph modelled collections. We use our framework to leverage the combination of features of different modalities through our formulation of faceted search. This study highlights the effect of different facets and link types in improving reachability of relevant information objects. The experiments are performed on the Image CLEF 2011 Wikipedia collection with about 400,000 documents and images. The results demonstrate that the combination of different facets is conductive to obtain higher reachability. We obtain 373% recall gain for very hard topics by using our graph model of the collection. Further, by adding semantic links to the collection, we gain a 10% increase in the overall recall.

Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking

How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method.

A Survey of Multimedia Recommender Systems: Challenges and Opportunities

Multimedia information has been extensively growing from a variety of sources such as cameras or video recorders. In order to select the useful multimedia objects, multimedia recommender system has been emerging as a tool to help users choose which multimedia objects might be interesting for them. However, given the complexity of multimedia objects, it is challenging to provide effective multimedia recommendations. In this paper, we therefore conduct a survey in both the multimedia information system and recommender system communities. We further focus on the works that span the two communities, especially the research on multimedia recommender systems. Based on our review, we propose a set of research challenges, which can be used to implicate the future research directions for multimedia recommender systems. For each research challenge, we have also provided the insights of how to perform the follow-up research.

Retrieval of Multimedia Information Using Content-Based Image Retrieval (CBIR) Techniques

This chapter will focus on the concept of Content-based image retrieval. Searching of an image or video database based on text based description is a manual labor intensive process. Descriptions of the file are usually typed manually for each image by human operators because the automatic generation of keywords for the images is difficult without incorporation of visual information and feature extraction. This method is impractical in today's multimedia information era. “Content-based” means that the search will analyze the actual contents of the image rather than the metadata such as keywords, tags, and descriptions associated with the image. The term “content” in this context might refer to colors, shapes, textures, or any other information that can be derived from the image itself. Several important sections are highlighted in this chapter, like architectures, query techniques, multidimensional indexing, video retrieval and different application sections of CBIR.

Visual image retrieval: seeking the alliance of concept-based and content-based paradigms

In the commercial use of picture collections, a heavy dependency continues to be exhibited on a concept-based image retrieval paradigm in which the query is verbalised by the client and resolved as a metadata text-matching operation.

The practical and philosophical challenges posed by the indexing aspect of image metadata construction are significant and frequently expressed. Nevertheless, it has taken image digitisation to bring this particular information retrieval problem to prominence in the research agenda. Metamorphosed into a binary data structure, the digital image offers some enticing processing opportunities which content-based image retrieval techniques are exploiting with developing success.

Drawing on studies of user need, this paper seeks to explain why a heavy dependency will continue to be placed on concept-based rather than content-based image retrieval techniques within archival image collections. In contrast, the promising nature of content-based techniques from the viewpoint of a growing clientele with less traditional visual information needs will also be considered.

The paper concludes by offering the view that, while both concept-based and content-based approaches suffer from operational limitations, the further development of a hybrid image retrieval paradigm which combines the two approaches makes a potentially valuable contribution to the research agenda for visual image retrieval.

Scalable analysis of Big pathology image data cohorts using efficient methods and high-performance computing strategies

Background

We describe a suite of tools and methods that form a core set of capabilities for researchers and clinical investigators to evaluate multiple analytical pipelines and quantify sensitivity and variability of the results while conducting large-scale studies in investigative pathology and oncology. The overarching objective of the current investigation is to address the challenges of large data sizes and high computational demands.

Results

The proposed tools and methods take advantage of state-of-the-art parallel machines and efficient content-based image searching strategies. The content based image retrieval (CBIR) algorithms can quickly detect and retrieve image patches similar to a query patch using a hierarchical analysis approach. The analysis component based on high performance computing can carry out consensus clustering on 500,000 data points using a large shared memory system.

Conclusions

Our work demonstrates efficient CBIR algorithms and high performance computing can be leveraged for efficient analysis of large microscopy images to meet the challenges of clinically salient applications in pathology. These technologies enable researchers and clinical investigators to make more effective use of the rich informational content contained within digitized microscopy specimens.

Content-based histopathology image retrieval using CometCloud

Background

The development of digital imaging technology is creating extraordinary levels of accuracy that provide support for improved reliability in different aspects of the image analysis, such as content-based image retrieval, image segmentation, and classification. This has dramatically increased the volume and rate at which data are generated. Together these facts make querying and sharing non-trivial and render centralized solutions unfeasible. Moreover, in many cases this data is often distributed and must be shared across multiple institutions requiring decentralized solutions. In this context, a new generation of data/information driven applications must be developed to take advantage of the national advanced cyber-infrastructure (ACI) which enable investigators to seamlessly and securely interact with information/data which is distributed across geographically disparate resources. This paper presents the development and evaluation of a novel content-based image retrieval (CBIR) framework. The methods were tested extensively using both peripheral blood smears and renal glomeruli specimens. The datasets and performance were evaluated by two pathologists to determine the concordance.

Results

The CBIR algorithms that were developed can reliably retrieve the candidate image patches exhibiting intensity and morphological characteristics that are most similar to a given query image. The methods described in this paper are able to reliably discriminate among subtle staining differences and spatial pattern distributions. By integrating a newly developed dual-similarity relevance feedback module into the CBIR framework, the CBIR results were improved substantially. By aggregating the computational power of high performance computing (HPC) and cloud resources, we demonstrated that the method can be successfully executed in minutes on the Cloud compared to weeks using standard computers.

Conclusions

In this paper, we present a set of newly developed CBIR algorithms and validate them using two different pathology applications, which are regularly evaluated in the practice of pathology. Comparative experimental results demonstrate excellent performance throughout the course of a set of systematic studies. Additionally, we present and evaluate a framework to enable the execution of these algorithms across distributed resources. We show how parallel searching of content-wise similar images in the dataset significantly reduces the overall computational time to ensure the practical utility of the proposed CBIR algorithms.

Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data

Medical imaging is fundamental to modern healthcare, and its widespread use has resulted in the creation of image databases, as well as picture archiving and communication systems. These repositories now contain images from a diverse range of modalities, multidimensional (three-dimensional or time-varying) images, as well as co-aligned multimodality images. These image collections offer the opportunity for evidence-based diagnosis, teaching, and research; for these applications, there is a requirement for appropriate methods to search the collections for images that have characteristics similar to the case(s) of interest. Content-based image retrieval (CBIR) is an image search technique that complements the conventional text-based retrieval of images by using visual features, such as color, texture, and shape, as search criteria. Medical CBIR is an established field of study that is beginning to realize promise when applied to multidimensional and multimodality medical data. In this paper, we present a review of state-of-the-art medical CBIR approaches in five main categories: two-dimensional image retrieval, retrieval of images with three or more dimensions, the use of nonimage data to enhance the retrieval, multimodality image retrieval, and retrieval from diverse datasets. We use these categories as a framework for discussing the state of the art, focusing on the characteristics and modalities of the information used during medical image retrieval.

Medical image retrieval: past and present

With the widespread dissemination of picture archiving and communication systems (PACSs) in hospitals, the amount of imaging data is rapidly increasing. Effective image retrieval systems are required to manage these complex and large image databases. The authors reviewed the past development and the present state of medical image retrieval systems including text-based and content-based systems. In order to provide a more effective image retrieval service, the intelligent content-based retrieval systems combined with semantic systems are required.

OVERVIEW AND FUTURE TRENDS OF MULTIMEDIA RESEARCH FOR CONTENT ACCESS AND DISTRIBUTION

The advances in information technology, computational capability, and communication networks have enabled large-scale data collection and distribution of vast amounts of multimedia data available to consumer and enterprise applications. With the proliferation of multimedia data and ever-growing requests for multimedia applications, reliable and efficient tools and techniques are urgently sought for multimedia content analysis and retrieval, as well as secure media streaming, distribution or communication. Though many research efforts have been devoted to these aspects, it is still far from maturity and there exist many open issues. In this paper, an overview of the challenges and issues as well as future trends on multimedia content access and distribution will be discussed. The focuses include: (i) how to bridge the gaps between the semantic meaning and low-level media characteristics; (ii) how to handle user perception subjectivity problem; (iii) how to provide multimedia network communication support for media streaming and P2P (peer-to-peer) media distribution; and (iv) how to ensure various aspects of secure media, including acquisition, processing, storage, and communication.

ACCURATE IMAGE RETRIEVAL BASED ON COMPACT COMPOSITE DESCRIPTORS AND RELEVANCE FEEDBACK INFORMATION

In this paper a new set of descriptors appropriate for image indexing and retrieval is proposed. The proposed descriptors address the tremendously increased need for efficient content-based image retrieval (CBIR) in many application areas such as the Internet, biomedicine, commerce and education. These applications commonly store image information in large image databases where the image information cannot be accessed or used unless the database is organized to allow efficient storage, browsing and retrieval. To be applicable in the design of large image databases, the proposed descriptors are compact, with the smallest requiring only 23 bytes per image. The proposed descriptors' structure combines color and texture information which are extracted using fuzzy approaches. To evaluate the performance of the proposed descriptors, the objective Average Normalized Modified Retrieval Rank (ANMRR) is used. Experiments conducted on five benchmarking image databases demonstrate the effectiveness of the proposed descriptors in outperforming other state-of-the-art descriptors. Also, a Auto Relevance Feedback (ARF) technique is introduced which is based on the proposed descriptors. This technique readjusts the initial retrieval results based on user preferences improving the retrieval score significantly. An online demo of the image retrieval system img(Anaktisi) that implements the proposed descriptors can be found at .

CONTENT-BASED IMAGE RETRIEVAL OF CULTURAL HERITAGE SYMBOLS BY INTERACTION OF VISUAL PERSPECTIVES

Content-based Image Retrieval (CBIR) has been an active area of research for retrieving similar images from large repositories, without the prerequisite of manual labeling. Most current CBIR algorithms can faithfully return a list of images that matches the visual perspective of their inventors, who might decide to use a certain combination of image features like edges, colors and textures of regions as well as their spatial distribution during processing. In practice, however, the retrieved images rarely correspond exactly to the results expected by the users, a problem that has come to be known as the semantic gap. In this paper, we propose a novel and extensible multidimensional approach called matrix of visual perspectives as a solution for addressing this semantic gap. Our approach exploits the dynamic cross-interaction (in other words, mix-and-match) of image features and similarity metrics to produce results that attempt to mimic the mental visual picture of the user. Experimental results on retrieving similar Japanese cultural heritage symbols called kamons by a prototype system confirm that the interaction of visual perspectives in the user can be effectively captured and reflected. The benefits of this approach are broader. They can be equally applicable to the development of CBIR systems for other types of images, whether cultural or noncultural, by adapting to different sets of application specific image features.

RETRIEVING IMAGES BY COMPARING HOMOGENEOUS COLOR AND TEXTURE OBJECTS IN THE IMAGE

An object-based image retrieval method is addressed in this paper. For that purpose, a new image segmentation algorithm and image comparing method between segmented objects are proposed. For image segmentation, color and textural features are extracted from each pixel in the image and these features are used as inputs into VQ (Vector Quantization) clustering method, which yields homogeneous objects in terms of color and texture. In this procedure, colors are quantized into a few dominant colors for simple representation and efficient retrieval. In the retrieval case, two comparing schemes are proposed. Comparisons between one query object and multi-objects of a database image and comparisons between multi-query objects and multi-objects of a database image are proposed. For fast retrieval, dominant object colors are key-indexed into the database.

Describable Visual Attributes for Face Verification and Image Search

We introduce the use of describable visual attributes for face verification and image search. Describable visual attributes are labels that can be given to an image to describe its appearance. This paper focuses on images of faces and the attributes used to describe them, although the concepts also apply to other domains. Examples of face attributes include gender, age, jaw shape, nose size, etc. The advantages of an attribute-based representation for vision tasks are manifold: They can be composed to create descriptions at various levels of specificity; they are generalizable, as they can be learned once and then applied to recognize new objects or categories without any further training; and they are efficient, possibly requiring exponentially fewer attributes (and training data) than explicitly naming each category. We show how one can create and label large data sets of real-world images to train classifiers which measure the presence, absence, or degree to which an attribute is expressed in images. These classifiers can then automatically label new images. We demonstrate the current effectiveness--and explore the future potential--of using attributes for face verification and image search via human and computational experiments. Finally, we introduce two new face data sets, named FaceTracer and PubFig, with labeled attributes and identities, respectively.

Biomedical imaging modality classification using combined visual features and textual terms

We describe an approach for the automatic modality classification in medical image retrieval task of the 2010 CLEF cross-language image retrieval campaign (ImageCLEF). This paper is focused on the process of feature extraction from medical images and fuses the different extracted visual features and textual feature for modality classification. To extract visual features from the images, we used histogram descriptor of edge, gray, or color intensity and block-based variation as global features and SIFT histogram as local feature. For textual feature of image representation, the binary histogram of some predefined vocabulary words from image captions is used. Then, we combine the different features using normalized kernel functions for SVM classification. Furthermore, for some easy misclassified modality pairs such as CT and MR or PET and NM modalities, a local classifier is used for distinguishing samples in the pair modality to improve performance. The proposed strategy is evaluated with the provided modality dataset by ImageCLEF 2010.

Content-Based Image Retrieval via Combination of Similarity Measures

A multiple (dis)similarity measure combination framework via normalization and weighting of measures is proposed to find suitable measure combinations in terms of retrieval accuracy and computational cost. In the combination of Manhattan and Hellinger distances, the computational time is more than 12 times faster and the retrieval accuracy improves or remains at the same level, when compared with Minkowski distance, a measure having the best retrieval accuracy in the single measure scenario. These performances are determined on a visual word based image retrieval system by using the Corel collections. Due to the reduction of computational cost and robustness of retrieval accuracy in this combination, applications include retrieval employing large number of images and categories in a database.

On Hierarchical Content-Based Image Retrieval by Dynamic Indexing and Guided Search

This paper presents a new approach to content-based image retrieval by using dynamic indexing and guided search in a hierarchical structure, and extending data mining and data warehousing techniques. The proposed algorithms include a wavelet-based scheme for multiple image feature extraction, the extension of a conventional data warehouse and an image database to an image data warehouse for dynamic image indexing. It also provides an image data schema for hierarchical image representation and dynamic image indexing, a statistically based feature selection scheme to achieve flexible similarity measures, and a feature component code to facilitate query processing and guide the search for the best matching. A series of case studies are reported, which include a wavelet-based image color hierarchy, classification of satellite images, tropical cyclone pattern recognition, and personal identification using multi-level palmprint and face features. Experimental results confirm that the new approach is feasible for content-based image retrieval.

IPSILON: incremental parsing for semantic indexing of latent concepts

A new framework for content-based image retrieval, which takes advantage of the source characterization property of a universal source coding scheme, is investigated. Based upon a new class of multidimensional incremental parsing algorithm, extended from the Lempel-Ziv incremental parsing code, the proposed method captures the occurrence pattern of visual elements from a given image. A linguistic processing technique, namely the latent semantic analysis (LSA) method, is then employed to identify associative ensembles of visual elements, which lay the foundation for intelligent visual information analysis. In 2-D applications, incremental parsing decomposes an image into elementary patches that are different from the conventional fixed square-block type patches. When used in compressive representations, it is amenable in schemes that do not rely on average distortion criteria, a methodology that is a departure from the conventional vector quantization. We call this methodology a parsed representation. In this article, we present our implementations of an image retrieval system, called IPSILON, with parsed representations induced by different perceptual distortion thresholds. We evaluate the effectiveness of the use of the parsed representations by comparing their performance with that of four image retrieval systems, one using the conventional vector quantization for visual information analysis under the same LSA paradigm, another using a method called SIMPLIcity which is based upon an image segmentation and integrated region matching, and the other two based upon query-by-semantic-example and query-by-visual-example. The first two of them were tested with 20,000 images of natural scenes, and the others were tested with a portion of the images. The experimental results show that the proposed parsed representation efficiently captures the salient features in visual images and the IPSILON systems outperform other systems in terms of retrieval precision and distortion robustness.

PathMiner: a Web-based tool for computer-assisted diagnostics in pathology

Large-scale, multisite collaboration has become indispensable for a wide range of research and clinical activities that rely on the capacity of individuals to dynamically acquire, share, and assess images and correlated data. In this paper, we report the development of a Web-based system, PathMiner , for interactive telemedicine, intelligent archiving, and automated decision support in pathology. The PathMiner system supports network-based submission of queries and can automatically locate and retrieve digitized pathology specimens along with correlated molecular studies of cases from "ground-truth" databases that exhibit spectral and spatial profiles consistent with a given query image. The statistically most probable diagnosis is provided to the individual who is seeking decision support. To test the system under real-case scenarios, a pipeline infrastructure was developed and a network-based test laboratory was established at strategic sites at the University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Medical School, Robert Wood Johnson University Hospital, the University of Pennsylvania School of Medicine, Hospital of the University of Pennsylvania, The Cancer Institute of New Jersey, and Rutgers University. The average five-class classification accuracy of the system was 93.18% based on a tenfold cross validation on a close dataset containing 3691 imaged specimens. We also conducted prospective performance studies with the PathMiner system in real applications in which the specimens exhibited large variations in staining characters compared with the training data. The average five-class classification accuracy in this open-set experiment was 87.22%. We also provide the comparative results with the previous literature and the PathMiner system shows superior performance.

VisualRank: applying PageRank to large-scale image search

Because of the relative ease in understanding and processing text, commercial image-search systems often rely on techniques that are largely indistinguishable from text-search. Recently, academic studies have demonstrated the effectiveness of employing image-based features to provide alternative or additional signals. However, it remains uncertain whether such techniques will generalize to a large number of popular web queries, and whether the potential improvement to search quality warrants the additional computational cost. In this work, we cast the image-ranking problem into the task of identifying "authority" nodes on an inferred visual similarity graph and propose VisualRank to analyze the visual link structures among images. The images found to be "authorities" are chosen as those that answer the image-queries well. To understand the performance of such an approach in a real system, we conducted a series of large-scale experiments based on the task of retrieving images for 2000 of the most popular products queries. Our experimental results show significant improvement, in terms of user satisfaction and relevancy, in comparison to the most recent Google Image Search results. Maintaining modest computational cost is vital to ensuring that this procedure can be used in practice; we describe the techniques required to make this system practical for large scale deployment in commercial search engines.

Geometry-based image retrieval in binary image databases

In this paper, a geometry-based image retrieval system is developed for multi-object images. We model both shape and topology of image objects using a structured representation called curvature tree (CT). The hierarchy of the CT reflects the inclusion relationships between the image objects. To facilitate shape-based matching, triangle-area representation (TAR) of each object is stored at the corresponding node in the CT. The similarity between two multi-object images is measured based on the maximum similarity subtree isomorphism (MSSI) between their CTs. For this purpose, we adopt a recursive algorithm to solve the MSSI problem and a very effective dynamic programming algorithm to measure the similarity between the attributed nodes. Our matching scheme agrees with many recent findings in psychology about the human perception of multi-object images. Experiments on a database of 13500 real and synthesized medical images and the MPEG-7 CE-1 database of 1400 shape images have shown the effectiveness of the proposed method.

Shape indexing using self-organizing maps

In this paper, we propose a novel approach to generate the topology-preserving mapping of structural shapes using self-organizing maps (SOMs). The structural information of the geometrical shapes is captured by relational attribute vectors. These vectors are quantised using an SOM. Using this SOM, a histogram is generated for every shape. These histograms are treated as inputs to train another SOM which yields a topology-preserving mapping of the geometric shapes. By appropriately choosing the relational vectors, it is possible to generate a mapping that is invariant to some chosen transformations, such as rotation, translation, scale, affine, or perspective transformations. Experimental results using trademark objects are presented to demonstrate the performance of the proposed methodology.