Spikiness and animacy as potential organizing principles of human ventral visual cortex

Considerable research has been devoted to understanding the fundamental organizing principles of the ventral visual pathway. A recent study revealed a series of 3-4 topographical maps arranged along the macaque inferotemporal (IT) cortex. The maps articulated a two-dimensional space based on the spikiness and animacy of visual objects, with "inanimate-spiky" and "inanimate-stubby" regions of the maps constituting two previously unidentified cortical networks. The goal of our study was to determine whether a similar functional organization might exist in human IT. To address this question, we presented the same object stimuli and images from "classic" object categories (bodies, faces, houses) to humans while recording fMRI activity at 7 Tesla. Contrasts designed to reveal the spikiness-animacy object space evoked extensive significant activation across human IT. However, unlike the macaque, we did not observe a clear sequence of complete maps, and selectivity for the spikiness-animacy space was deeply and mutually entangled with category-selectivity. Instead, we observed multiple new stimulus preferences in category-selective regions, including functional sub-structure related to object spikiness in scene-selective cortex. Taken together, these findings highlight spikiness as a promising organizing principle of human IT and provide new insights into the role of category-selective regions in visual object processing.

Binocular Rivalry Impact on Macroblock-Loss Error Concealment for Stereoscopic 3D Video Transmission

Three-dimensional video services delivered through wireless communication channels have to deal with numerous challenges due to the limitations of both the transmission channel's bandwidth and receiving devices. Adverse channel conditions, delays, or jitters can result in bit errors and packet losses, which can alter the appearance of stereoscopic 3D (S3D) video. Due to the perception of dissimilar patterns by the two human eyes, they can not be fused into a stable composite pattern in the brain and hence try to dominate by suppressing each other. Thus, a psychovisual sensation that is called binocular rivalry occurs. As a result, undetectable changes causing irritating flickering effects are seen, leading to visual discomforts such as eye strain, headache, nausea, and weariness. This study addresses the observer's quality of experience (QoE) by analyzing the binocular rivalry impact on the macroblock (MB) losses in a frame and its error propagation due to predictive frame encoding in stereoscopic video transmission systems. To simulate the processing of experimental videos, the Joint Test Model (JM) reference software has been used as it is recommended by the International Telecommunication Union (ITU). Existing error concealing techniques were then applied to the contiguous lost MBs for a variety of transmission impairments. In order to validate the authenticity of the simulated packet loss environment, several objective evaluations were carried out. Standard numbers of subjects were then engaged in the subjective testing of common 3D video sequences. The results were then statistically examined using a standard Student's t-test, allowing the impact of binocular rivalry to be compared to that of a non-rivalry error condition. The major goal is to assure error-free video communication by minimizing the negative impacts of binocular rivalry and boosting the ability to efficiently integrate 3D video material to improve viewers' overall QoE.

Subjective and objective quality assessment of gastrointestinal endoscopy images: From manual operation to artificial intelligence

Gastrointestinal endoscopy has been identified as an important tool for cancer diagnosis and therapy, particularly for treating patients with early gastric cancer (EGC). It is well known that the quality of gastroscope images is a prerequisite for achieving a high detection rate of gastrointestinal lesions. Owing to manual operation of gastroscope detection, in practice, it possibly introduces motion blur and produces low-quality gastroscope images during the imaging process. Hence, the quality assessment of gastroscope images is the key process in the detection of gastrointestinal endoscopy. In this study, we first present a novel gastroscope image motion blur (GIMB) database that includes 1,050 images generated by imposing 15 distortion levels of motion blur on 70 lossless images and the associated subjective scores produced with the manual operation of 15 viewers. Then, we design a new artificial intelligence (AI)-based gastroscope image quality evaluator (GIQE) that leverages the newly proposed semi-full combination subspace to learn multiple kinds of human visual system (HVS) inspired features for providing objective quality scores. The results of experiments conducted on the GIMB database confirm that the proposed GIQE showed more effective performance compared with its state-of-the-art peers.

MAGNet: A Camouflaged Object Detection Network Simulating the Observation Effect of a Magnifier

In recent years, protecting important objects by simulating animal camouflage has been widely employed in many fields. Therefore, camouflaged object detection (COD) technology has emerged. COD is more difficult to achieve than traditional object detection techniques due to the high degree of fusion of objects camouflaged with the background. In this paper, we strive to more accurately and efficiently identify camouflaged objects. Inspired by the use of magnifiers to search for hidden objects in pictures, we propose a COD network that simulates the observation effect of a magnifier called the MAGnifier Network (MAGNet). Specifically, our MAGNet contains two parallel modules: the ergodic magnification module (EMM) and the attention focus module (AFM). The EMM is designed to mimic the process of a magnifier enlarging an image, and AFM is used to simulate the observation process in which human attention is highly focused on a particular region. The two sets of output camouflaged object maps were merged to simulate the observation of an object by a magnifier. In addition, a weighted key point area perception loss function, which is more applicable to COD, was designed based on two modules to give greater attention to the camouflaged object. Extensive experiments demonstrate that compared with 19 cutting-edge detection models, MAGNet can achieve the best comprehensive effect on eight evaluation metrics in the public COD dataset. Additionally, compared to other COD methods, MAGNet has lower computational complexity and faster segmentation. We also validated the model's generalization ability on a military camouflaged object dataset constructed in-house. Finally, we experimentally explored some extended applications of COD.

Dynamic Range of Luminance Perception in Acute Hypobaric Hypoxia

Rizzi, Alessandro, Alice Plutino, Diego Vecchi, Anton Giulio Guadagno, and Marco Lucertini. Dynamic range of luminance perception in acute hypobaric hypoxia. High Alt Med Biol. 23:313-318, 2022.-The effects of acute hypobaric hypoxia, as potentially experienced in high altitude flight, on the detection of low contrast targets within a high dynamic range (DR) of luminance were investigated. The tests were performed after 10 minutes of adaptation to an ambient luminance level of 0.2 lx. Twenty-four individuals were exposed to hypobaric hypoxia at an altitude of 18,000 ft (5,490 m) inside a hypobaric chamber and were asked to identify the darkest one perceived from a grid of progressively less luminous patches. The results were compared with those obtained by the same subjects at sea level. The results indicate a global reduction of the DR perception during hypoxia, with a mean decrease of 0.023 cd/m² (standard deviation ±0.064), which resulted statistically significant (p < 0.05). No significant correlations with oxygen saturation levels were observed. These findings might have practical applications in the design of cockpit's instruments, especially those dedicated to night-time flight missions at high altitudes.

No-reference image quality assessment for confocal endoscopy images with perceptual local descriptor

SIGNIFICANCE

Confocal endoscopy images often suffer distortions, resulting in image quality degradation and information loss, increasing the difficulty of diagnosis and even leading to misdiagnosis. It is important to assess image quality and filter images with low diagnostic value before diagnosis.

AIM

We propose a no-reference image quality assessment (IQA) method for confocal endoscopy images based on Weber's law and local descriptors. The proposed method can detect the severity of image degradation by capturing the perceptual structure of an image.

APPROACH

We created a new dataset of 642 confocal endoscopy images to validate the performance of the proposed method. We then conducted extensive experiments to compare the accuracy and speed of the proposed method with other state-of-the-art IQA methods.

RESULTS

Experimental results demonstrate that the proposed method achieved an SROCC of 0.85 and outperformed other IQA methods.

CONCLUSIONS

Given its high consistency in subjective quality assessment, the proposed method can screen high-quality images in practical applications and contribute to diagnosis.

Learning to Predict Page View on College Official Accounts With Quality-Aware Features

At present, most of departments in colleges have their own official accounts, which have become the primary channel for announcements and news. In the official accounts, the popularity of articles is influenced by many different factors, such as the content of articles, the aesthetics of the layout, and so on. This paper mainly studies how to learn a computational model for predicting page view on college official accounts with quality-aware features extracted from pictures. First, we built a new picture database by collecting 1,000 pictures from the official accounts of nine well-known universities in the city of Beijing. Then, we proposed a new model for predicting page view by using a selective ensemble technology to fuse three sets of quality-aware features that could represent how a picture looks. Experimental results show that the proposed model has achieved competitive performance against state-of-the-art relevant models on the task for inferring page view from pictures on college official accounts.

A New Photographic Reproduction Method Based on Feature Fusion and Virtual Combined Histogram Equalization

A desirable photographic reproduction method should have the ability to compress high-dynamic-range images to low-dynamic-range displays that faithfully preserve all visual information. However, during the compression process, most reproduction methods face challenges in striking a balance between maintaining global contrast and retaining majority of local details in a real-world scene. To address this problem, this study proposes a new photographic reproduction method that can smoothly take global and local features into account. First, a highlight/shadow region detection scheme is used to obtain prior information to generate a weight map. Second, a mutually hybrid histogram analysis is performed to extract global/local features in parallel. Third, we propose a feature fusion scheme to construct the virtual combined histogram, which is achieved by adaptively fusing global/local features through the use of Gaussian mixtures according to the weight map. Finally, the virtual combined histogram is used to formulate the pixel-wise mapping function. As both global and local features are simultaneously considered, the output image has a natural and visually pleasing appearance. The experimental results demonstrated the effectiveness of the proposed method and the superiority over other seven state-of-the-art methods.

Photographic Reproduction and Enhancement Using HVS-Based Modified Histogram Equalization

Photographic reproduction and enhancement is challenging because it requires the preservation of all the visual information during the compression of the dynamic range of the input image. This paper presents a cascaded-architecture-type reproduction method that can simultaneously enhance local details and retain the naturalness of original global contrast. In the pre-processing stage, in addition to using a multiscale detail injection scheme to enhance the local details, the Stevens effect is considered for adapting different luminance levels and normally compressing the global feature. We propose a modified histogram equalization method in the reproduction stage, where individual histogram bin widths are first adjusted according to the property of overall image content. In addition, the human visual system (HVS) is considered so that a luminance-aware threshold can be used to control the maximum permissible width of each bin. Then, the global tone is modified by performing histogram equalization on the output modified histogram. Experimental results indicate that the proposed method can outperform the five state-of-the-art methods in terms of visual comparisons and several objective image quality evaluations.

An ecologically motivated image dataset for deep learning yields better models of human vision

Deep neural networks provide the current best models of visual information processing in the primate brain. Drawing on work from computer vision, the most commonly used networks are pretrained on data from the ImageNet Large Scale Visual Recognition Challenge. This dataset comprises images from 1,000 categories, selected to provide a challenging testbed for automated visual object recognition systems. Moving beyond this common practice, we here introduce ecoset, a collection of >1.5 million images from 565 basic-level categories selected to better capture the distribution of objects relevant to humans. Ecoset categories were chosen to be both frequent in linguistic usage and concrete, thereby mirroring important physical objects in the world. We test the effects of training on this ecologically more valid dataset using multiple instances of two neural network architectures: AlexNet and vNet, a novel architecture designed to mimic the progressive increase in receptive field sizes along the human ventral stream. We show that training on ecoset leads to significant improvements in predicting representations in human higher-level visual cortex and perceptual judgments, surpassing the previous state of the art. Significant and highly consistent benefits are demonstrated for both architectures on two separate functional magnetic resonance imaging (fMRI) datasets and behavioral data, jointly covering responses to 1,292 visual stimuli from a wide variety of object categories. These results suggest that computational visual neuroscience may take better advantage of the deep learning framework by using image sets that reflect the human perceptual and cognitive experience. Ecoset and trained network models are openly available to the research community.

Entropy Based Data Expansion Method for Blind Image Quality Assessment

Image quality assessment (IQA) is a fundamental technology for image applications that can help correct low-quality images during the capture process. The ability to expand distorted images and create human visual system (HVS)-aware labels for training is the key to performing IQA tasks using deep neural networks (DNNs), and image quality is highly sensitive to changes in entropy. Therefore, a new data expansion method based on entropy and guided by saliency and distortion is proposed in this paper. We introduce saliency into a large-scale expansion strategy for the first time. We regionally add distortion to a set of original images to obtain a distorted image database and label the distorted images using entropy. The careful design of the distorted images and the entropy-based labels fully reflects the influences of both saliency and distortion on quality. The expanded database plays an important role in the application of a DNN for IQA. Experimental results on IQA databases demonstrate the effectiveness of the expansion method, and the network’s prediction effect on the IQA databases is found to be improved compared with its predecessor algorithm. Therefore, we conclude that a data expansion approach that fully reflects HVS-aware quality factors is beneficial for IQA. This study presents a novel method for incorporating saliency into IQA, namely, representing it as regional distortion.

Figure-Ground Modulation in the Human Lateral Geniculate Nucleus Is Distinguishable from Top-Down Attention

Nearly all of the information that reaches the primary visual cortex (V1) of the brain passes from the retina through the lateral geniculate nucleus (LGN) of the thalamus. Although the LGN's role in relaying feedforward signals from the retina to the cortex is well understood [1, 2], the functional role of the extensive feedback it receives from the cortex has remained elusive [3-6]. Here, we investigated whether corticothalamic feedback may contribute to perceptual processing in the LGN in a manner that is distinct from top-down effects of attention [7-10]. We used high-resolution fMRI at 7 Tesla to simultaneously measure responses to orientation-defined figures in the human LGN and V1. We found robust enhancement of perceptual figures throughout the early visual system, which could be distinguished from the effects of covert spatial attention [11-13]. In a second experiment, we demonstrated that figure enhancement occurred in the LGN even when the figure and surrounding background were presented dichoptically (i.e., to different eyes). As binocular integration primarily occurs in V1 [14, 15], these results implicate a mechanism of automatic, contextually sensitive feedback from binocular visual cortex underlying figure-ground modulation in the LGN. Our findings elucidate the functional mechanisms of this core function of the visual system [16-18], which allows people to segment and detect meaningful figures in complex visual environments. The involvement of the LGN in this rich, contextually informed visual processing-despite showing minimal feedforward selectivity for visual features [19, 20]-underscores the role of recurrent processing at the earliest stages of visual processing.

Local Parallel Cross Pattern: A Color Texture Descriptor for Image Retrieval

Riding the wave of visual sensor equipment (e.g., personal smartphones, home security cameras, vehicle cameras, and camcorders), image retrieval (IR) technology has received increasing attention due to its potential applications in e-commerce, visual surveillance, and intelligent traffic. However, determining how to design an effective feature descriptor has been proven to be the main bottleneck for retrieving a set of images of interest. In this paper, we first construct a six-layer color quantizer to extract a color map. Then, motivated by the human visual system, we design a local parallel cross pattern (LPCP) in which the local binary pattern (LBP) map is amalgamated with the color map in “parallel” and “cross” manners. Finally, to reduce the computational complexity and improve the robustness to image rotation, the LPCP is extended to the uniform local parallel cross pattern (ULPCP) and the rotation-invariant local parallel cross pattern (RILPCP), respectively. Extensive experiments are performed on eight benchmark datasets. The experimental results validate the effectiveness, efficiency, robustness, and computational complexity of the proposed descriptors against eight state-of-the-art color texture descriptors to produce an in-depth comparison. Additionally, compared with a series of Convolutional Neural Network (CNN)-based models, the proposed descriptors still achieve competitive results.

Sex differences in the human visual system

This Mini-Review summarizes a wide range of sex differences in the human visual system, with a primary focus on sex differences in visual perception and its neural basis. We highlight sex differences in both basic and high-level visual processing, with evidence from behavioral, neurophysiological, and neuroimaging studies. We argue that sex differences in human visual processing, no matter how small or subtle, support the view that females and males truly see the world differently. We acknowledge some of the controversy regarding sex differences in human vision and propose that such controversy should be interpreted as a source of motivation for continued efforts to assess the validity and reliability of published sex differences and for continued research on sex differences in human vision and the nervous system in general. © 2016 Wiley Periodicals, Inc.

Medical image watermarking using bit threshold map based on just noticeable distortion in discrete cosine transform

BACKGROUND

Medical images stored in a hospital system are generally confidential and integrated and require strict security. However, medical information stored on digital medical imaging systems, as well as picture archiving and communication systems (PACS), are vulnerable to attack when the data are transferred over wireless or wired communication networks.

OBJECTIVE

To solve this problem, a watermarking algorithm for medical images is proposed using a bit threshold map based on just noticeable distortion (JND) in the discrete cosine transform (DCT) method.

METHODS

The low-frequency component comprises a considerable amount of the signal energy for most images. As a result, it has a crucial effect on the image quality. Therefore, in this paper, the proposed algorithm embeds watermarks based on the low-frequency components of the image, such as the DC coefficient of the DCT.

RESULTS

When watermarks are embedded in a low frequency area, the subjective image quality is often degraded. To compensate for the degradation of the imperceptibility of the watermarking system, which results from embedding watermarks in the low-frequency component, this research considers the human visual system. In addition, the embedding strength of the JND value is used to improve the watermarking imperceptibility.

CONCLUSIONS

We applied the proposed watermarking algorithm to a variety of medical images using a computer simulation. The algorithm's performance was verified using a variety of attacks for eliminating watermarks. The simulation results show that the proposed algorithm robustly provides protection against a variety of possible attacks.

Automatic intensity windowing of mammographic images based on a perceptual metric

PURPOSE

Initial auto-adjustment of the window level WL and width WW applied to mammographic images. The proposed intensity windowing (IW) method is based on the maximization of the mutual information (MI) between a perceptual decomposition of the original 12-bit sources and their screen displayed 8-bit version. Besides zoom, color inversion and panning operations, IW is the most commonly performed task in daily screening and has a direct impact on diagnosis and the time involved in the process.

METHODS

The authors present a human visual system and perception-based algorithm named GRAIL (Gabor-relying adjustment of image levels). GRAIL initially measures a mammogram's quality based on the MI between the original instance and its Gabor-filtered derivations. From this point on, the algorithm performs an automatic intensity windowing process that outputs the WL/WW that best displays each mammogram for screening. GRAIL starts with the default, high contrast, wide dynamic range 12-bit data, and then maximizes the graphical information presented in ordinary 8-bit displays. Tests have been carried out with several mammogram databases. They comprise correlations and an ANOVA analysis with the manual IW levels established by a group of radiologists. A complete MATLAB implementation of GRAIL is available at https://github.com/TheAnswerIsFortyTwo/GRAIL.

RESULTS

Auto-leveled images show superior quality both perceptually and objectively compared to their full intensity range and compared to the application of other common methods like global contrast stretching (GCS). The correlations between the human determined intensity values and the ones estimated by our method surpass that of GCS. The ANOVA analysis with the upper intensity thresholds also reveals a similar outcome. GRAIL has also proven to specially perform better with images that contain micro-calcifications and/or foreign X-ray-opaque elements and with healthy BI-RADS A-type mammograms. It can also speed up the initial screening time by a mean of 4.5 s per image.

CONCLUSIONS

A novel methodology is introduced that enables a quality-driven balancing of the WL/WW of mammographic images. This correction seeks the representation that maximizes the amount of graphical information contained in each image. The presented technique can contribute to the diagnosis and the overall efficiency of the breast screening session by suggesting, at the beginning, an optimal and customized windowing setting for each mammogram.

The effect of ambient illumination on handheld display image quality

Handheld devices such as smartphones and tablets are becoming useful in the medical field, as they allow physicians, radiologists, and researchers to analyze images with the benefit of mobile accessibility. However, for handheld devices to be effective, the display must be able to perform well in a wide range of ambient illumination conditions. We conducted visual experiments to quantify user performance for testing the image quality of two current-generation devices in different ambient illumination conditions while measuring ambient light levels with a real-time illuminance meter. We found and quantified that due to the high reflectivity of handheld devices, performance deteriorates as the user moves from dark areas into environments of greater ambient illumination. The quantitative analysis suggests that differences in display reflection coefficients do not affect the low illumination performance of the device but rather the performance at higher levels of illumination.

Independent sources of spontaneous BOLD fluctuation along the visual pathway

In resting-state functional magnetic resonance imaging (fMRI) experiments, correlation analysis can be used to identify clusters of cortical regions that may be functionally connected. Although such functional connectivity is often assumed to reflect cortico-cortical connections, a potential confound is the contribution of subcortical brain regions, many of which have strong anatomical connectivity to cortical regions and may also enable cortico-cortical interactions through trans-thalamic pathways. To investigate this, we performed resting state fMRI of the human visual system, including cortical regions and subcortical nuclei of the pulvinar and lateral geniculate. Regression analysis was used to investigate the dependence of the measured inter-regional correlations upon afferents from specific retinal, thalamic and cortical regions as well as systemic global signal fluctuation. A high level of inter-hemispheric correlation (cc = 0.95) was found in the visual cortex that could not be explained by activity in the subcortical nuclei investigated; in addition a relatively low level of inter-hemispheric correlation (cc = 0.39-0.42) was found in vision-related thalamic nuclei that could not be explained by direct anatomical connections or their cortical inputs. These findings suggest that spontaneous fMRI signal correlations within the human visual system originate from a mixture of independent signal sources that may be transmitted through thalamo-cortical, cortico-thalamic, and cortico-cortical connections either trans-callosal or trans-thalamic in origin. Our findings thus call for more cautious interpretation of resting state functional connectivity in terms of any single type of anatomical connectivity.

Sensor and display human factors based design constraints for head mounted and tele-operation systems

For mobile imaging systems in head mounted displays and tele-operation systems it is important to maximize the amount of visual information transmitted to the human visual system without exceeding its input capacity. This paper aims to describe the design constraints on the imager and display systems of head mounted devices and tele-operated systems based upon the capabilities of the human visual system. We also present the experimental results of methods to improve the amount of visual information conveyed to a user when trying to display a high dynamic range image on a low dynamic range display.

Why and how is soft copy reading possible in clinical practice?

The properties of the human visual system (HVS) relevant to the diagnostic process are described after a brief introduction on the general problems and advantages of using soft copy for primary radiology interpretations. At various spatial and temporal frequencies the contrast sensitivity defines the spatial resolution of the eye-brain system and the sensitivity to flicker. The adaptation to the displayed radiological scene and the ambient illumination determine the dynamic range for the operation of the HVS. Although image display devices are determined mainly by state-of-the-art technology, analysis of the HVS may suggest technical characteristics for electronic displays that will help to optimize the display to the operation of the HVS. These include display size, spatial resolution, contrast resolution, luminance range, and noise, from which further consequences for the technical components of a monitor follow. It is emphasized that routine monitor quality control must be available in clinical practice. These image quality measures must be simple enough to be applied as part of the daily routine. These test instructions might also serve as elements of technical acceptance and constancy tests.