Channel and Spatial Attention Regression Network for Cup-to-Disc Ratio Estimation

Is Attention all You Need in Medical Image Analysis? A Review

Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. Despite their important advances, typical CNN have relatively limited capabilities in modelling "global" pixel interactions, which restricts their generalisation ability to understand out-of-distribution data with different "global" information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments ("Transf/Attention") which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced an analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated.

Cyanobacteria cell prediction using interpretable deep learning model with observed, numerical, and sensing data assemblage

Massive cyanobacterial blooms in river water causes adverse impacts on aquatic ecosystems and water quality. Complex and diverse data sources are available to investigate the cyanobacteria phenomena, including in situ data, synthetic measurements, and remote sensing images. Deep learning attention models can process these intricate sources to forecast cyanobacteria by identifying important variables in the data sources. However, deep learning attention models for predicting cyanobacteria have rarely been studied using an assemblage of various datasets. Thus, in this study, a convolutional neural network (CNN) model with a convolutional block attention module (CNN_an) was developed to predict cyanobacterial cell concentrations by using the observed cell data from field monitoring, chlorophyll-a distribution map from hyperspectral image sensing, and simulated water quality outputs from a hydrodynamic model. Then, the prediction performance of the CNN_an model was compared to an environmental fluid dynamics code (EFDC) simulation and a CNN model without an attention network. The seasonal variations of the predicted cyanobacteria that was obtained from CNN_an showed the best agreement with the observed variations with Nash-Sutcliffe efficiency values higher than 0.76 when compared to the EFDC and CNN predictions. The daily hydrodynamic outputs allowed the prediction of cyanobacteria cells, while the rich information of the chlorophyll-a map contributed to the improvement of the prediction performance at certain periods. Moreover, the attention network visualized the importance of the additional chlorophyll-a map and improved the CNN_an model prediction performance by refining the input features. Therefore, this study demonstrated that a deep learning model with data assemblage is practically feasible for predicting the presence of harmful algae in inland water.