State-of-the-Art Deep Learning Methods for Objects Detection in Remote Sensing Satellite Images

Introduction: Object detection in remotely sensed satellite images is critical to socio-economic, bio-physical, and environmental monitoring, necessary for the prevention of natural disasters such as flooding and fires, socio-economic service delivery, and general urban and rural planning and management. Whereas deep learning approaches have recently gained popularity in remotely sensed image analysis, they have been unable to efficiently detect image objects due to complex landscape heterogeneity, high inter-class similarity and intra-class diversity, and difficulty in acquiring suitable training data that represents the complexities, among others. Methods: To address these challenges, this study employed multi-object detection deep learning algorithms with a transfer learning approach on remotely sensed satellite imagery captured on a heterogeneous landscape. In the study, a new dataset of diverse features with five object classes collected from Google Earth Engine in various locations in southern KwaZulu-Natal province in South Africa was used to evaluate the models. The dataset images were characterized with objects that have varying sizes and resolutions. Five (5) object detection methods based on R-CNN and YOLO architectures were investigated via experiments on our newly created dataset. Conclusions: This paper provides a comprehensive performance evaluation and analysis of the recent deep learning-based object detection methods for detecting objects in high-resolution remote sensing satellite images. The models were also evaluated on two publicly available datasets: Visdron and PASCAL VOC2007. Results showed that the highest detection accuracy of the vegetation and swimming pool instances was more than 90%, and the fastest detection speed 0.2 ms was observed in YOLOv8.

Road Risk-Index Analysis Using Satellite Products

This paper proposes a service called intelligent routing using satellite products (IRUS) that can be used in order to analyze risks to the road infrastructure during bad weather conditions, such as heavy rainfall, storms, or floods. By diminishing movement risk, rescuers can arrive safely at their destination. To analyze these routes, the application uses both data provided by Sentinel satellites from the Copernicus program and meteorological data from local weather stations. Moreover, the application uses algorithms to determine the night driving time. From this analysis we obtain a risk index for each road provided by Google Maps API and then we present the path alongside the risk index in a friendly graphic interface. In order to obtain an accurate risk index, the application analyzes both recent and past data (up to 12 months).

A Residual-Inception U-Net (RIU-Net) Approach and Comparisons with U-Shaped CNN and Transformer Models for Building Segmentation from High-Resolution Satellite Images

Building segmentation is crucial for applications extending from map production to urban planning. Nowadays, it is still a challenge due to CNNs' inability to model global context and Transformers' high memory need. In this study, 10 CNN and Transformer models were generated, and comparisons were realized. Alongside our proposed Residual-Inception U-Net (RIU-Net), U-Net, Residual U-Net, and Attention Residual U-Net, four CNN architectures (Inception, Inception-ResNet, Xception, and MobileNet) were implemented as encoders to U-Net-based models. Lastly, two Transformer-based approaches (Trans U-Net and Swin U-Net) were also used. Massachusetts Buildings Dataset and Inria Aerial Image Labeling Dataset were used for training and evaluation. On Inria dataset, RIU-Net achieved the highest IoU score, F1 score, and test accuracy, with 0.6736, 0.7868, and 92.23%, respectively. On Massachusetts Small dataset, Attention Residual U-Net achieved the highest IoU and F1 scores, with 0.6218 and 0.7606, and Trans U-Net reached the highest test accuracy, with 94.26%. On Massachusetts Large dataset, Residual U-Net accomplished the highest IoU and F1 scores, with 0.6165 and 0.7565, and Attention Residual U-Net attained the highest test accuracy, with 93.81%. The results showed that RIU-Net was significantly successful on Inria dataset. On Massachusetts datasets, Residual U-Net, Attention Residual U-Net, and Trans U-Net provided successful results.

An Innovative Fusion-Based Scenario for Improving Land Crop Mapping Accuracy

The accuracy of land crop maps obtained from satellite images depends on the type of feature selection algorithm and classifier. Each of these algorithms have different efficiency in different conditions; therefore, developing a suitable strategy for combining the capabilities of different algorithms in preparing a land crop map with higher accuracy can be very useful. The objective of this study was to develop a fusion-based framework for improving land crop mapping accuracy. First, the features were retrieved using the Sentinel 1, Sentinel 2, and Landsat-8 imagery. Then, training data and various feature selection algorithms including recursive feature elimination (RFE), random forest (RF), and Boruta were used for optimal feature selection. Various classifiers, including artificial neural network (ANN), support vector machine (SVM), and RF, were implemented to create maps of land crops relying on optimal features and training data. After that, in order to increase the result accuracy, maps of land crops derived from several scenarios were fused using a fusion-based voting strategy at the level of decision, and new maps of land crops and classification uncertainty maps were prepared. Subsequently, the performance of different scenarios was evaluated and compared. Among the feature selection algorithms, RF accuracy was higher than RFE and Boruta. Moreover, the efficiency of RF was higher than SVM and ANN. The overall accuracy of the voting scenario was higher than all other scenarios. The finding of this research demonstrated that combining the features' capabilities extracted from sensors in different spectral ranges, different feature selection algorithms, and classifiers improved the land crop classification accuracy.

Automatic Target Detection from Satellite Imagery Using Machine Learning

Object detection is a vital step in satellite imagery-based computer vision applications such as precision agriculture, urban planning and defense applications. In satellite imagery, object detection is a very complicated task due to various reasons including low pixel resolution of objects and detection of small objects in the large scale (a single satellite image taken by Digital Globe comprises over 240 million pixels) satellite images. Object detection in satellite images has many challenges such as class variations, multiple objects pose, high variance in object size, illumination and a dense background. This study aims to compare the performance of existing deep learning algorithms for object detection in satellite imagery. We created the dataset of satellite imagery to perform object detection using convolutional neural network-based frameworks such as faster RCNN (faster region-based convolutional neural network), YOLO (you only look once), SSD (single-shot detector) and SIMRDWN (satellite imagery multiscale rapid detection with windowed networks). In addition to that, we also performed an analysis of these approaches in terms of accuracy and speed using the developed dataset of satellite imagery. The results showed that SIMRDWN has an accuracy of 97% on high-resolution images, while Faster RCNN has an accuracy of 95.31% on the standard resolution (1000 × 600). YOLOv3 has an accuracy of 94.20% on standard resolution (416 × 416) while on the other hand SSD has an accuracy of 84.61% on standard resolution (300 × 300). When it comes to speed and efficiency, YOLO is the obvious leader. In real-time surveillance, SIMRDWN fails. When YOLO takes 170 to 190 milliseconds to perform a task, SIMRDWN takes 5 to 103 milliseconds.

Tree Crowns Segmentation and Classification in Overlapping Orchards Based on Satellite Images and Unsupervised Learning Algorithms

Smart agriculture is a new concept that combines agriculture and new technologies to improve the yield's quality and quantity as well as facilitate many tasks for farmers in managing orchards. An essential factor in smart agriculture is tree crown segmentation, which helps farmers automatically monitor their orchards and get information about each tree. However, one of the main problems, in this case, is when the trees are close to each other, which means that it would be difficult for the algorithm to delineate the crowns correctly. This paper used satellite images and machine learning algorithms to segment and classify trees in overlapping orchards. The data used are images from the Moroccan Mohammed VI satellite, and the study region is the OUARGHA citrus orchard located in Morocco. Our approach starts by segmenting the rows inside the parcel and finding all the trees there, getting their canopies, and classifying them by size. In general, the model inputs the parcel's image and other field measurements to classify the trees into three classes: missing/weak, normal, or big. Finally, the results are visualized in a map containing all the trees with their classes. For the results, we obtained a score of 0.93 F-measure in rows segmentation. Additionally, several field comparisons were performed to validate the classification, dozens of trees were compared and the results were very good. This paper aims to help farmers to quickly and automatically classify trees by crown size, even if there are overlapping orchards, in order to easily monitor each tree's health and understand the tree's distribution in the field.

Using Satellite Images and Deep Learning to Identify Associations Between County-Level Mortality and Residential Neighborhood Features Proximal to Schools: A Cross-Sectional Study

What is the relationship between mortality and satellite images as elucidated through the use of Convolutional Neural Networks? Background: Following a century of increase, life expectancy in the United States has stagnated and begun to decline in recent decades. Using satellite images and street view images, prior work has demonstrated associations of the built environment with income, education, access to care, and health factors such as obesity. However, assessment of learned image feature relationships with variation in crude mortality rate across the United States has been lacking. Objective: We sought to investigate if county-level mortality rates in the U.S. could be predicted from satellite images. Methods: Satellite images of neighborhoods surrounding schools were extracted with the Google Static Maps application programming interface for 430 counties representing ~68.9% of the US population. A convolutional neural network was trained using crude mortality rates for each county in 2015 to predict mortality. Learned image features were interpreted using Shapley Additive Feature Explanations, clustered, and compared to mortality and its associated covariate predictors. Results: Predicted mortality from satellite images in a held-out test set of counties was strongly correlated to the true crude mortality rate (Pearson r = 0.72). Direct prediction of mortality using a deep learning model across a cross-section of 430 U.S. counties identified key features in the environment (e.g., sidewalks, driveways, and hiking trails) associated with lower mortality. Learned image features were clustered, and we identified 10 clusters that were associated with education, income, geographical region, race, and age. Conclusions: The application of deep learning techniques to remotely-sensed features of the built environment can serve as a useful predictor of mortality in the United States. Although we identified features that were largely associated with demographic information, future modeling approaches that directly identify image features associated with health-related outcomes have the potential to inform targeted public health interventions.

Vehicle Detection in Overhead Satellite Images Using a One-Stage Object Detection Model

In order to improve the traffic in large cities and to avoid congestion, advanced methods of detecting and predicting vehicle behaviour are needed. Such methods require complex information regarding the number of vehicles on the roads, their positions, directions, etc. One way to obtain this information is by analyzing overhead images collected by satellites or drones, and extracting information from them through intelligent machine learning models. Thus, in this paper we propose and present a one-stage object detection model for finding vehicles in satellite images using the RetinaNet architecture and the Cars Overhead With Context dataset. By analyzing the results obtained by the proposed model, we show that it has a very good vehicle detection accuracy and a very low detection time, which shows that it can be employed to successfully extract data from real-time satellite or drone data.

Surface Heterogeneity-Involved Estimation of Sample Size for Accuracy Assessment of Land Cover Product from Satellite Imagery

Sample size estimation is a key issue for validating land cover products derived from satellite images. Based on the fact that present sample size estimation methods account for the characteristics of the Earth’s subsurface, this study developed a model for estimating sample size by considering the scale effect and surface heterogeneity. First, we introduced a watershed with different areas to indicate the scale effect on the sample size. Then, by employing an all-subsets regression feature selection method, three landscape indicators describing the aggregation and diversity of the land cover patches were selected (from 14 indicators) as the main factors for indicating the surface heterogeneity. Finally, we developed a multi-level linear model for sample size estimation using explanatory variables, including the estimated sample size (n) calculated from the traditional statistical model, size of the test region, and three landscape indicators. As reference data for developing this model, we employed a case study in the Jiangxi Province using a 30 m spatial resolution global land cover product (Globeland30) from 2010 as a classified map, and national 30 m land use/cover change (LUCC) data from 2010 in China. The results showed that the adjusted square coefficient of R² is 0.79, indicating that the joint explanatory ability of all predictive variables in the model to the sample size is 79%. This means that the predictability of this model is at a good level. By comparing the sample size N_S obtained by the developed multi-level linear model and n as calculated from the statistics model, we find that N_S is much smaller than n, which mainly contributes to the concerns regarding surface heterogeneity in this study. The validity of the established model is tested and is proven as effective in the Anhui Province. This indicates that the estimated sample size from considering the scale effect and spatial heterogeneity in this study achieved the same accuracy as that calculated from a probability statistical model, while simultaneously saving more time, labour, and money in the accuracy assessment of a land cover dataset.

Multi-Mission Earth Observation Data Processing System

The surge in the number of earth observation satellites being launched worldwide is placing significant pressure on the satellite-direct ground receiving stations that are responsible for systematic data acquisition, processing, archiving, and dissemination of earth observation data. Growth in the number of satellite sensors has a bearing on the ground segment payload data processing systems due to the complexity, volume, and variety of the data emanating from the different sensors. In this paper, we have aimed to present a generic, multi-mission, modularized payload data processing system that we are implementing to optimize satellite data processing from historical and current sensors, directly received at the South African National Space Agency's (SANSA) ground receiving station. We have presented the architectural framework for the multi-mission processing system, which is comprised of five processing modules, i.e., the data ingestion module, a radiometric and geometric processing module, atmospheric correction and Analysis Ready Data (ARD) module, Value Added Products (VAPS) module, and lastly, a packaging and delivery module. Our results indicate that the open architecture, multi-mission processing system, when implemented, eliminated the bottlenecks linked with proprietary mono-mission systems. The customizable architecture enabled us to optimize our processing in line with our hardware capacities, and that resulted in significant gains in large-scale image processing efficiencies. The modularized, multi-mission data processing enabled seamless end-to-end image processing, as demonstrated by the capability of the multi-mission system to execute geometric and radiometric corrections to the extent of making it analysis-ready. The processing workflows were highly scalable and enabled us to generate higher-level thematic information products from the ingestion of raw data.

Population-level study links short-wavelength nighttime illumination with breast cancer incidence in a major metropolitan area

Several population-level studies revealed a positive association between breast cancer (BC) incidence and artificial light at night (ALAN) exposure. However, the effect of short-wavelength illumination, implicated by laboratory research and small-scale cohort studies as the main driving force behind BC-ALAN association, has not been supported by any population-level study carried out to date. We investigated a possible link between BC and ALAN of different subspectra using a multi-spectral year-2011 satellite image, taken from the International Space Station, and superimposing it with year-2013 BC incidence data available for the Great Haifa Metropolitan Area in Israel. The analysis was performed using both ordinary least square (OLS) and spatial dependency models, controlling for socioeconomic and locational attributes of the study area. The study revealed strong associations between BC and blue and green light subspectra (B = 0.336 ± 0.001 and B = 0.335 ± 0.002, respectively; p < 0.01), compared to a somewhat weaker effect for the red subspectrum (B = 0.056 ± 0.001; p < 0.01). However, spatial dependency models, controlling for spatial autocorrelation of regression residuals, confirmed only a positive association between BC incidence and short-wavelength (blue) ALAN subspectrum (z = 2.462, p < 0.05) while reporting insignificant associations between BC and either green (z = 1.425, p > 0.1) or red (z = -0.604, p > 0.1) subspectra. The obtained result is in line with the results of laboratory- and small-scale cohort studies linking short-wavelength nighttime illumination with circadian disruption and melatonin suppression. The detected effect of blue lights on BC incidence may help to develop informed illumination policies aimed at minimizing the adverse health effects of ALAN exposure on human health.

Geometric Positioning for Satellite Imagery without Ground Control Points by Exploiting Repeated Observation

With the development of space technology and the performance of remote sensors, high-resolution satellites are continuously launched by countries around the world. Due to high efficiency, large coverage and not being limited by the spatial regulation, satellite imagery becomes one of the important means to acquire geospatial information. This paper explores geometric processing using satellite imagery without ground control points (GCPs). The outcome of spatial triangulation is introduced for geo-positioning as repeated observation. Results from combining block adjustment with non-oriented new images indicate the feasibility of geometric positioning with the repeated observation. GCPs are a must when high accuracy is demanded in conventional block adjustment; the accuracy of direct georeferencing with repeated observation without GCPs is superior to conventional forward intersection and even approximate to conventional block adjustment with GCPs. The conclusion is drawn that taking the existing oriented imagery as repeated observation enhances the effective utilization of previous spatial triangulation achievement, which makes the breakthrough for repeated observation to improve accuracy by increasing the base-height ratio and redundant observation. Georeferencing tests using data from multiple sensors and platforms with the repeated observation will be carried out in the follow-up research.

The influence of land use change on landslide susceptibility zonation: the Briga catchment test site (Messina, Italy)

The spatial distribution of landslides is influenced by different climatic conditions and environmental settings including topography, morphology, hydrology, lithology, and land use. In this work, we have attempted to evaluate the influence of land use change on landslide susceptibility (LS) for a small study area located in the southern part of the Briga catchment, along the Ionian coast of Sicily (Italy). On October 1, 2009, the area was hit by an intense rainfall event that triggered abundant slope failures and resulted in widespread erosion. After the storm, an inventory map showing the distribution of pre-event and event landslides was prepared for the area. Moreover, two different land use maps were developed: the first was obtained through a semi-automatic classification of digitized aerial photographs acquired in 1954, the second through the combination of supervised classifications of two recent QuickBird images. Exploiting the two land use maps and different land use scenarios, LS zonations were prepared through multivariate statistical analyses. Differences in the susceptibility models were analyzed and quantified to evaluate the effects of land use change on the susceptibility zonation. Susceptibility maps show an increase in the areal percentage and number of slope units classified as unstable related to the increase in bare soils to the detriment of forested areas.

Analyzing the Variation of Building Density Using High Spatial Resolution Satellite Images: the Example of Shanghai City

Building density is an important issue in urban planning and land management. In the article, building coverage ratio (BCR) and floor area ratio (FAR) values extracted from high resolution satellite images were used to indicate buildings’ stretching on the surface and growth along the third dimension within areas of interest in Shanghai City, P.R. China. The results show that the variation of FAR is higher than that of BCR in the inner circle, and that the newer commercial centers have higher FAR and lower BCR values, while the traditional commercial areas have higher FAR and BCR ratios. By comparing different residential areas, it was found that the historical “Shikumen” areas and the old residential areas built before 1980s have higher BCR and lower FAR, while the new residential areas have higher FAR and lower BCR, except for the villa areas. These results suggest that both older building areas and villa areas use land resources in an inefficient way, and therefore better planning and management of urban land are needed for those fast economic growing regions.