Deep-learning-based information mining from ocean remote-sensing imagery

Deep Learning for Ocean Forecasting: A Comprehensive Review of Methods, Applications, and Datasets

As a longstanding scientific challenge, accurate and timely ocean forecasting has always been a sought-after goal for ocean scientists. However, traditional theory-driven numerical ocean prediction (NOP) suffers from various challenges, such as the indistinct representation of physical processes, inadequate application of observation assimilation, and inaccurate parameterization of models, which lead to difficulties in obtaining effective knowledge from massive observations, and enormous computational challenges. With the successful evolution of data-driven deep learning in various domains, it has been demonstrated to mine patterns and deep insights from the ever-increasing stream of oceanographic spatiotemporal data, which provides novel possibilities for revolution in ocean forecasting. Deep-learning-based ocean forecasting (DLOF) is anticipated to be a powerful complement to NOP. Nowadays, researchers attempt to introduce deep learning into ocean forecasting and have achieved significant progress that provides novel motivations for ocean science. This article provides a comprehensive review of the state-of-the-art DLOF research regarding model architectures, spatiotemporal multiscales, and interpretability while specifically demonstrating the feasibility of developing hybrid architectures that incorporate theory-driven and data-driven models. Moreover, we comprehensively evaluate DLOF from datasets, benchmarks, and cloud computing. Finally, the limitations of current research and future trends of DLOF are also discussed and prospected.

Advancing forecasting capabilities: A contrastive learning model for forecasting tropical cyclone rapid intensification

Tropical cyclones (TCs), particularly those that rapidly intensify (RI), pose a significant threat due to the uncertainty in forecasting them. RI TC periods, which intensify by at least 13 m/s within 24 h, remain challenging to forecast accurately. Existing models achieve a probability of detection (POD) of 82.6% and a false alarm rate (FARate) of 27.2%. To address this, we developed a contrastive-based RI TC forecasting (RITCF-contrastive) model, utilizing satellite infrared imagery alongside atmospheric and oceanic data. The RITCF-contrastive model was tested on 1,149 TC periods in the Northwest Pacific from 2020 to 2021, achieving a POD of 92.3% and a FARate of 8.9%. RITCF-contrastive improves on previous models by addressing sample imbalance and incorporating TC structural features, leading to a 11.7% improvement in POD and a 3 times reduction in FARate compared to existing deep learning methods. The RITCF-contrastive model not only enhances RI TC forecasting but also offers a unique approach to forecasting these dangerous weather events.

Deep learning-based fishing ground prediction with multiple environmental factors

Improving the accuracy of fishing ground prediction for oceanic economic species has always been one of the most concerning issues in fisheries research. Recent studies have confirmed that deep learning has achieved superior results over traditional methods in the era of big data. However, the deep learning-based fishing ground prediction model with a single environment suffers from the problem that the area of the fishing ground is too large and not concentrated. In this study, we developed a deep learning-based fishing ground prediction model with multiple environmental factors using neon flying squid (Ommastrephes bartramii) in Northwest Pacific Ocean as an example. Based on the modified U-Net model, the approach involves the sea surface temperature, sea surface height, sea surface salinity, and chlorophyll a as inputs, and the center fishing ground as the output. The model is trained with data from July to November in 2002-2019, and tested with data of 2020. We considered and compared five temporal scales (3, 6, 10, 15, and 30 days) and seven multiple environmental factor combinations. By comparing different cases, we found that the optimal temporal scale is 30 days, and the optimal multiple environmental factor combination contained SST and Chl a. The inclusion of multiple factors in the model greatly improved the concentration of the center fishing ground. The selection of a suitable combination of multiple environmental factors is beneficial to the precise spatial distribution of fishing grounds. This study deepens the understanding of the mechanism of environmental field influence on fishing grounds from the perspective of artificial intelligence and fishery science.

Utilizing deep learning algorithms for automated oil spill detection in medium resolution optical imagery

This study evaluates the performance of three typical convolutional neural network based deep learning algorithms for oil spill detection using medium-resolution optical satellite imagery from Sentinel-2 MSI, Landsat-8 OLI, and Landsat-9 OLI2. Oil slick training and validation dataset were created through a semi-automatic labeling approach, based on chronic and accidental oil spill cases reported worldwide. The research enhances UNet, BiSeNetV2, and DeepLabV3+ architectures by integrating attention mechanisms including the Squeeze-and-Excitation module (SE), Convolutional Block Attention Module (CBAM), and a Simple, parameter-free Attention Module (SimAM), analyzing the optimal model for oil spill detection. Notably, UNet integrated with CBAM, especially with sun glint as a feature, significantly outperformed others, achieving a micro-average F1 score of 88.8 %. This research highlights deep learning's potential in optical remote sensing for oil spill detection, stressing its escalating relevance with the growing deployment of medium- to high-resolution optical satellites.

Can three-dimensional nitrate structure be reconstructed from surface information with artificial intelligence? - A proof-of-concept study

Nitrate is one of the essential variables in the ocean that is a primary control of the upper ocean pelagic ecosystem. Its three-dimensional (3D) structure is vital for understanding the dynamic and ecosystem. Although several gridded nitrate products exist, the possibility of reconstructing the 3D structure of nitrate from surface data has never been exploited. In this study, we employed two advanced artificial intelligence (AI) networks, U-net and Earthformer, to reconstruct nitrate concentration in the Indian Ocean from surface data. Simulation from an ecosystem model was utilized as the labeling data to train and test the AI networks, with wind vectors, wind stress, sea surface temperature, sea surface chlorophyll-a, solar radiation, and precipitation as the input. We compared the performance of two networks and different pre-processing methods. With the input features decomposed into climatology and anomaly components, the Earthformer achieved optimal reconstruction results with a lower normalized mean square error (NRMSE = 0.1591), spatially and temporally, outperforming U-net (NRMSE = 0.2007) and the climatology prediction (NRMSE = 0.2089). Furthermore, Earthformer was more capable of identifying interannual nitrate anomalies. With a network interpretation technique, we quantified the spatio-temporal importance of every input feature in the best case (Earthformer with decomposed inputs). The influence of different input features on nitrate concentration in the adjacent Java Sea exhibited seasonal variation, stronger than the interannual one. The feature importance highlighted the role of dynamic factors, particularly the wind, matching our understanding of the dynamic controls of the ecosystem. Our reconstruction and network interpretation technique can be extended to other ecosystem variables, providing new possibilities in studies of marine environment and ecology from an AI perspective.

Satellite-Based Global Sea Surface Oxygen Mapping and Interpretation with Spatiotemporal Machine Learning

The assessment of dissolved oxygen (DO) concentration at the sea surface is essential for comprehending the global ocean oxygen cycle and associated environmental and biochemical processes as it serves as the primary site for photosynthesis and sea-air exchange. However, limited comprehensive measurements and imprecise numerical simulations have impeded the study of global sea surface DO and its relationship with environmental challenges. This paper presents a novel spatiotemporal information embedding machine-learning framework that provides explanatory insights into the underlying driving mechanisms. By integrating extensive in situ data and high-resolution satellite data, the proposed framework successfully generated high-resolution (0.25° × 0.25°) estimates of DO concentration with exceptional accuracy (R2 = 0.95, RMSE = 11.95 μmol/kg, and test number = 2805) for near-global sea surface areas from 2010 to 2018, uncertainty estimated to be ±13.02 μmol/kg. The resulting sea surface DO data set exhibits precise spatial distribution and reveals compelling correlations with prominent marine phenomena and environmental stressors. Leveraging its interpretability, our model further revealed the key influence of marine factors on surface DO and their implications for environmental issues. The presented machine-learning framework offers an improved DO data set with higher resolution, facilitating the exploration of oceanic DO variability, deoxygenation phenomena, and their potential consequences for environments.

SLWE-Net: An improved lightweight U-Net for Sargassum extraction from GOCI images

The Sargassum bloom has severely impacted the ecological environment of the East China Sea and the Yellow Sea, causing significant economic losses. In recent years, deep learning has seen extensive development due to its outstanding feature extraction capabilities. However, the deep learning process typically involves a large number of parameters and computations. To address this issue, this paper proposes a lightweight deep learning network based on the U-Net framework, called SLWE-NET, which uses lightweight modules to replace the feature extraction modules in U-Net. In this experiment, SLWE-Net performed the best in both extraction accuracy and model lightweight. Compared to the formal U-Net, the number of parameters decreased by 65.83 %, the model size reduced from 94.97 MB to 32.51 MB, and the mIoU increased to 93.81 %. Therefore, the method proposed in this paper is beneficial for Sargassum extraction and provides a basis for operational monitoring.

Detection of marine oil-like features in Sentinel-1 SAR images by supplementary use of deep learning and empirical methods: Performance assessment for the Great Barrier Reef marine park

Continuous monitoring of oil discharges in coastal and open ocean waters using Earth Observation (EO) has undeniably contributed to diminishing their occurrence wherever a detection system was in place, such as in Europe (EMSA's CleanSeaNet) or in the United States (NOAA's OR&R). This study describes the development and testing of a semi-automated oil slick detection system tailored to the Great Barrier Reef (GBR) marine park solely based on EO data as no such service was routinely available in Australia until recently. In this study, a large, curated, historical global dataset of SAR imagery acquired by Sentinel-1 SAR, now publicly available, is used to assess classification techniques, namely an empirical approach and a deep learning model, to discriminate between oil-like features and look-alikes in the scenes acquired over the marine park. An evaluation of this detection system on 10 Sentinel-1 SAR images of the GBR using two performance metrics - the detection accuracy and the false-positive rate (FPR) - shows that the classifiers perform best when combined (accuracy >98 %; FPR 0.01) rather than when used separately. This study demonstrates the benefit of sequentially combining classifiers to improve the detection and monitoring of unreported oil discharge events in SAR imagery. The workflow has also been tested outside the GBR, demonstrating its robustness when applied to other regions such as Australia's Northwest Shelf, Southeast Asia and the Pacific.

Long-term spatial variation of algal blooms extracted using the U-net model from 10 years of GOCI imagery in the East China Sea

Long-term satellite missions could help to provide insights into spatial and temporal variations in algal blooms. However, the traditional reflectance-based method has limitations in regards to determining the available threshold for algal bloom detection among the time-varying observation conditions. In terms of extracting useful information from long-term data series precisely and efficiently, the deep learning method has shown its superiority over traditional algorithms in batch data processing. In this study, a U-net model for algal bloom extraction along the coast of the East China Sea was developed using GOCI images. The U-net model was trained with two different datasets that were constructed with six-band channels (all visible bands from GOCI imagery) and RGB-band channels (bands of 443, 555, and 680 nm from GOCI imagery). The quantitative assessment from the U-net models suggests that the U-net model trained with the six-band channel datasets outperformed the RGB-band channel datasets, with increases of 23.6%, 18.1%, and 12.5% in terms of accuracy, precision, and F-score, respectively. The validation map derived from the U-net model trained with six-band channel datasets also showed considerable matching with the ground-truth maps. By using the U-net model, the occurrence of algal blooms was automatically extracted from GOCI images. A 10-year time series of GOCI data collected between 2011 and 2020 was derived using an output-trained U-net model to explore spatial variation along the coast of the ECS. It was found that the most affected areas of the algal blooms varied by year, but were mainly located in the Zhoushan and Zhejiang coasts. Additionally, by performing principal component analysis on the daily meteorological data during April and August 2011-2020, factors related to algal bloom occurrence were discussed.

Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images

In this article, we focus on the challenging multicategory instance segmentation problem in remote sensing images (RSIs), which aims at predicting the categories of all instances and localizing them with pixel-level masks. Although many landmark frameworks have demonstrated promising performance in instance segmentation, the complexity in the background and scale variability instances still remain challenging, for instance, segmentation of RSIs. To address the above problems, we propose an end-to-end multicategory instance segmentation model, namely, the semantic attention (SEA) and scale complementary network, which mainly consists of a SEA module and a scale complementary mask branch (SCMB). The SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map and reduce the background noise's interference. To handle the undersegmentation of geospatial instances with large varying scales, we design the SCMB that extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales to sufficiently leverage the multiscale information. We conduct comprehensive experiments to evaluate the effectiveness of our proposed method on the iSAID dataset and the NWPU Instance Segmentation dataset and achieve promising performance.

Use of semantic segmentation for mapping Sargassum on beaches

The unusual arrival of Sargassum on Caribbean beaches is an emerging problem that has generated numerous challenges. The monitoring, visualization, and estimation of Sargassum coverage on the beaches remain a constant complication. This study proposes a new mapping methodology to estimate Sargassum coverage on the beaches. Semantic segmentation of geotagged photographs allows the generation of accurate maps showing the percent coverage of Sargassum. The first dataset of segmented Sargassum images was built for this study and used to train the proposed model. The results demonstrate that the currently proposed method has an accuracy of 91%, improving on the results reported in the state-of-the-art method where data was also collected through a crowdsourcing scheme, in which only information on the presence and absence of Sargassum is displayed.

Research on COD Soft Measurement Technology Based on Multi-Parameter Coupling Analysis Method

This paper presents a soft measurement technique for COD (Chemical Oxygen Demand) based on the multiparameter coupling analysis method. First, through mechanism analysis and correlation analysis of historical data during the measurement process, water quality parameters, such as hydrogen potential (PH), dissolved oxygen (DO), turbidity (TU), and electrical conductivity (EC), can be used to estimate COD values. To further improve the estimation accuracy of the water quality parameter model, we adopted a modeling method combining a BP neural network and support vector machine, which showed an average relative error of 6.13% and an absolute coefficient of up to 0.9605. Finally, experiments in a lake environment demonstrate that this method shows excellent performance, with highly reliable and accurate prediction results.

A Neural Network Method for Retrieving Sea Surface Wind Speed for C-Band SAR

Based on the Ocean Projection and Extension neural Network (OPEN) method, a novel approach is proposed to retrieve sea surface wind speed for C-band synthetic aperture radar (SAR). In order to prove the methodology with a robust dataset, five-year normalized radar cross section (NRCS) measurements from the advanced scatterometer (ASCAT), a well-known side-looking radar sensor, are used to train the model. In situ wind data from direct buoy observations, instead of reanalysis wind data or model results, are used as the ground truth in the OPEN model. The model is applied to retrieve sea surface winds from two independent data sets, ASCAT and Sentinel-1 SAR data, and has been well-validated using buoy measurements from the National Oceanic and Atmospheric Administration (NOAA) and China Meteorological Administration (CMA), and the ASCAT coastal wind product. The comparison between the OPEN model and four C-band model (CMOD) versions (CMOD4, CMOD-IFR2, CMOD5.N, and CMOD7) further indicates the good performance of the proposed model for C-band SAR sensors. It is anticipated that the use of high-resolution SAR data together with the new wind speed retrieval method can provide continuous and accurate ocean wind products in the future.

Improving Ocean Forecasting Using Deep Learning and Numerical Model Integration

In this paper, we propose a novel method to enhance the accuracy of a real-time ocean forecasting system. The proposed system consists of a real-time restoration system of satellite ocean temperature based on a deep generative inpainting network (GIN) and assimilation of satellite data with the initial fields of the numerical ocean model. The deep learning real-time ocean forecasting system is as fast as conventional forecasting systems, while also showing enhanced performance. Our results showed that the difference in temperature between in situ observation and actual forecasting results was improved by about 0.5 °C in daily average values in the open sea, which suggests that cutting back the temporal gaps between data assimilation and forecasting enhances the accuracy of the forecasting system in the open ocean. The proposed approach can provide more accurate forecasts with an efficient operation time.

Characteristics and Trends of Ocean Remote Sensing Research from 1990 to 2020: A Bibliometric Network Analysis and Its Implications

The ocean is of great significance in the climate system, global resources and strategic decision making. With the continuous improvement in remote sensing technology, ocean remote sensing research has increasingly become an important topic for resource development and environmental protection. This paper uses bibliometric analysis method and VOSviewer visual software to conduct analysis. The analysis focuses on the period from 1990 to 2020. The analysis results show that articles have been steadily increasing over the past two decades. Scholars and researchers form the United States, China and Europe (mainly Western European countries), as well as NASA, Chinese Academy of Sciences and NOAA have bigger influence in this field to some extent. Among them, the United States and NASA holds the core leading position. Moreover, global cooperation in this field presents certain characteristics of geographical distribution. This study also reveals journals that include the most publications and subject categories that are highly relevant to related fields. Cluster analysis shows that remote sensing, ocean color, MODIS (or Moderate Resolution Imaging Spectroradiometer), chlorophy, sea ice and climate change are main research hotspots. In addition, in the context of climate warming, researchers have improved monitoring technology for remote sensing to warn and protect ocean ecosystems in hotspots (the Arctic and Antarctica). The valuable results obtained from this study will help academic professionals keep informed of the latest developments and identify future research directions in the field related to ocean remote sensing.

Semantic Segmentation of Metoceanic Processes Using SAR Observations and Deep Learning

Through the Synthetic Aperture Radar (SAR) embarked on the satellites Sentinel-1A and Sentinel-1B of the Copernicus program, a large quantity of observations is routinely acquired over the oceans. A wide range of features from both oceanic (e.g., biological slicks, icebergs, etc.) and meteorologic origin (e.g., rain cells, wind streaks, etc.) are distinguishable on these acquisitions. This paper studies the semantic segmentation of ten metoceanic processes either in the context of a large quantity of image-level groundtruths (i.e., weakly-supervised framework) or of scarce pixel-level groundtruths (i.e., fully-supervised framework). Our main result is that a fully-supervised model outperforms any tested weakly-supervised algorithm. Adding more segmentation examples in the training set would further increase the precision of the predictions. Trained on 20 × 20 km imagettes acquired from the WV acquisition mode of the Sentinel-1 mission, the model is shown to generalize, under some assumptions, to wide-swath SAR data, which further extents its application domain to coastal areas.

Calibration Experiments of CFOSAT Wavelength in the Southern South China Sea by Artificial Neural Networks

The wave data measured by CFOSAT (China France Oceanography Satellite) have been validated mainly based on numerical model outputs and altimetry products on a global scale. It is still necessary to further calibrate the data for specific regions, e.g., the southern South China Sea. This study analyses the practicability of calibrating the dominant wavelength by using artificial neural networks and mean impact value analysis based on two sets of buoy data with a 2-year observation period and contemporaneous ERA5 reanalysis data. The artificial neural network modeling experiments are repeated 1000 times randomly by Monte Carlo methods to avoid sampling uncertainty. Both experimental results based on the random sampling method and chronological sampling method are performed. Independent buoy observations are used to validate the calibration model. The results show that although there are obvious differences between the CFOSAT wavelength data and the field observations, the parameters observed by the satellite itself can effectively calibrate the data. In addition to the wavelength, nadir significant wave height, nadir wind speed, and the distance between the calibration point and satellite observation point are the most important parameters for the calibration. Accurate data from other sources, such as ERA5, would be helpful to further improve the calibration results. The variable contributing the most to the calibration effect is the mean wave period, which virtually provides relatively accurate wavelength information for the calibration network. These results verify the possibility of synchronous self-calibration for the CFOSAT wavelength data and provide a reference for the further calibration of the satellite products in other regions.

OUP accepted manuscript

Uncertainties in ocean-mixing parameterizations are primary sources for ocean and climate modeling biases. Due to lack of process understanding, traditional physics-driven parameterizations perform unsatisfactorily in the tropics. Recent advances in the deep-learning method and the new availability of long-term turbulence measurements provide an opportunity to explore data-driven approaches to parameterizing oceanic vertical-mixing processes. Here, we describe a novel parameterization based on an artificial neural network trained using a decadal-long time record of hydrographic and turbulence observations in the tropical Pacific. This data-driven parameterization achieves higher accuracy than current parameterizations, demonstrating good generalization ability under physical constraints. When integrated into an ocean model, our parameterization facilitates improved simulations in both ocean-only and coupled modeling. As a novel application of machine learning to the geophysical fluid, these results show the feasibility of using limited observations and well-understood physical constraints to construct a physics-informed deep-learning parameterization for improved climate simulations.

A Dual-Polarimetric SAR Ship Detection Dataset and a Memory-Augmented Autoencoder-Based Detection Method

With the development of imaging and space-borne satellite technology, a growing number of multipolarized SAR imageries have been implemented for object detection. However, most of the existing public SAR ship datasets are grayscale images under single polarization mode. To make full use of the polarization characteristics of multipolarized SAR, a dual-polarimetric SAR dataset specifically used for ship detection is presented in this paper (DSSDD). For construction, 50 dual-polarimetric Sentinel-1 SAR images were cropped into 1236 image slices with the size of 256 × 256 pixels. The variances and covariance of both VV and VH polarization were fused into R,G,B channels of the pseudo-color image. Each ship was labeled with both a rotatable bounding box (RBox) and a horizontal bounding box (BBox). Apart from 8-bit pseudo-color images, DSSDD also provides 16-bit complex data for readers. Two prevalent object detectors R³Det and Yolo-v4 were implemented on DSSDD to establish the baselines of the detectors with the RBox and BBox respectively. Furthermore, we proposed a weakly supervised ship detection method based on anomaly detection via advanced memory-augmented autoencoder (MemAE), which can significantly remove false alarms generated by the two-parameter CFAR algorithm applied upon our dual-polarimetric dataset. The proposed advanced MemAE method has the advantages of a lower annotation workload, high efficiency, good performance even compared with supervised methods, making it a promising direction for ship detection in dual-polarimetric SAR images. The dataset is available on github.

Research Trends in the Remote Sensing of Phytoplankton Blooms: Results from Bibliometrics

Phytoplankton blooms have caused many serious public safety incidents and eco-environmental problems worldwide and became a focus issue for research. Accurate and rapid monitoring of phytoplankton blooms is critical for forecasting, treating, and management. With the advantages of large spatial coverage and high temporal resolution, remote sensing has been widely used to monitor phytoplankton blooms. Numerous advances have been made in the remote sensing of phytoplankton blooms, biomass, and phenology over the past several decades. To fully understand the development history, research hotspots, and future trends of remote-sensing technology in the study of phytoplankton blooms, we conducted a comprehensive review to systematically analyze the research trends in the remote sensing of phytoplankton blooms through bibliometrics. Our findings showed that research on the use of remote-sensing technology in this field increased substantially in the past 30 years. “Oceanography,” “Environmental Sciences,” and “Remote Sensing” are the most popular subject categories. Remote Sensing of Environment, Journal of Geophysical Research: Oceans, and International Journal of Remote Sensing were the journals with the most published articles. The results of the analysis of international influence and cooperation showed that the United States had the greatest influence in this field and that the cooperation between China and the United States was the closest. The Chinese Academy of Sciences published the largest number of papers, reaching 542 articles. Keyword and topic analysis results showed that “phytoplankton,” “chlorophyll,” and “ocean” were the most frequently occurring keywords, while “eutrophication management and monitoring,” “climate change,” “lakes,” and “remote-sensing algorithms” were the most popular research topics in recent years. Researchers are now paying increasing attention to the phenological response of phytoplankton under the conditions of climate change and the application of new remote-sensing methods. With the development of new remote-sensing technology and the expansion of phytoplankton research, future research should focus on (1) accurate observation of phytoplankton blooms; (2) the traits of phytoplankton blooms; and (3) the drivers, early warning, and management of phytoplankton blooms. In addition, we discuss the future challenges and opportunities in the use of remote sensing in phytoplankton blooms. Our review will promote a deeper and wider understanding of the field.

Exploring data mining: facets and emerging trends

Purpose

Data mining along with its varied technologies like numerical mining, textual mining, multimedia mining, web mining, sentiment analysis and big data mining proves itself as an emerging field and manifests itself in the form of different techniques such as information mining; big data mining; big data mining and Internet of Things (IoT); and educational data mining. This paper aims to discuss how these technologies and techniques are used to derive information and, eventually, knowledge from data.

Design/methodology/approach

An extensive review of literature on data mining and its allied techniques was carried to ascertain the emerging procedures and techniques in the domain of data mining. Clarivate Analytic’s Web of Science and Sciverse Scopus were explored to discover the extent of literature published on Data Mining and its varied facets. Literature was searched against various keywords such as data mining; information mining; big data; big data and IoT; and educational data mining. Further, the works citing the literature on data mining were also explored to visualize a broad gamut of emerging techniques about this growing field.

Findings

The study validates that knowledge discovery in databases has rendered data mining as an emerging field; the data present in these databases paves the way for data mining techniques and analytics. This paper provides a unique view about the usage of data, and logical patterns derived from it, how new procedures, algorithms and mining techniques are being continuously upgraded for their multipurpose use for the betterment of human life and experiences.

Practical implications

The paper highlights different aspects of data mining, its different technological approaches, and how these emerging data technologies are used to derive logical insights from data and make data more meaningful.

Originality/value

The paper tries to highlight the current trends and facets of data mining.

A Deep Learning and GIS Approach for the Optimal Positioning of Wave Energy Converters

Renewable Energy Sources provide a viable solution to the problem of ever-increasing climate change. For this reason, several countries focus on electricity production using alternative sources. In this paper, the optimal positioning of the installation of wave energy converters is examined taking into account geospatial and technical limitations. Geospatial constraints depend on Land Use classes and seagrass of the coastal areas, while technical limitations include meteorological conditions and the morphology of the seabed. Suitable installation areas are selected after the exclusion of points that do not meet the aforementioned restrictions. We implemented a Deep Neural Network that operates based on heterogeneous data fusion, in this case satellite images and time series of meteorological data. This fact implies the definition of a two-branches architecture. The branch that is trained with image data provides for the localization of dynamic geospatial classes in the potential installation area, whereas the second one is responsible for the classification of the region according to the potential wave energy using wave height and period time series. In making the final decision on the suitability of the potential area, a large number of static land use data play an important role. These data are combined with neural network predictions for the optimizing positioning of the Wave Energy Converters. For the sake of completeness and flexibility, a Multi-Task Neural Network is developed. This model, in addition to predicting the suitability of an area depending on seagrass patterns and wave energy, also predicts land use classes through Multi-Label classification process. The proposed methodology is applied in the marine area of the city of Sines, Portugal. The first neural network achieves 98.7% Binary Classification accuracy, while the Multi-Task Neural Network 97.5% in the same metric and 93.5% in the F1 score of the Multi-Label classification output.

High Spatial-Resolution Red Tide Detection in the Southern Coast of Korea Using U-Net from PlanetScope Imagery

Red tides caused by Margalefidinium polykrikoides occur continuously along the southern coast of Korea, where there are many aquaculture cages, and therefore, prompt monitoring of bloom water is required to prevent considerable damage. Satellite-based ocean-color sensors are widely used for detecting red tide blooms, but their low spatial resolution restricts coastal observations. Contrarily, terrestrial sensors with a high spatial resolution are good candidate sensors, despite the lack of spectral resolution and bands for red tide detection. In this study, we developed a U-Net deep learning model for detecting M. polykrikoides blooms along the southern coast of Korea from PlanetScope imagery with a high spatial resolution of 3 m. The U-Net model was trained with four different datasets that were constructed with randomly or non-randomly chosen patches consisting of different ratios of red tide and non-red tide pixels. The qualitative and quantitative assessments of the conventional red tide index (RTI) and four U-Net models suggest that the U-Net model, which was trained with a dataset of non-randomly chosen patches including non-red tide patches, outperformed RTI in terms of sensitivity, precision, and F-measure level, accounting for an increase of 19.84%, 44.84%, and 28.52%, respectively. The M. polykrikoides map derived from U-Net provides the most reasonable red tide patterns in all water areas. Combining high spatial resolution images and deep learning approaches represents a good solution for the monitoring of red tides over coastal regions.

SAFFNet: Self-Attention-Based Feature Fusion Network for Remote Sensing Few-Shot Scene Classification

In real applications, it is necessary to classify new unseen classes that cannot be acquired in training datasets. To solve this problem, few-shot learning methods are usually adopted to recognize new categories with only a few (out-of-bag) labeled samples together with the known classes available in the (large-scale) training dataset. Unlike common scene classification images obtained by CCD (Charge-Coupled Device) cameras, remote sensing scene classification datasets tend to have plentiful texture features rather than shape features. Therefore, it is important to extract more valuable texture semantic features from a limited number of labeled input images. In this paper, a multi-scale feature fusion network for few-shot remote sensing scene classification is proposed by integrating a novel self-attention feature selection module, denoted as SAFFNet. Unlike a pyramidal feature hierarchy for object detection, the informative representations of the images with different receptive fields are automatically selected and re-weighted for feature fusion after refining network and global pooling operation for a few-shot remote sensing classification task. Here, the feature weighting value can be fine-tuned by the support set in the few-shot learning task. The proposed model is evaluated on three publicly available datasets for few shot remote sensing scene classification. Experimental results demonstrate the effectiveness of the proposed SAFFNet to improve the few-shot classification accuracy significantly compared to other few-shot methods and the typical multi-scale feature fusion network.

Subsurface Temperature Estimation from Sea Surface Data Using Neural Network Models in the Western Pacific Ocean

Estimating the ocean subsurface thermal structure (OSTS) based on multisource sea surface data in the western Pacific Ocean is of great significance for studying ocean dynamics and El Niño phenomenon, but it is challenging to accurately estimate the OSTS from sea surface parameters in the area. This paper proposed an improved neural network model to estimate the OSTS from 0–2000 m from multisource sea surface data including sea surface temperature (SST), sea surface salinity (SSS), sea surface height (SSH), and sea surface wind (SSW). In the model experiment, the rasterized monthly average data from 2005–2015 and 2016 were selected as the training and testing set, respectively. The results showed that the sea surface parameters selected in the paper had a positive effect on the estimation process, and the average RMSE value of the ocean subsurface temperature (OST) estimated by the proposed model was 0.55 °C. Moreover, there were pronounced seasonal variation signals in the upper layers (the upper 200 m), however, this signal gradually diminished with increasing depth. Compared with known estimation models such as the random forest (RF), the multiple linear regression (MLR), and the extreme gradient boosting (XGBoost), the proposed model outperformed these models under the data conditions of the paper. This research can provide an advanced artificial intelligence technique for estimating subsurface thermohaline structure in major sea areas.

Unsupervised Domain Adaption for High-Resolution Coastal Land Cover Mapping with Category-Space Constrained Adversarial Network

Coastal land cover mapping (CLCM) across image domains presents a fundamental and challenging segmentation task. Although adversaries-based domain adaptation methods have been proposed to address this issue, they always implement distribution alignment via a global discriminator while ignoring the data structure. Additionally, the low inter-class variances and intricate spatial details of coastal objects may entail poor presentation. Therefore, this paper proposes a category-space constrained adversarial method to execute category-level adaptive CLCM. Focusing on the underlying category information, we introduce a category-level adversarial framework to align semantic features. We summarize two diverse strategies to extract category-wise domain labels for source and target domains, where the latter is driven by self-supervised learning. Meanwhile, we generalize the lightweight adaptation module to multiple levels across a robust baseline, aiming to fine-tune the features at different spatial scales. Furthermore, the self-supervised learning approach is also leveraged as an improvement strategy to optimize the result within segmented training. We examine our method on two converse adaptation tasks and compare them with other state-of-the-art models. The overall visualization results and evaluation metrics demonstrate that the proposed method achieves excellent performance in the domain adaptation CLCM with high-resolution remotely sensed images.

Instance Segmentation for Large, Multi-Channel Remote Sensing Imagery Using Mask-RCNN and a Mosaicking Approach

Instance segmentation is the state-of-the-art in object detection, and there are numerous applications in remote sensing data where these algorithms can produce significant results. Nevertheless, one of the main problems is that most algorithms use Red, Green, and Blue (RGB) images, whereas Satellite images often present more channels that can be crucial to improve performance. Therefore, the present work brings three contributions: (a) conversion system from ground truth polygon data into the Creating Common Object in Context (COCO) annotation format; (b) Detectron2 software source code adaptation and application on multi-channel imagery; and (c) large scene image mosaicking. We applied the procedure in a Center Pivot Irrigation System (CPIS) dataset with ground truth produced by the Brazilian National Water Agency (ANA) and Landsat-8 Operational Land Imager (OLI) imagery (7 channels with 30-m resolution). Center pivots are a modern irrigation system technique with massive growth potential in Brazil and other world areas. The round shapes with different textures, colors, and spectral behaviors make it appropriate to use Deep Learning instance segmentation. We trained the model using 512 × 512-pixel sized patches using seven different backbone structures (ResNet50- Feature Pyramid Network (FPN), Resnet50-DC5, ResNet50-C4, Resnet101-FPN, Resnet101-DC5, ResNet101-FPN, and ResNeXt101-FPN). The model evaluation used standard COCO metrics (Average Precision (AP), AP50, AP75, APsmall, APmedium, and AR100). ResNeXt101-FPN had the best results, with a 3% advantage over the second-best model (ResNet101-FPN). We also compared the ResNeXt101-FPN model in the seven-channel and RGB imagery, where the multi-channel model had a 3% advantage, demonstrating great improvement using a larger number of channels. This research is also the first with a mosaicking algorithm using instance segmentation models, where we tested in a 1536 × 1536-pixel image using a non-max suppression sorted by area method. The proposed methodology is innovative and suitable for many other remote sensing problems and medical imagery that often present more channels.

Sensors, Features, and Machine Learning for Oil Spill Detection and Monitoring: A Review

Remote sensing technologies and machine learning (ML) algorithms play an increasingly important role in accurate detection and monitoring of oil spill slicks, assisting scientists in forecasting their trajectories, developing clean-up plans, taking timely and urgent actions, and applying effective treatments to contain and alleviate adverse effects. Review and analysis of different sources of remotely sensed data and various components of ML classification systems for oil spill detection and monitoring are presented in this study. More than 100 publications in the field of oil spill remote sensing, published in the past 10 years, are reviewed in this paper. The first part of this review discusses the strengths and weaknesses of different sources of remotely sensed data used for oil spill detection. Necessary preprocessing and preparation of data for developing classification models are then highlighted. Feature extraction, feature selection, and widely used handcrafted features for oil spill detection are subsequently introduced and analyzed. The second part of this review explains the use and capabilities of different classical and developed state-of-the-art ML techniques for oil spill detection. Finally, an in-depth discussion on limitations, open challenges, considerations of oil spill classification systems using remote sensing, and state-of-the-art ML algorithms are highlighted along with conclusions and insights into future directions.

Retrieval of Sea Surface Wind Speed from Spaceborne SAR over the Arctic Marginal Ice Zone with a Neural Network

In this paper, we presented a method for retrieving sea surface wind speed (SSWS) from Sentinel-1 synthetic aperture radar (SAR) horizontal-horizontal (HH) polarization data in extra-wide (EW) swath mode, which have been extensively acquired over the Arctic for polar monitoring. In contrast to the conventional algorithm, i.e., using a geophysical model function (GMF) to retrieve SSWS by spaceborne SAR, we introduced an alternative retrieval method based on a GMF-guided neural network. The SAR normalized radar cross section, incidence angle, and wind direction are used as the inputs of a back propagation (BP) neural network, and the output is the SSWS. The network is developed based on 11,431 HH-polarized EW images acquired in the marginal ice zone (MIZ) of the Arctic from 2015 to 2018 and their collocated scatterometer wind measurements. Verification of the neural network based on the testing dataset yields a bias of 0.23 m/s and a root mean square error (RMSE) of 1.25 m/s compared to the scatterometer wind data for wind speeds less than approximately 30 m/s. Further comparison of the SAR retrieved SSWS with independent buoy measurements shows a bias and RMSE of 0.12 m/s and 1.42 m/s, respectively. We also analyzed the uncertainty of the retrieval when reanalysis model wind direction data are used as inputs to the neural network. By combining the detected sea ice cover information based on SAR data, sea ice and marine-meteorological parameters can be derived simultaneously by spaceborne SAR at a high spatial resolution in the Arctic.