The new Seafloor Observatory (OBSEA) for remote and long-term coastal ecosystem monitoring

Image dataset for benchmarking automated fish detection and classification algorithms

Multiparametric video-cabled marine observatories are becoming strategic to monitor remotely and in real-time the marine ecosystem. Those platforms can achieve continuous, high-frequency and long-lasting image data sets that require automation in order to extract biological time series. The OBSEA, located at 4 km from Vilanova i la Geltrú at 20 m depth, was used to produce coastal fish time series continuously over the 24-h during 2013-2014. The image content of the photos was extracted via tagging, resulting in 69917 fish tags of 30 taxa identified. We also provided a meteorological and oceanographic dataset filtered by a quality control procedure to define real-world conditions affecting image quality. The tagged fish dataset can be of great importance to develop Artificial Intelligence routines for the automated identification and classification of fishes in extensive time-lapse image sets.

Mobile robotic platforms for the acoustic tracking of deep-sea demersal fishery resources

Knowing the displacement capacity and mobility patterns of industrially exploited (i.e., fished) marine resources is key to establishing effective conservation management strategies in human-impacted marine ecosystems. Acquiring accurate behavioral information of deep-sea fished ecosystems is necessary to establish the sizes of marine protected areas within the framework of large international societal programs (e.g., European Community H2020, as part of the Blue Growth economic strategy). However, such information is currently scarce, and high-frequency and prolonged data collection is rarely available. Here, we report the implementation of autonomous underwater vehicles and remotely operated vehicles as an aid for acoustic long-baseline localization systems for autonomous tracking of Norway lobster (Nephrops norvegicus), one of the key living resources exploited in European waters. In combination with seafloor moored acoustic receivers, we detected and tracked the movements of 33 tagged lobsters at 400-m depth for more than 3 months. We also identified the best procedures to localize both the acoustic receivers and the tagged lobsters, based on algorithms designed for off-the-shelf acoustic tags identification. Autonomous mobile platforms that deliver data on animal behavior beyond traditional fixed platform capabilities represent an advance for prolonged, in situ monitoring of deep-sea benthic animal behavior at meter spatial scales.

The Hierarchic Treatment of Marine Ecological Information from Spatial Networks of Benthic Platforms

Measuring biodiversity simultaneously in different locations, at different temporal scales, and over wide spatial scales is of strategic importance for the improvement of our understanding of the functioning of marine ecosystems and for the conservation of their biodiversity. Monitoring networks of cabled observatories, along with other docked autonomous systems (e.g., Remotely Operated Vehicles [ROVs], Autonomous Underwater Vehicles [AUVs], and crawlers), are being conceived and established at a spatial scale capable of tracking energy fluxes across benthic and pelagic compartments, as well as across geographic ecotones. At the same time, optoacoustic imaging is sustaining an unprecedented expansion in marine ecological monitoring, enabling the acquisition of new biological and environmental data at an appropriate spatiotemporal scale. At this stage, one of the main problems for an effective application of these technologies is the processing, storage, and treatment of the acquired complex ecological information. Here, we provide a conceptual overview on the technological developments in the multiparametric generation, storage, and automated hierarchic treatment of biological and environmental information required to capture the spatiotemporal complexity of a marine ecosystem. In doing so, we present a pipeline of ecological data acquisition and processing in different steps and prone to automation. We also give an example of population biomass, community richness and biodiversity data computation (as indicators for ecosystem functionality) with an Internet Operated Vehicle (a mobile crawler). Finally, we discuss the software requirements for that automated data processing at the level of cyber-infrastructures with sensor calibration and control, data banking, and ingestion into large data portals.

Video Image Enhancement and Machine Learning Pipeline for Underwater Animal Detection and Classification at Cabled Observatories

An understanding of marine ecosystems and their biodiversity is relevant to sustainable use of the goods and services they offer. Since marine areas host complex ecosystems, it is important to develop spatially widespread monitoring networks capable of providing large amounts of multiparametric information, encompassing both biotic and abiotic variables, and describing the ecological dynamics of the observed species. In this context, imaging devices are valuable tools that complement other biological and oceanographic monitoring devices. Nevertheless, large amounts of images or movies cannot all be manually processed, and autonomous routines for recognizing the relevant content, classification, and tagging are urgently needed. In this work, we propose a pipeline for the analysis of visual data that integrates video/image annotation tools for defining, training, and validation of datasets with video/image enhancement and machine and deep learning approaches. Such a pipeline is required to achieve good performance in the recognition and classification tasks of mobile and sessile megafauna, in order to obtain integrated information on spatial distribution and temporal dynamics. A prototype implementation of the analysis pipeline is provided in the context of deep-sea videos taken by one of the fixed cameras at the LoVe Ocean Observatory network of Lofoten Islands (Norway) at 260 m depth, in the Barents Sea, which has shown good classification results on an independent test dataset with an accuracy value of 76.18% and an area under the curve (AUC) value of 87.59%.

Annual rhythms of temporal niche partitioning in the Sparidae family are correlated to different environmental variables

The seasonal timing of recurring biological processes is essential for organisms living in temperate regions. While ample knowledge of these processes exists for terrestrial environments, seasonal timing in the marine environment is relatively understudied. Here, we characterized the annual rhythm of habitat use in six fish species belonging to the Sparidae family, highlighting the main environmental variables that correlate to such rhythms. The study was conducted at a coastal artificial reef through a cabled observatory system, which allowed gathering underwater time-lapse images every 30 minutes consecutively over 3 years. Rhythms of fish counts had a significant annual periodicity in four out of the six studied species. Species-specific temporal patterns were found, demonstrating a clear annual temporal niche partitioning within the studied family. Temperature was the most important environmental variable correlated with fish counts in the proximity of the artificial reef, while daily photoperiod and salinity were not important. In a scenario of human-induced rapid environmental change, tracking phenological shifts may provide key indications about the effects of climate change at both species and ecosystem level. Our study reinforces the efficacy of underwater cabled video-observatories as a reliable tool for long-term monitoring of phenological events.

Tracking Fish Abundance by Underwater Image Recognition

Marine cabled video-observatories allow the non-destructive sampling of species at frequencies and durations that have never been attained before. Nevertheless, the lack of appropriate methods to automatically process video imagery limits this technology for the purposes of ecosystem monitoring. Automation is a prerequisite to deal with the huge quantities of video footage captured by cameras, which can then transform these devices into true autonomous sensors. In this study, we have developed a novel methodology that is based on genetic programming for content-based image analysis. Our aim was to capture the temporal dynamics of fish abundance. We processed more than 20,000 images that were acquired in a challenging real-world coastal scenario at the OBSEA-EMSO testing-site. The images were collected at 30-min. frequency, continuously for two years, over day and night. The highly variable environmental conditions allowed us to test the effectiveness of our approach under changing light radiation, water turbidity, background confusion, and bio-fouling growth on the camera housing. The automated recognition results were highly correlated with the manual counts and they were highly reliable when used to track fish variations at different hourly, daily, and monthly time scales. In addition, our methodology could be easily transferred to other cabled video-observatories.

New Vectorial Propulsion System and Trajectory Control Designs for Improved AUV Mission Autonomy

Autonomous Underwater Vehicles (AUV) are proving to be a promising platform design for multidisciplinary autonomous operability with a wide range of applications in marine ecology and geoscience. Here, two novel contributions towards increasing the autonomous navigation capability of a new AUV prototype (the Guanay II) as a mix between a propelled vehicle and a glider are presented. Firstly, a vectorial propulsion system has been designed to provide full vehicle maneuverability in both horizontal and vertical planes. Furthermore, two controllers have been designed, based on fuzzy controls, to provide the vehicle with autonomous navigation capabilities. Due to the decoupled system propriety, the controllers in the horizontal plane have been designed separately from the vertical plane. This class of non-linear controllers has been used to interpret linguistic laws into different zones of functionality. This method provided good performance, used as interpolation between different rules or linear controls. Both improvements have been validated through simulations and field tests, displaying good performance results. Finally, the conclusion of this work is that the Guanay II AUV has a solid controller to perform autonomous navigation and carry out vertical immersions.

Real-Time Seismic Data from the Bottom Sea

An anchored marine seismometer, acquiring real-time seismic data, has been built and tested. The system consists of an underwater seismometer, a surface buoy, and a mooring line that connects them. Inductive communication through the mooring line provides an inexpensive, reliable, and flexible solution. Prior to the deployment the dynamics of the system have been simulated numerically in order to find optimal materials, cables, buoys, and connections under critical marine conditions. The seismometer used is a high sensitivity triaxial broadband geophone able to measure low vibrational signals produced by the underwater seismic events. The power to operate the surface buoy is provided by solar panels. Additional batteries are needed for the underwater unit. In this paper we also present the first results and an earthquake detection of a prototype system that demonstrates the feasibility of this concept. The seismometer transmits continuous data at a rate of 1000 bps to a controller equipped with a radio link in the surface buoy. A GPS receiver on the surface buoy has been configured to perform accurate timestamps on the seismic data, which makes it possible to integrate the seismic data from these marine seismometers into the existing seismic network.

Middleware for Plug and Play Integration of Heterogeneous Sensor Resources into the Sensor Web

The study of global phenomena requires the combination of a considerable amount of data coming from different sources, acquired by different observation platforms and managed by institutions working in different scientific fields. Merging this data to provide extensive and complete data sets to monitor the long-term, global changes of our oceans is a major challenge. The data acquisition and data archival procedures usually vary significantly depending on the acquisition platform. This lack of standardization ultimately leads to information silos, preventing the data to be effectively shared across different scientific communities. In the past years, important steps have been taken in order to improve both standardization and interoperability, such as the Open Geospatial Consortium's Sensor Web Enablement (SWE) framework. Within this framework, standardized models and interfaces to archive, access and visualize the data from heterogeneous sensor resources have been proposed. However, due to the wide variety of software and hardware architectures presented by marine sensors and marine observation platforms, there is still a lack of uniform procedures to integrate sensors into existing SWE-based data infrastructures. In this work, a framework aimed to enable sensor plug and play integration into existing SWE-based data infrastructures is presented. First, an analysis of the operations required to automatically identify, configure and operate a sensor are analysed. Then, the metadata required for these operations is structured in a standard way. Afterwards, a modular, plug and play, SWE-based acquisition chain is proposed. Finally different use cases for this framework are presented.

A Multi-Objective Partition Method for Marine Sensor Networks Based on Degree of Event Correlation

Existing marine sensor networks acquire data from sea areas that are geographically divided, and store the data independently in their affiliated sea area data centers. In the case of marine events across multiple sea areas, the current network structure needs to retrieve data from multiple data centers, and thus severely affects real-time decision making. In this study, in order to provide a fast data retrieval service for a marine sensor network, we use all the marine sensors as the vertices, establish the edge based on marine events, and abstract the marine sensor network as a graph. Then, we construct a multi-objective balanced partition method to partition the abstract graph into multiple regions and store them in the cloud computing platform. This method effectively increases the correlation of the sensors and decreases the retrieval cost. On this basis, an incremental optimization strategy is designed to dynamically optimize existing partitions when new sensors are added into the network. Experimental results show that the proposed method can achieve the optimal layout for distributed storage in the process of disaster data retrieval in the China Sea area, and effectively optimize the result of partitions when new buoys are deployed, which eventually will provide efficient data access service for marine events.

Looking inside the Ocean: Toward an Autonomous Imaging System for Monitoring Gelatinous Zooplankton

Marine plankton abundance and dynamics in the open and interior ocean is still an unknown field. The knowledge of gelatinous zooplankton distribution is especially challenging, because this type of plankton has a very fragile structure and cannot be directly sampled using traditional net based techniques. To overcome this shortcoming, Computer Vision techniques can be successfully used for the automatic monitoring of this group.This paper presents the GUARD1 imaging system, a low-cost stand-alone instrument for underwater image acquisition and recognition of gelatinous zooplankton, and discusses the performance of three different methodologies, Tikhonov Regularization, Support Vector Machines and Genetic Programming, that have been compared in order to select the one to be run onboard the system for the automatic recognition of gelatinous zooplankton. The performance comparison results highlight the high accuracy of the three methods in gelatinous zooplankton identification, showing their good capability in robustly selecting relevant features. In particular, Genetic Programming technique achieves the same performances of the other two methods by using a smaller set of features, thus being the most efficient in avoiding computationally consuming preprocessing stages, that is a crucial requirement for running on an autonomous imaging system designed for long lasting deployments, like the GUARD1. The Genetic Programming algorithm has been installed onboard the system, that has been operationally tested in a two-months survey in the Ligurian Sea, providing satisfactory results in terms of monitoring and recognition performances.

A new colorimetrically-calibrated automated video-imaging protocol for day-night fish counting at the OBSEA coastal cabled observatory

Field measurements of the swimming activity rhythms of fishes are scant due to the difficulty of counting individuals at a high frequency over a long period of time. Cabled observatory video monitoring allows such a sampling at a high frequency over unlimited periods of time. Unfortunately, automation for the extraction of biological information (i.e., animals' visual counts per unit of time) is still a major bottleneck. In this study, we describe a new automated video-imaging protocol for the 24-h continuous counting of fishes in colorimetrically calibrated time-lapse photographic outputs, taken by a shallow water (20 m depth) cabled video-platform, the OBSEA. The spectral reflectance value for each patch was measured between 400 to 700 nm and then converted into standard RGB, used as a reference for all subsequent calibrations. All the images were acquired within a standardized Region Of Interest (ROI), represented by a 2 × 2 m methacrylate panel, endowed with a 9-colour calibration chart, and calibrated using the recently implemented "3D Thin-Plate Spline" warping approach in order to numerically define color by its coordinates in n-dimensional space. That operation was repeated on a subset of images, 500 images as a training set, manually selected since acquired under optimum visibility conditions. All images plus those for the training set were ordered together through Principal Component Analysis allowing the selection of 614 images (67.6%) out of 908 as a total corresponding to 18 days (at 30 min frequency). The Roberts operator (used in image processing and computer vision for edge detection) was used to highlights regions of high spatial colour gradient corresponding to fishes' bodies. Time series in manual and visual counts were compared together for efficiency evaluation. Periodogram and waveform analysis outputs provided very similar results, although quantified parameters in relation to the strength of respective rhythms were different. Results indicate that automation efficiency is limited by optimum visibility conditions. Data sets from manual counting present the larger day-night fluctuations in comparison to those derived from automation. This comparison indicates that the automation protocol subestimate fish numbers but it is anyway suitable for the study of community activity rhythms.

Biological data extraction from imagery - How far can we go? A case study from the Mid-Atlantic Ridge

In the past few decades, hydrothermal vent research has progressed immensely, resulting in higher-quality samples and long-term studies. With time, scientists are becoming more aware of the impacts of sampling on the faunal communities and are looking for less invasive ways to investigate the vent ecosystems. In this perspective, imagery analysis plays a very important role. With this study, we test which factors can be quantitatively and accurately assessed based on imagery, through comparison with faunal sampling. Twelve instrumented chains were deployed on the Atlantic Eiffel Tower hydrothermal edifice and the corresponding study sites were subsequently sampled. Discrete, quantitative samples were compared to the imagery recorded during the experiment. An observer-effect was tested, by comparing imagery data gathered by different scientists. Most factors based on image analyses concerning Bathymodiolus azoricus mussels were shown to be valid representations of the corresponding samples. Additional ecological assets, based exclusively on imagery, were included.

Ocean bottom seismometer: design and test of a measurement system for marine seismology

The Ocean Bottom Seismometer (OBS) is a key instrument for the geophysical study of sea sub-bottom layers. At present, more reliable autonomous instruments capable of recording underwater for long periods of time and therefore handling large data storage are needed. This paper presents a new Ocean Bottom Seismometer designed to be used in long duration seismic surveys. Power consumption and noise level of the acquisition system are the key points to optimize the autonomy and the data quality. To achieve our goals, a new low power data logger with high resolution and Signal–to-Noise Ratio (SNR) based on Compact Flash memory card is designed to enable continuous data acquisition. The equipment represents the achievement of joint work from different scientific and technological disciplines as electronics, mechanics, acoustics, communications, information technology, marine geophysics, etc. This easy to handle and sophisticated equipment allows the recording of useful controlled source and passive seismic data, as well as other time varying data, with multiple applications in marine environment research. We have been working on a series of prototypes for ten years to improve many of the aspects that make the equipment easy to handle and useful to work in deep-water areas. Ocean Bottom Seismometers (OBS) have received growing attention from the geoscience community during the last forty years. OBS sensors recording motion of the ocean floor hold key information in order to study offshore seismicity and to explore the Earth’s crust. In a seismic survey, a series of OBSs are placed on the seabed of the area under study, where they record either natural seismic activity or acoustic signals generated by compressed air-guns on the ocean surface. The resulting data sets are subsequently used to model both the earthquake locations and the crustal structure.

The new pelagic Operational Observatory of the Catalan Sea (OOCS) for the multisensor coordinated measurement of atmospheric and oceanographic conditions

The new pelagic Operational Observatory of the Catalan Sea (OOCS) for the coordinated multisensor measurement of atmospheric and oceanographic conditions has been recently installed (2009) in the Catalan Sea (41°39'N, 2°54'E; Western Mediterranean) and continuously operated (with minor maintenance gaps) until today. This multiparametric platform is moored at 192 m depth, 9.3 km off Blanes harbour (Girona, Spain). It is composed of a buoy holding atmospheric sensors and a set of oceanographic sensors measuring the water conditions over the upper 100 m depth. The station is located close to the head of the Blanes submarine canyon where an important multispecies pelagic and demersal fishery gives the station ecological and economic relevance. The OOCS provides important records on atmospheric and oceanographic conditions, the latter through the measurement of hydrological and biogeochemical parameters, at depths with a time resolution never attained before for this area of the Mediterranean. Twenty four moored sensors and probes operating in a coordinated fashion provide important data on Essential Ocean Variables (EOVs; UNESCO) such as temperature, salinity, pressure, dissolved oxygen, chlorophyll fluorescence, and turbidity. In comparison with other pelagic observatories presently operating in other world areas, OOCS also measures photosynthetic available radiation (PAR) from above the sea surface and at different depths in the upper 50 m. Data are recorded each 30 min and transmitted in real-time to a ground station via GPRS. This time series is published and automatically updated at the frequency of data collection on the official OOCS website (http://www.ceab.csic.es/~oceans). Under development are embedded automated routines for the in situ data treatment and assimilation into numerical models, in order to provide a reliable local marine processing forecast. In this work, our goal is to detail the OOCS multisensor architecture in relation to the coordinated capability for the remote, continuous and prolonged monitoring of atmospheric and oceanographic conditions, including data communication and storage. Accordingly, time series of measurements for a number of biological parameters will be presented for the summer months of 2011. Marine hindcast outputs from the numerical models implemented for simulating the conditions over the study area are shown. The strong changes of atmospheric conditions recorded in the last years over the area have altered the marine conditions of living organisms, but the dimension of the impact remains unclear. The OOCS multisensor coordinated monitoring has been specifically designed to address this issue, thus contributing to better understand the present environmental fluctuations and to provide a sound basis for a more accurate marine forecast system.

Automated image analysis for the detection of benthic crustaceans and bacterial mat coverage using the VENUS undersea cabled network

The development and deployment of sensors for undersea cabled observatories is presently biased toward the measurement of habitat variables, while sensor technologies for biological community characterization through species identification and individual counting are less common. The VENUS cabled multisensory network (Vancouver Island, Canada) deploys seafloor camera systems at several sites. Our objective in this study was to implement new automated image analysis protocols for the recognition and counting of benthic decapods (i.e., the galatheid squat lobster, Munida quadrispina), as well as for the evaluation of changes in bacterial mat coverage (i.e., Beggiatoa spp.), using a camera deployed in Saanich Inlet (103 m depth). For the counting of Munida we remotely acquired 100 digital photos at hourly intervals from 2 to 6 December 2009. In the case of bacterial mat coverage estimation, images were taken from 2 to 8 December 2009 at the same time frequency. The automated image analysis protocols for both study cases were created in MatLab 7.1. Automation for Munida counting incorporated the combination of both filtering and background correction (Median- and Top-Hat Filters) with Euclidean Distances (ED) on Red-Green-Blue (RGB) channels. The Scale-Invariant Feature Transform (SIFT) features and Fourier Descriptors (FD) of tracked objects were then extracted. Animal classifications were carried out with the tools of morphometric multivariate statistic (i.e., Partial Least Square Discriminant Analysis; PLSDA) on Mean RGB (RGBv) value for each object and Fourier Descriptors (RGBv+FD) matrices plus SIFT and ED. The SIFT approach returned the better results. Higher percentages of images were correctly classified and lower misclassification errors (an animal is present but not detected) occurred. In contrast, RGBv+FD and ED resulted in a high incidence of records being generated for non-present animals. Bacterial mat coverage was estimated in terms of Percent Coverage and Fractal Dimension. A constant Region of Interest (ROI) was defined and background extraction by a Gaussian Blurring Filter was performed. Image subtraction within ROI was followed by the sum of the RGB channels matrices. Percent Coverage was calculated on the resulting image. Fractal Dimension was estimated using the box-counting method. The images were then resized to a dimension in pixels equal to a power of 2, allowing subdivision into sub-multiple quadrants. In comparisons of manual and automated Percent Coverage and Fractal Dimension estimates, the former showed an overestimation tendency for both parameters. The primary limitations on the automatic analysis of benthic images were habitat variations in sediment texture and water column turbidity. The application of filters for background corrections is a required preliminary step for the efficient recognition of animals and bacterial mat patches.