Laser system for identification, tracking, and control of flying insects

Revealing complex mosquito behaviour: a review of current automated video tracking systems suitable for tracking mosquitoes in the field

Mosquito-borne pathogens continue to cause tremendous suffering, morbidity and mortality. For many of these diseases, vector control remains the most effective approach. The development and deployment of effective and efficient mosquito control products and strategies require a profound understanding of mosquito behaviour. To study complex mosquito behaviour, automated video tracking of mosquito flight paths has proven to be a comprehensive approach, and several video tracking approaches have emerged in recent years, making the choice for a suitable system challenging. Here, we conducted a literature review by searching PubMed and Google Scholar, and we identified 66 publications focusing on mosquito video tracking, which made use of eight different systems. We then compared and scored those video tracking systems by assessing their performance in the laboratory as well as their potential suitability for tracking mosquito behaviour in a field setting. While all eight systems have produced valuable information on mosquito behaviour, for tracking mosquitoes in the field, 'Braid', 'EthoVision XT' and 'Trackit3D' appear to be the most suitable systems as they need small disk capacity and are well adaptable to different settings. However, the optimal choice will ultimately depend on the specifications required to answer a given research question, the financial resources available and user preferences.

Monitoring Mosquito Abundance: Comparing an Optical Sensor with a Trapping Method

Optical sensors have shown significant promise in offering additional data to track insect populations. This article presents a comparative study between abundance measurements obtained from a novel near-infrared optical sensor and physical traps. The optical instrument, named an Entomological Bistatic Optical Sensor System, or eBoss, is a non-destructive sensor operating in the near-infrared spectral range and designed to continuously monitor the population of flying insects. The research compares the mosquito aerial density (#/m3) obtained through the eBoss with trap counts from eight physical traps during an eight-month field study. The eBoss recorded over 302,000 insect sightings and assessed the aerial density of all airborne insects as well as male and female mosquitoes specifically with a resolution of one minute. This capability allows for monitoring population trends throughout the season as well as daily activity peaks. The results affirmed the correlation between the two methods. While optical instruments do not match traps in terms of taxonomic accuracy, the eBoss offered greater temporal resolution (one minute versus roughly three days) and statistical significance owing to its much larger sample size. These outcomes further indicate that entomological optical sensors can provide valuable complementary data to more common methods to monitor flying insect populations, such as mosquitoes or pollinators.

Laser weeding of common weed species

The massive use of herbicides since the 1950s has resulted in increasing problems with herbicideresistant weeds and pollution of the environment, including food, feed, and water. These side effects have resulted in political pressures to reduce herbicide application. The European Commission aims to reduce the use and risk of chemicals and more hazardous pesticides in the EU. Therefore, new weed control methods are in demand. Laser weeding might be an alternative to replace or supplement herbicides and other weed control methods in an Integrated Weed Management (IPM) strategy. This work aimed to investigate how increasing laser energy affected common weeds when the apical meristem was exposed to irradiation at the early stages of development. A 50 W thulium-doped fibre laser with a diameter of 2 mm and a wavelength of 2 µm was used. The highest efficacy of laser irradiation was achieved when the grass weed (Alopecurus myosuroides) had one leaf and the dicot species were at the cotyledon stage. There was a large difference between the species' susceptibility to irradiation probably caused by differences in morphology and growth habit. At the 4-leaf stage, most of the species regrew after irradiation. Laser weeding may be a solution to replace or supplement other weed control methods in some crops, but in general the weeds must be irradiated when they are at the cotyledon to 2-leaf stage to avoid regrowth.

Measuring the Flight Trajectory of a Free-Flying Moth on the Basis of Noise-Reduced 3D Point Cloud Time Series Data

Pest control is crucial in crop production; however, the use of chemical pesticides, the primary method of pest control, poses environmental issues and leads to insecticide resistance in pests. To overcome these issues, laser zapping has been studied as a clean pest control technology against the nocturnal cotton leafworm, Spodoptera litura, which has high fecundity and causes severe damage to various crops. For better sighting during laser zapping, it is important to measure the coordinates and speed of moths under low-light conditions. To achieve this, we developed an automatic detection pipeline based on point cloud time series data from stereoscopic images. We obtained 3D point cloud data from disparity images recorded under infrared and low-light conditions. To identify S. litura, we removed noise from the data using multiple filters and a support vector machine. We then computed the size of the outline box and directional angle of the 3D point cloud time series to determine the noisy point clouds. We visually inspected the flight trajectories and found that the size of the outline box and the movement direction were good indicators of noisy data. After removing noisy data, we obtained 68 flight trajectories, and the average flight speed of free-flying S. litura was 1.81 m/s.

Temperature Dependency of Insect's Wingbeat Frequencies: An Empirical Approach to Temperature Correction

This study examines the relationship between the wingbeat frequency of flying insects and ambient temperature, leveraging data from over 302,000 insect observations obtained using a near-infrared optical sensor during an eight-month field experiment. By measuring the wingbeat frequency as well as wing and body optical cross-sections of each insect in conjunction with the ambient temperature, we identified five clusters of insects and analyzed how their average wingbeat frequencies evolved over temperatures ranging from 10 °C to 38 °C. Our findings reveal a positive correlation between temperature and wingbeat frequency, with a more pronounced increase observed at higher wingbeat frequencies. Frequencies increased on average by 2.02 Hz/°C at 50 Hz, and up to 9.63 Hz/°C at 525 Hz, and a general model is proposed. This model offers a valuable tool for correcting wingbeat frequencies with temperature, enhancing the accuracy of insect clustering by optical and acoustic sensors. While this approach does not account for species-specific responses to temperature changes, our research provides a general insight, based on all species present during the field experiment, into the intricate dynamics of insect flight behavior in relation to environmental factors.

An optical system to detect, surveil, and kill flying insect vectors of human and crop pathogens

Sustainable and effective means to control flying insect vectors are critically needed, especially with widespread insecticide resistance and global climate change. Understanding and controlling vectors requires accurate information about their movement and activity, which is often lacking. The Photonic Fence (PF) is an optical system that uses machine vision, infrared light, and lasers to identify, track, and interdict vectors in flight. The PF examines an insect's outline, flight speed, and other flight parameters and if these match those of a targeted vector species, then a low-power, retina-safe laser kills it. We report on proof-of-concept tests of a large, field-sized PF (30 mL × 3 mH) conducted with Aedes aegypti, a mosquito that transmits dangerous arboviruses, and Diaphorina citri, a psyllid which transmits the fatal huanglongbing disease of citrus. In tests with the laser engaged, < 1% and 3% of A. aegypti and D. citri, respectfully, were recovered versus a 38% and 19% recovery when the lacer was silenced. The PF tracked, but did not intercept the orchid bee, Euglossa dilemma. The system effectively intercepted flying vectors, but not bees, at a distance of 30 m, heralding the use of photonic energy, rather than chemicals, to control flying vectors.

Automated detection of the yellow-legged hornet (Vespa velutina) using an optical sensor with machine learning

BACKGROUND

The yellow-legged hornet (Vespa velutina) is native to Southeast Asia and is an invasive alien species of concern in many countries. More effective management of populations of V. velutina could be achieved through more widespread and intensive monitoring in the field, however current methods are labor intensive and costly. To address this issue, we have assessed the performance of an optical sensor combined with a machine learning model to classify V. velutina and native wasps/hornets and bees. Our aim is to use the results of the present work as a step towards the development of a monitoring solution for V. velutina in the field.

RESULTS

We recorded a total 935 flights from three bee species: Apis mellifera, Bombus terrestris and Osmia bicornis; and four wasp/hornet species: Polistes dominula, Vespula germanica, Vespa crabro and V. velutina. The machine learning model achieved an average accuracy for species classification of 80.1 ± 13.9% and 74.5 ± 7.0% for V. velutina. V. crabro had the highest level of misclassification, confused mainly with V. velutina and P. dominula. These results were obtained using a 14-value peak and valley feature derived from the wingbeat power spectral density.

CONCLUSION

This study demonstrates that the wingbeat recordings from a flying insect sensor can be used with machine learning methods to differentiate V. velutina from six other Hymenoptera species in the laboratory and this knowledge could be used to help develop a tool for use in integrated invasive alien species management programs. © 2022 The Authors. Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry.

A novel optical sensor system for the automatic classification of mosquitoes by genus and sex with high levels of accuracy

Background

Every year, more than 700,000 people die from vector-borne diseases, mainly transmitted by mosquitoes. Vector surveillance plays a major role in the control of these diseases and requires accurate and rapid taxonomical identification. New approaches to mosquito surveillance include the use of acoustic and optical sensors in combination with machine learning techniques to provide an automatic classification of mosquitoes based on their flight characteristics, including wingbeat frequency. The development and application of these methods could enable the remote monitoring of mosquito populations in the field, which could lead to significant improvements in vector surveillance.

Methods

A novel optical sensor prototype coupled to a commercial mosquito trap was tested in laboratory conditions for the automatic classification of mosquitoes by genus and sex. Recordings of > 4300 laboratory-reared mosquitoes of Aedes and Culex genera were made using the sensor. The chosen genera include mosquito species that have a major impact on public health in many parts of the world. Five features were extracted from each recording to form balanced datasets and used for the training and evaluation of five different machine learning algorithms to achieve the best model for mosquito classification.

Results

The best accuracy results achieved using machine learning were: 94.2% for genus classification, 99.4% for sex classification of Aedes, and 100% for sex classification of Culex. The best algorithms and features were deep neural network with spectrogram for genus classification and gradient boosting with Mel Frequency Cepstrum Coefficients among others for sex classification of either genus.

Conclusions

To our knowledge, this is the first time that a sensor coupled to a standard mosquito suction trap has provided automatic classification of mosquito genus and sex with high accuracy using a large number of unique samples with class balance. This system represents an improvement of the state of the art in mosquito surveillance and encourages future use of the sensor for remote, real-time characterization of mosquito populations.

Graphical abstract

Supplementary Information

The online version contains supplementary material available at 10.1186/s13071-022-05324-5.

Wearable Vibration Sensor for Measuring the Wing Flapping of Insects

In this study, we fabricated a novel wearable vibration sensor for insects and measured their wing flapping. An analysis of insect wing deformation in relation to changes in the environment plays an important role in understanding the underlying mechanism enabling insects to dynamically interact with their surrounding environment. It is common to use a high-speed camera to measure the wing flapping; however, it is difficult to analyze the feedback mechanism caused by the environmental changes caused by the flapping because this method applies an indirect measurement. Therefore, we propose the fabrication of a novel film sensor that is capable of measuring the changes in the wingbeat frequency of an insect. This novel sensor is composed of flat silver particles admixed with a silicone polymer, which changes the value of the resistor when a bending deformation occurs. As a result of attaching this sensor to the wings of a moth and a dragonfly and measuring the flapping of the wings, we were able to measure the frequency of the flapping with high accuracy. In addition, as a result of simultaneously measuring the relationship between the behavior of a moth during its search for an odor source and its wing flapping, it became clear that the frequency of the flapping changed depending on the frequency of the odor reception. From this result, a wearable film sensor for an insect that can measure the displacement of the body during a particular behavior was fabricated.

Optical tracking and laser-induced mortality of insects during flight

Addressing the need for novel insect observation and control tools, the Photonic Fence detects and tracks mosquitoes and other flying insects and can apply lethal doses of laser light to them. Previously, we determined lethal exposure levels for a variety of lasers and pulse conditions on anesthetized Anopheles stephensi mosquitoes. In this work, similar studies were performed while the subjects were freely flying within transparent cages two meters from the optical system; a proof-of-principle demonstration of a 30 m system was also performed. From the dose–response curves of mortality data created as a function of various beam diameter, pulse width, and power conditions at visible and near-infrared wavelengths, the visible wavelengths required significantly lower laser exposure than near infrared wavelengths to disable subjects, though near infrared sources remain attractive given their cost and retina safety. The flight behavior of the subjects and the performance of the tracking system were found to have no impact on the mortality outcomes for pulse durations up to 25 ms, which appears to be the ideal duration to minimize required laser power. The results of this study affirm the practicality of using optical approaches to protect people and crops from pestilent flying insects.

Automated moth flight analysis in the vicinity of artificial light

Instrumentation and software for the automated analysis of insect flight trajectories is described, intended for quantifying the behavioural dynamics of moths in the vicinity of artificial light. For its time, this moth imaging system was relatively advanced and revealed hitherto undocumented insights into moth flight behaviour. The illumination source comprised a 125 W mercury vapour light, operating in the visible and near ultraviolet wavelengths, mounted on top of a mobile telescopic mast at heights of 5 and 7.1 m, depending upon the experiment. Moths were imaged in early September, at night and in field conditions, using a ground level video camera with associated optics including a heated steering mirror, wide angle lens and an electronic image intensifier. Moth flight coordinates were recorded at a rate of 50 images per second (fields) and transferred to a computer using a light pen (the only non-automated operation in the processing sequence). Software extracted ground speed vectors and, by instantaneous subtraction of wind speed data supplied by fast-response anemometers, the airspeed vectors. Accumulated density profiles of the track data revealed that moths spend most of their time at a radius of between 40 and 50 cm from the source, and rarely fly directly above it, from close range. Furthermore, the proportion of insects caught by the trap as a proportion of the number influenced by the light (and within the field of view of the camera) was very low; of 1600 individual tracks recorded over five nights, a total of only 12 were caught. Although trap efficiency is strongly dependent on trap height, time of night, season, moonlight and weather, the data analysis confirmed that moths do not exhibit straightforward positive phototaxis. In general, trajectory patterns become more complex with reduced distance from the illumination, with higher recorded values of speeds and angular velocities. However, these characteristics are further qualified by the direction of travel of the insect; the highest accelerations tended to occur when the insect was at close range, but moving away from the source. Rather than manifesting a simple positive phototaxis, the trajectories were suggestive of disorientation. Based on the data and the complex behavioural response, mathematical models were developed that described ideal density distribution in calm air and light wind speed conditions. The models did not offer a physiological hypothesis regarding the behavioural changes, but rather were tools for quantification and prediction. Since the time that the system was developed, instrumentation, computers and software have advanced considerably, allowing much more to be achieved at a small fraction of the original cost. Nevertheless, the analytical tools remain useful for automated trajectory analysis of airborne insects.

Collective behavior quantification on human odor effects against female Aedes aegypti mosquitoes-Open source development

Classifying and quantifying mosquito activity includes a plethora of categories, ranging from measuring flight speeds, repellency, feeding rates, and specific behaviors such as home entry, swooping and resting, among others. Entomologists have been progressing more toward using machine vision for efficiency for this endeavor. Digital methods have been used to study the behavior of insects in labs, for instance via three-dimensional tracking with specialized cameras to observe the reaction of mosquitoes towards human odor, heat and CO₂, although virtually none was reported for several important fields, such as repellency studies which have a significant need for a proper response quantification. However, tracking mosquitoes individually is a challenge and only limited number of specimens can be studied. Although tracking large numbers of individual insects is hailed as one of the characteristics of an ideal automated image-based tracking system especially in 3D, it also is a costly method, often requiring specialized hardware and limited access to the algorithms used for mapping the specimens. The method proposed contributes towards (a) unlimited open source use, (b) a low-cost setup, (c) complete guide for any entomologist to adapt in terms of hardware and software, (d) simple to use, and (e) a lightweight data output for collective behavior analysis of mosquitoes. The setup is demonstrated by testing a simple response of mosquitoes in the presence of human odor versus control, one session with continuous human presence as a stimuli and the other with periodic presence. A group of female Aedes aegypti (Linnaeus) mosquitoes are released into a white-background chamber with a transparent acrylic panel on one side. The video feed of the mosquitoes are processed using filtered contours in a threshold-adjustable video. The mosquitoes in the chamber are mapped on the raster where the coordinates of each mosquito are recorded with the corresponding timestamp. The average distance of the blobs within the frames against time forms a spectra where behavioral patterns can be observed directly, whether any collective effect is observed. With this method, 3D tracking will not be required and a more straightforward data output can be obtained.

Automated Surveillance of Fruit Flies

Insects of the Diptera order of the Tephritidae family cause costly, annual crop losses worldwide. Monitoring traps are important components of integrated pest management programs used against fruit flies. Here we report the modification of typical, low-cost plastic traps for fruit flies by adding the necessary optoelectronic sensors to monitor the entrance of the trap in order to detect, time-stamp, GPS tag, and identify the species of incoming insects from the optoacoustic spectrum analysis of their wingbeat. We propose that the incorporation of automated streaming of insect counts, environmental parameters and GPS coordinates into informative visualization of collective behavior will finally enable better decision making across spatial and temporal scales, as well as administrative levels. The device presented is at product level of maturity as it has solved many pending issues presented in a previously reported study.