Deep Learning in Plant Phenological Research: A Systematic Literature Review

Digital Repeat Photography Application for Flowering Stage Classification of Selected Woody Plants

Digital repeat photography is currently applied mainly in geophysical studies of ecosystems. However, its role as a tool that can be utilized in conventional phenology, tracking a plant's seasonal developmental cycle, is growing. This study's main goal was to develop an easy-to-reproduce, single-camera-based novel approach to determine the flowering phases of 12 woody plants of various deciduous species. Field observations served as binary class calibration datasets (flowering and non-flowering stages). All the image RGB parameters, designated for each plant separately, were used as plant features for the models' parametrization. The training data were subjected to various transformations to achieve the best classifications using the weighted k-nearest neighbors algorithm. The developed models enabled the flowering classifications at the 0, 1, 2, 3, and 5 onset day shift (absolute values) for 2, 3, 3, 2, and 2 plants, respectively. For 9 plants, the presented method enabled the flowering duration estimation, which is a valuable yet rarely used parameter in conventional phenological studies. We found the presented method suitable for various plants, despite their petal color and flower size, until there is a considerable change in the crown color during the flowering stage.

Bridging technology and ecology: enhancing applicability of deep learning and UAV-based flower recognition

The decline of insect biomass, including pollinators, represents a significant ecological challenge, impacting both biodiversity and ecosystems. Effective monitoring of pollinator habitats, especially floral resources, is essential for addressing this issue. This study connects drone and deep learning technologies to their practical application in ecological research. It focuses on simplifying the application of these technologies. Updating an object detection toolbox to TensorFlow (TF) 2 enhanced performance and ensured compatibility with newer software packages, facilitating access to multiple object recognition models - Faster Region-based Convolutional Neural Network (Faster R-CNN), Single-Shot-Detector (SSD), and EfficientDet. The three object detection models were tested on two datasets of UAV images of flower-rich grasslands, to evaluate their application potential in practice. A practical guide for biologists to apply flower recognition to Unmanned Aerial Vehicle (UAV) imagery is also provided. The results showed that Faster RCNN had the best overall performance with a precision of 89.9% and a recall of 89%, followed by EfficientDet, which excelled in recall but at a lower precision. Notably, EfficientDet demonstrated the lowest model complexity, making it a suitable choice for applications requiring a balance between efficiency and detection performance. Challenges remain, such as detecting flowers in dense vegetation and accounting for environmental variability.

Advancing plant biology through deep learning-powered natural language processing

The application of deep learning methods, specifically the utilization of Large Language Models (LLMs), in the field of plant biology holds significant promise for generating novel knowledge on plant cell systems. The LLM framework exhibits exceptional potential, particularly with the development of Protein Language Models (PLMs), allowing for in-depth analyses of nucleic acid and protein sequences. This analytical capacity facilitates the discernment of intricate patterns and relationships within biological data, encompassing multi-scale information within DNA or protein sequences. The contribution of PLMs extends beyond mere sequence patterns and structure--function recognition; it also supports advancements in genetic improvements for agriculture. The integration of deep learning approaches into the domain of plant sciences offers opportunities for major breakthroughs in basic research across multi-scale plant traits. Consequently, the strategic application of deep learning methodologies, particularly leveraging the potential of LLMs, will undoubtedly play a pivotal role in advancing plant sciences, plant production, plant uses and propelling the trajectory toward sustainable agroecological and agro-food transitions.

Multispecies deep learning using citizen science data produces more informative plant community models

In the age of big data, scientific progress is fundamentally limited by our capacity to extract critical information. Here, we map fine-grained spatiotemporal distributions for thousands of species, using deep neural networks (DNNs) and ubiquitous citizen science data. Based on 6.7 M observations, we jointly model the distributions of 2477 plant species and species aggregates across Switzerland with an ensemble of DNNs built with different cost functions. We find that, compared to commonly-used approaches, multispecies DNNs predict species distributions and especially community composition more accurately. Moreover, their design allows investigation of understudied aspects of ecology. Including seasonal variations of observation probability explicitly allows approximating flowering phenology; reweighting predictions to mirror cover-abundance allows mapping potentially canopy-dominant tree species nationwide; and projecting DNNs into the future allows assessing how distributions, phenology, and dominance may change. Given their skill and their versatility, multispecies DNNs can refine our understanding of the distribution of plants and well-sampled taxa in general.

Enhancing Apple Cultivar Classification Using Multiview Images

Apple cultivar classification is challenging due to the inter-class similarity and high intra-class variations. Human experts do not rely on single-view features but rather study each viewpoint of the apple to identify a cultivar, paying close attention to various details. Following our previous work, we try to establish a similar multiview approach for machine-learning (ML)-based apple classification in this paper. In our previous work, we studied apple classification using one single view. While these results were promising, it also became clear that one view alone might not contain enough information in the case of many classes or cultivars. Therefore, exploring multiview classification for this task is the next logical step. Multiview classification is nothing new, and we use state-of-the-art approaches as a base. Our goal is to find the best approach for the specific apple classification task and study what is achievable with the given methods towards our future goal of applying this on a mobile device without the need for internet connectivity. In this study, we compare an ensemble model with two cases where we use single networks: one without view specialization trained on all available images without view assignment and one where we combine the separate views into a single image of one specific instance. The two latter options reflect dataset organization and preprocessing to allow the use of smaller models in terms of stored weights and number of operations than an ensemble model. We compare the different approaches based on our custom apple cultivar dataset. The results show that the state-of-the-art ensemble provides the best result. However, using images with combined views shows a decrease in accuracy by 3% while requiring only 60% of the memory for weights. Thus, simpler approaches with enhanced preprocessing can open a trade-off for classification tasks on mobile devices.

Opportunistic plant observations reveal spatial and temporal gradients in phenology

Opportunistic plant records provide a rapidly growing source of spatiotemporal plant observation data. Here, we used such data to explore the question whether they can be used to detect changes in species phenologies. Examining 19 herbaceous and one woody plant species in two consecutive years across Europe, we observed significant shifts in their flowering phenology, being more pronounced for spring-flowering species (6-17 days) compared to summer-flowering species (1-6 days). Moreover, we show that these data are suitable to model large-scale relationships such as "Hopkins' bioclimatic law" which quantifies the phenological delay with increasing elevation, latitude, and longitude. Here, we observe spatial shifts, ranging from -5 to 50 days per 1000 m elevation to latitudinal shifts ranging from -1 to 4 days per degree northwards, and longitudinal shifts ranging from -1 to 1 day per degree eastwards, depending on the species. Our findings show that the increasing volume of purely opportunistic plant observation data already provides reliable phenological information, and therewith can be used to support global, high-resolution phenology monitoring in the face of ongoing climate change.

Deep learning to extract the meteorological by-catch of wildlife cameras

Microclimate-proximal climatic variation at scales of metres and minutes-can exacerbate or mitigate the impacts of climate change on biodiversity. However, most microclimate studies are temperature centric, and do not consider meteorological factors such as sunshine, hail and snow. Meanwhile, remote cameras have become a primary tool to monitor wild plants and animals, even at micro-scales, and deep learning tools rapidly convert images into ecological data. However, deep learning applications for wildlife imagery have focused exclusively on living subjects. Here, we identify an overlooked opportunity to extract latent, ecologically relevant meteorological information. We produce an annotated image dataset of micrometeorological conditions across 49 wildlife cameras in South Africa's Maloti-Drakensberg and the Swiss Alps. We train ensemble deep learning models to classify conditions as overcast, sunshine, hail or snow. We achieve 91.7% accuracy on test cameras not seen during training. Furthermore, we show how effective accuracy is raised to 96% by disregarding 14.1% of classifications where ensemble member models did not reach a consensus. For two-class weather classification (overcast vs. sunshine) in a novel location in Svalbard, Norway, we achieve 79.3% accuracy (93.9% consensus accuracy), outperforming a benchmark model from the computer vision literature (75.5% accuracy). Our model rapidly classifies sunshine, snow and hail in almost 2 million unlabelled images. Resulting micrometeorological data illustrated common seasonal patterns of summer hailstorms and autumn snowfalls across mountains in the northern and southern hemispheres. However, daily patterns of sunshine and shade diverged between sites, impacting daily temperature cycles. Crucially, we leverage micrometeorological data to demonstrate that (1) experimental warming using open-top chambers shortens early snow events in autumn, and (2) image-derived sunshine marginally outperforms sensor-derived temperature when predicting bumblebee foraging. These methods generate novel micrometeorological variables in synchrony with biological recordings, enabling new insights from an increasingly global network of wildlife cameras.

Bridging the gap: how to adopt opportunistic plant observations for phenology monitoring

Plant phenology plays a vital role in assessing climate change. To monitor this, individual plants are traditionally visited and observed by trained volunteers organized in national or international networks - in Germany, for example, by the German Weather Service, DWD. However, their number of observers is continuously decreasing. In this study, we explore the feasibility of using opportunistically captured plant observations, collected via the plant identification app Flora Incognita to determine the onset of flowering and, based on that, create interpolation maps comparable to those of the DWD. Therefore, the opportunistic observations of 17 species collected in 2020 and 2021 were assigned to "Flora Incognita stations" based on location and altitude in order to mimic the network of stations forming the data basis for the interpolation conducted by the DWD. From the distribution of observations, the percentile representing onset of flowering date was calculated using a parametric bootstrapping approach and then interpolated following the same process as applied by the DWD. Our results show that for frequently observed, herbaceous and conspicuous species, the patterns of onset of flowering were similar and comparable between both data sources. We argue that a prominent flowering stage is crucial for accurately determining the onset of flowering from opportunistic plant observations, and we discuss additional factors, such as species distribution, location bias and societal events contributing to the differences among species and phenology data. In conclusion, our study demonstrates that the phenological monitoring of certain species can benefit from incorporating opportunistic plant observations. Furthermore, we highlight the potential to expand the taxonomic range of monitored species for phenological stage assessment through opportunistic plant observation data.

Crop-saving with AI: latest trends in deep learning techniques for plant pathology

Plant diseases pose a major threat to agricultural production and the food supply chain, as they expose plants to potentially disruptive pathogens that can affect the lives of those who are associated with it. Deep learning has been applied in a range of fields such as object detection, autonomous vehicles, fraud detection etc. Several researchers have tried to implement deep learning techniques in precision agriculture. However, there are pros and cons to the approaches they have opted for disease detection and identification. In this survey, we have made an attempt to capture the significant advancements in machine-learning based disease detection. We have discussed prevalent datasets and techniques that have been employed as well as highlighted emerging approaches being used for plant disease detection. By exploring these advancements, we aim to present a comprehensive overview of the prominent approaches in precision agriculture, along with their associated challenges and potential improvements. This paper delves into the challenges associated with the implementation and briefly discusses the future trends. Overall, this paper presents a bird's eye view of plant disease datasets, deep learning techniques, their accuracies and the challenges associated with them. Our insights will serve as a valuable resource for researchers and practitioners in the field. We hope that this survey will inform and inspire future research efforts, ultimately leading to improved precision agriculture practices and enhanced crop health management.

Geometric Morphometric Versus Genomic Patterns in a Large Polyploid Plant Species Complex

Plant species complexes represent a particularly interesting example of taxonomically complex groups (TCGs), linking hybridization, apomixis, and polyploidy with complex morphological patterns. In such TCGs, mosaic-like character combinations and conflicts of morphological data with molecular phylogenies present a major problem for species classification. Here, we used the large polyploid apomictic European Ranunculus auricomus complex to study relationships among five diploid sexual progenitor species and 75 polyploid apomictic derivate taxa, based on geometric morphometrics using 11,690 landmarked objects (basal and stem leaves, receptacles), genomic data (97,312 RAD-Seq loci, 48 phased target enrichment genes, 71 plastid regions) from 220 populations. We showed that (1) observed genomic clusters correspond to morphological groupings based on basal leaves and concatenated traits, and morphological groups were best resolved with RAD-Seq data; (2) described apomictic taxa usually overlap within trait morphospace except for those taxa at the space edges; (3) apomictic phenotypes are highly influenced by parental subgenome composition and to a lesser extent by climatic factors; and (4) allopolyploid apomictic taxa, compared to their sexual progenitor, resemble a mosaic of ecological and morphological intermediate to transgressive biotypes. The joint evaluation of phylogenomic, phenotypic, reproductive, and ecological data supports a revision of purely descriptive, subjective traditional morphological classifications.