Opportunities for improving the rigor of management effectiveness evaluations in protected areas

Introducing Elinor for monitoring the governance and management of area-based conservation

Monitoring the governance and management effectiveness of area-based conservation has long been recognized as an important foundation for achieving national and global biodiversity goals and enabling adaptive management. However, there are still many barriers that prevent conservation actors, including those affected by governance and management systems from implementing conservation activities and programs and from gathering and using data on governance and management to inform decision-making across spatial scales and through time. We explored current and past efforts to assess governance and management effectiveness and barriers actors face in using the resulting data and insights to inform conservation decision-making. To help overcome these barriers, we developed Elinor, a free and open-source monitoring tool that builds on the work of Nobel Prize winner Elinor Ostrom to facilitate the gathering, storing, sharing, analyzing, and use of data on environmental governance and management across spatial scales and for areas under different governance and management types. We consider the process of codesigning and piloting Elinor with conservation scientists and practitioners and the main components of the assessment and online data system. We also consider how Elinor complements existing approaches by addressing governance and management in a single assessment at a high level for different types of area-based conservation, providing flexible options for data collection, and integrating a data system with an assessment that can support data use and sharing across different spatial scales, including global monitoring of the Global Biodiversity Framework. Although challenges will continue, the process of developing Elinor and the tool itself offer tangible solutions to barriers that prevent the systematic collection and use of governance and management data. With broader uptake, Elinor can play a valuable role in enabling more effective, inclusive, and durable area-based conservation.

Progress developing the concept of other effective area-based conservation measures

In 2010, the introduction of other effective area-based conservation measures (OECMs) into international policy caused a paradigm shift in area-based conservation, which included consideration of areas outside formal protected areas and places where biodiversity conservation may not be a management objective for the site. Despite the importance of this shift for global conservation, conservation science and policy have been slow to engage with the concept of OECMs. As the world moves toward protecting 30% of the Earth by 2030, it is imperative to develop evidence-based guidance for how to identify effective conservation measures, especially tools to help evaluate and monitor the biodiversity outcomes associated with potential OECMs. To understand the current progress in developing the concept of OECMs, I evaluated the peer-reviewed literature to consolidate and synthesize current knowledge. I conducted a thematic analysis of papers to identify the types of challenges and opportunities being discussed and lessons from studies evaluating the effectiveness of OECMs. Only 105 studies mentioned OECMs, and those that did rarely move beyond superficial mention of OECMs as part of area-based conservation. Around one-half of studies listed potential risks or benefits of OECMs but none provided evidence these issues have materialized. Twenty-three studies attempted to identify potential OECMs, although specific case studies were rare. The 7 studies that evaluated existing OECMs were highly critical of how they had been implemented to date. Studies that evaluated conservation outcomes were extremely rare, and suggested effectiveness must be judged on a case-by-case basis. The current literature not only leaves many gaps in the science required to operationalize the concept of OECMs, but also often raises additional questions that need to be addressed. If these gaps are not filled by robust science, the promised benefits for biodiversity from OECMs may never be realized.

Exploring evolution characteristics of eco-environment quality in the Yangtze River Basin based on remote sensing ecological index

As an important ecological-economic development area in China, scientific understanding of the spatial and temporal changes in eco-environment quality (EEQ) and its drivers in the Yangtze River Basin (YRB) is crucial for the effective implementation of ecological protection projects in the YRB. To address the lack of large-scale EEQ assessment in the YRB, this paper uses the Google Earth Engine (GEE) platform and the Remote Sensing Ecological Index (RSEI) to investigate the spatial and temporal characteristics of EEQ in the YRB from 2000 to 2020, and to analyze the impact of various factors on the EEQ of the YRB. This study showed that: (1) The overall EEQ of YRB was at the 'good' grade over the past 20 years, showing an increasing trend, with the value changing from 0.70 to 0.77. (2) The YRB's EEQ has positive spatial aggregation characteristics, with the northern part of the Jialing River basin and the Han River basin exhibiting a high-high aggregation type and the upper reaches exhibiting a low-low aggregation type. (3) In the past 20 years, the human activities had a greater impact on the EEQ of the YRB; moreover, all factors had a greater impact on the EEQ than a single factor. The interaction between the biological abundance index and population density had the most effect, with a q-value of 0.737 in 2020.

Defining performance thresholds for effective management of biodiversity within protected areas

Performance thresholds are an important tool for determining successful conservation outcomes. They provide an objective means of defining good ecological condition and have been endorsed as an essential part of best practice in protected area (PA) management within the International Union for the Conservation of Nature Green List of Protected and Conserved Areas Standard. With a growing number of PAs attaining Green List status globally, thresholds developed by PAs on the Green List present an excellent resource with which to identify the attributes of well-defined performance thresholds. We examined 349 thresholds associated with PAs on the Green List to determine whether they were specific and measurable (i.e., factors recognized as essential for setting well-defined targets). We assessed whether thresholds were defined quantitatively and whether definitions included ambiguous terms (e.g., stable numbers). We identified six different ways thresholds were expressed and found that many thresholds were expressed as management objectives, rather than ecological condition thresholds, although this trend improved over time. Approximately one-half of the performance thresholds lacked the necessary specificity to delineate successful outcomes. Our results enabled us to develop a checklist of information required to set robust performance thresholds. Recommendations include that thresholds should be quantitatively defined, including quantitative estimates of the limits of acceptable change (LAC) around the target condition. To ensure transparency, a rationale and associated evidence should be provided to support the threshold and the LAC. When accompanied by a rationale and quantitative estimate of the current condition of the value, unambiguously defined thresholds with a quantitative LAC provide an objective means of demonstrating that successful conservation outcomes have been achieved. These recommendations will help conservation managers apply the Green List Standard and improve the measurement of conservation outcomes more broadly.

BREMi-A New Tool for the Evaluation of UNESCO Biosphere Reserve Management Effectiveness: Case-study in the Arab Man and Biosphere (ArabMAB) Regional Network

Scholars and practitioners have been striving to develop straightforward and effective tools to measure protected area management effectiveness (PAME). UNESCO Biosphere Reserves (BR), with their unique functional and zonation schemes are monitored according to their compulsory 10-year Periodic Review (PR), which is useful for UNESCO's evaluation purposes but lacks comprehensiveness and utility for adaptive management. Based on existing PAME methodologies, we develop and propose the first quantitative tool for the evaluation of BR management effectiveness, that would enhance and complement the currently used qualitative PR report, and serve the rapid evaluation needed for BR managers to monitor, evaluate, and adapt their management approach to achieve the three functions of BRs. The tool consists of 65 indicators, embodied within the 6 elements of the World Commission on Protected Areas Framework. We then tested this tool, named Biosphere Reserve Effectiveness of Management index (BREMi) to evaluate management effectiveness across the Arab Man and the Biosphere Reserve network involving 17 BRs spanning 8 countries of the Middle East and North Africa. BREMi scores ranged from 4.43 to 8.65 (on a scale between 0 and 10), with a mean of 6.31 ± 1.040. All indicators were considered valuable measures of progress by our respondents, as well as by independent experts. We discuss our findings in light of available literature concerning the Arab region and through the conceptual frames of adaptive management and resilience. Finally, we discuss where the BREMi tool would be most useful for BR management authorities in the iterative process of evaluation and adaptive management.

Assessment of the environmental changes in arid and semiarid mining areas using long time-series Landsat images

Coal mining provides energy to support the development of society. However, mining affects the environment. Thus, substantial attention has been placed on environmental change in mining areas. Research assessing the environmental change in arid and semiarid mining areas has mainly been conducted by analyzing the variation in a single factor. However, the impact of coal mining on the environment is manifold, and there is no comprehensive assessment method. In this paper, we used a remote sensing index referred to as the comprehensive evaluation index (CEI) to comprehensively assess the environmental changes in arid and semiarid mining areas. The CEI in this study was calculated by integrating the changes in vegetation coverage (VC), land surface temperature (LST), and soil moisture (SM) using long time-series Landsat images. The Shendong mining area located in Northwest China was the study area. The results indicate that the environment of the Shendong mining area improved overall from 2000 to 2018. Environmentally degraded areas accounted for approximately 7.59% of the mining area and were concentrated in areas damaged by coal mining.

Free Fallin’? The decline in evidence-based decision-making by Canada’s protected areas managers

The conservation of biodiversity requires various forms of evidence to ensure effective outcomes. In this study, we provide an updated assessment of the state of evidence-based decision-making in Canada’s protected areas organizations by examining practitioner perceptions of: ( i) the value and use of various forms of evidence, ( ii) the availability of evidence to support decisions, and ( iii) the extent to which various institutional and behavioural barriers influence the use of evidence. Our results compare national surveys conducted in 2019 and 2013, revealing a significant and concerning decline in the use of all forms of evidence. We found significant declines in the use of peer-reviewed literature, local knowledge, and Indigenous knowledge. Our results correspondingly demonstrate a host of systemic barriers to the effective use of evidence, including a lack of trust, how to deal with uncertainty, and limited training. These challenges persist at a time when the quantity of information is greater than ever, and recognition of the value of Indigenous knowledge is relatively high (and increasing). Leadership is required to cultivate more relevant evidence, to embed scientists and Indigenous Knowledge-Holders in conservation organizations, to (re)establishing knowledge sharing forums, and to establish accountability and reporting measures to support efforts aimed at effectively achieving Canada’s biodiversity conservation goals.

Evaluation of the Management Effectiveness of German National Parks—Experiences, Results, Lessons Learned and Future Prospects

Since 2005 until today, experience has been gained in the preparation, implementation and impacts of the evaluation of the management effectiveness of German national parks. This process began with the development of a quality set containing fields of action, criteria, standards and a questionnaire to assess the state of national park management. This quality set was applied in the first voluntary full evaluation of German national parks, which took place from 2009 to 2012. An assessment of the full evaluation and the following interim evaluation (2015–2018) demonstrated the positive effects of the evaluation for the national parks, but also revealed some weaknesses of the quality set and the evaluation process. For this reason, work has been underway since 2019 to further improve the evaluation method; however, this has not yet been completed. The article provides an overview of the entire process. It concludes with considerations on the transferability of the evaluation method to other countries and gives some recommendations as to the most important aspects to be considered when evaluating the management effectiveness of national parks.

Top-down and bottom-up processes to implement biological monitoring in protected areas

Achieving biodiversity conservation objectives and targets requires environmental management agencies to undertake monitoring. Several management practices have been used but are they successful? Using SANParks as an example, we focus on three cases of national parks that use different techniques to implement environmental monitoring. We assessed a top-down, bottom-up and an integrated approach to setting targets using Table Mountain, Mokala and Bontebok National Parks respectively. Attainment of national and international objectives from State of Biodiversity assessment scores and achievement of objectives within individual Park Management Plans served as measurement criteria. We highlight several reasons for lack of achievement of objectives and targets for protected areas and illustrate that setting of objectives and monitoring must have complete full integration into park operations, both on staff and budget scales, rather than be seen as an external function from service divisions within parks or external service providers. An integrated approach to setting of objectives and targets of national and international importance is best achieved via the robust implementation of strategic adaptive management with interventions, measurement, reflection and adaptation integrated as learning by doing. Our results may not be unique but only a few organizations implement adaptive management in its full context.

Expanding Staff Voice in Protected Area Management Effectiveness Assessments within Kenya's Maasai Mara National Reserve

The concept of protected areas (PA) is one of the most widely used policy tools for biodiversity conservation including habitat and wildlife protection. Despite the importance and emphasis placed on protected area management, biodiversity is still in decline and more species are in danger of extinction. Some analyses have indicated that more than 40% of protected areas are poorly managed. To improve management effectiveness, the inclusion of diverse stakeholder information in articulating management strategies has been strongly encouraged; however, stakeholder involvement is often poorly integrated, and an opportunity granted only to select stakeholder groups, with PA staff, especially at lower organizational levels, minimally involved. Further, protected area management effectiveness (PAME) assessments are most frequently used as an aggregated outcome measure of effectiveness but these data should also inform management practice. Thus, for PA managers to obtain a greater understanding of issues impacting their effectiveness, they would benefit from including the voices of staff at all working levels. Therefore, the purpose of this investigation was to survey 135 wardens and rangers at all organizational levels from the two administrative sectors of Maasai Mara National Reserve to determine their perceptions of management effectiveness and to determine if significant differences existed across staff levels and administrative sectors. Significant differences were found to exist across staff levels and administrative authorities supporting the need for expanded staff voice in establishing effective PA management plans.

Evidence-based decision-making in Canada’s protected areas organizations: Implications for management effectiveness

Aichi Biodiversity Target 19 calls on Parties to the United Nations Convention on Biological Diversity (CBD) to improve, share, transfer, and apply knowledge. In this study, we provide an initial assessment of the state of evidence-based decision-making in Canada’s protected areas organizations by examining (1) the value and use of various forms of evidence by managers and (2) the extent to which institutional conditions enable or inhibit the use of evidence in decision-making. Results revealed that although managers value and use many forms of evidence in their decision-making, information produced by staff and their organizations are given priority. Other forms of evidence, such as Indigenous knowledge and peer-reviewed information, are valued and used less. The most significant barriers to evidence-based decision-making were limited financial resources, lack of staff, inadequate timeframes for decision-making, a lack of monitoring programs, and a disconnect between researchers and decision-makers. Overall, our results suggest that the potential benefits of evidence-based approaches are not being maximized in Canada’s protected areas organizations. We propose several recommendations to introduce or improve the use of diverse forms of evidence to enhance management effectiveness of Canada’s protected areas and by extension conservation outcomes.

Towards quantitative condition assessment of biodiversity outcomes: Insights from Australian marine protected areas

Protected area management effectiveness (PAME) evaluation is increasingly undertaken to evaluate governance, assess conservation outcomes and inform evidence-based management of protected areas (PAs). Within PAME, quantitative approaches to assess biodiversity outcomes are now emerging, where biological monitoring data are directly assessed against quantitative (numerically defined) condition categories (termed quantitative condition assessments). However, more commonly qualitative condition assessments are employed in PAME, which use descriptive condition categories and are evaluated largely with expert judgement that can be subject to a range of biases, such as linguistic uncertainty and overconfidence. Despite the benefits of increased transparency and repeatability of evaluations, quantitative condition assessments are rarely used in PAME. To understand why, we interviewed practitioners from all Australian marine protected area (MPA) networks, which have access to long-term biological monitoring data and are developing or conducting PAME evaluations. Our research revealed that there is a desire within management agencies to implement quantitative condition assessment of biodiversity outcomes in Australian MPAs. However, practitioners report many challenges in transitioning from undertaking qualitative to quantitative condition assessments of biodiversity outcomes, which are hampering progress. Challenges include a lack of agency capacity (staff numbers and money), knowledge gaps, and diminishing public and political support for PAs. We point to opportunities to target strategies that will assist agencies overcome these challenges, including new decision support tools, approaches to better finance conservation efforts, and to promote more management relevant science. While a single solution is unlikely to achieve full evidence-based conservation, we suggest ways for agencies to target strategies and advance PAME evaluations toward best practice.

Measuring impact of protected area management interventions: current and future use of the Global Database of Protected Area Management Effectiveness

Protected areas (PAs) are at the forefront of conservation efforts, and yet despite considerable progress towards the global target of having 17% of the world's land area within protected areas by 2020, biodiversity continues to decline. The discrepancy between increasing PA coverage and negative biodiversity trends has resulted in renewed efforts to enhance PA effectiveness. The global conservation community has conducted thousands of assessments of protected area management effectiveness (PAME), and interest in the use of these data to help measure the conservation impact of PA management interventions is high. Here, we summarize the status of PAME assessment, review the published evidence for a link between PAME assessment results and the conservation impacts of PAs, and discuss the limitations and future use of PAME data in measuring the impact of PA management interventions on conservation outcomes. We conclude that PAME data, while designed as a tool for local adaptive management, may also help to provide insights into the impact of PA management interventions from the local-to-global scale. However, the subjective and ordinal characteristics of the data present significant limitations for their application in rigorous scientific impact evaluations, a problem that should be recognized and mitigated where possible.

Do global indicators of protected area management effectiveness make sense? A case study from Siberia

Driven by the underperformance of many protected areas (PAs), protected area management effectiveness (PAME) evaluations are increasingly being conducted to assess PAs in meeting specified objectives. A number of PAME tools have been developed, many of which are based on the IUCN-WCPA framework constituting six evaluative elements (context, planning, input, process, output, and outcomes). In a quest for a more universal tool and using this framework, Leverington et al. (Environ Manag 46(5):685-698, 2010) developed a common scale and list of 33 headline indicators, purported to be representative across a wide range of management effectiveness evaluation tools. The usefulness of such composite tools and the relative weighting of indicators are still being debated. Here, we utilize these headline indicators as a benchmark to assess PAME in 37 PAs of four types in Krasnoyarsk Kray, Russia, and compare these with global results. Moreover, we review the usefulness of these indicators in the Krasnoyarsk context based on the opinions of local PA management teams. Overall, uncorrected management scores for studied PAs were slightly better (mean = 5.66 ± 0.875) than the global average, with output and outcome elements being strongest, and planning and process scores lower. Score variability is influenced by PA size, location, and type. When scores were corrected based on indicator importance, the mean score significantly increased to 5.75 ± 0.858. We emphasize idiosyncrasies of Russian PA management, including the relative absence of formal management plans and limited efforts toward local community beneficiation, and how such contextual differences may confound PAME scores when indicator weights are treated equal.

Are we missing the boat? Current uses of long-term biological monitoring data in the evaluation and management of marine protected areas

Protected area management agencies are increasingly using management effectiveness evaluation (MEE) to better understand, learn from and improve conservation efforts around the globe. Outcome assessment is the final stage of MEE, where conservation outcomes are measured to determine whether management objectives are being achieved. When quantitative monitoring data are available, best-practice examples of outcome assessments demonstrate that data should be assessed against quantitative condition categories. Such assessments enable more transparent and repeatable integration of monitoring data into MEE, which can promote evidence-based management and improve public accountability and reporting. We interviewed key informants from marine protected area (MPA) management agencies to investigate how scientific data sources, especially long-term biological monitoring data, are currently informing conservation management. Our study revealed that even when long-term monitoring results are available, management agencies are not using them for quantitative condition assessment in MEE. Instead, many agencies conduct qualitative condition assessments, where monitoring results are interpreted using expert judgment only. Whilst we found substantial evidence for the use of long-term monitoring data in the evidence-based management of MPAs, MEE is rarely the sole mechanism that facilitates the knowledge transfer of scientific evidence to management action. This suggests that the first goal of MEE (to enable environmental accountability and reporting) is being achieved, but the second and arguably more important goal of facilitating evidence-based management is not. Given that many MEE approaches are in their infancy, recommendations are made to assist management agencies realize the full potential of long-term quantitative monitoring data for protected area evaluation and evidence-based management.

Measuring the accuracy of management effectiveness evaluations of protected areas

Evaluating the effectiveness of protected area management to help improve management outcomes is becoming an increasingly common practice. The evaluation tools developed and implemented in over 100 countries generally rely on the expert judgements of protected area managers. Despite the growing collection and use of management effectiveness evaluation data, there have been no previous attempts to measure the accuracy of these data. We measured the accuracy of managers' judgements about the conditions in their reserves by collecting independent field data. We also assessed how accurately the evaluation tool reflected managers' views by conducting semi-structured interviews with 23 protected area managers from New South Wales, Australia. We found that managers made highly accurate judgements of the extent of a common weed species, Rubus fruticosus (blackberry), but often misinterpreted the scope, scale and timeframe of the evaluation. These framing effects can lead to error being introduced into the evaluation dataset, affecting the precision of evaluations such that they cannot be reliably compared among reserves. We suggest that the wording of evaluation questions needs to be explicit about the assessment frame to minimize the influence of framing effects on management effectiveness evaluations.

Evaluating the management effectiveness of five protected areas in Taiwan using WWF's RAPPAM

This study adopts and modifies the WWF Rapid assessment and prioritization of protected areas management methodology (RAPPAM) to evaluate the management effectiveness of five protected areas in Taiwan. The results indicate that, unlike the situation in most developing countries, the threats and pressures faced by protected areas in Taiwan come mainly from the outside-with pollution as the most common pressure and threat, and difficult for their management authorities to deal with effectively. The categories and extent of these pressures and threats are related to remoteness and geographic location of the protected areas. All five cases under study reveal a similar management approach, clear management goals, adequate basic infrastructure and clear management decision-making; on the down side, however, all of them suffer from the lack of an effective comprehensive management plan, inadequate or poor quality human resources and insufficient funding. The present study suggests that first priority should be given to strengthening management planning in order to improve management effectiveness of protected areas in Taiwan. The adjustment made to RAPPAM in this study was to amend and take the format of the management plan as the basis for evaluation material preparations and open the discussion to encourage stakeholders' participation to open the dialogue among them. The results indicate that, although the system evaluation design still has some constraints, the quality of information collected is improved and can respond more directly to the specific demands of the respective areas.