Reinforcement learning for decision-making under deep uncertainty

Planning under complex uncertainty often asks for plans that can adapt to changing future conditions. To inform plan development during this process, exploration methods have been used to explore the performance of candidate policies given uncertainties. Nevertheless, these methods hardly enable adaptation by themselves, so extra efforts are required to develop the final adaptive plans, hence compromising the overall decision-making efficiency. This paper introduces Reinforcement Learning (RL) that employs closed-loop control as a new exploration method that enables automated adaptive policy-making for planning under uncertainty. To investigate its performance, we compare RL with a widely-used exploration method, Multi-Objective Evolutionary Algorithm (MOEA), in two hypothetical problems via computational experiments. Our results indicate the complementarity of the two methods. RL makes better use of its exploration history, hence always providing higher efficiency and providing better policy robustness in the presence of parameter uncertainty. MOEA quantifies objective uncertainty in a more intuitive way, hence providing better robustness to objective uncertainty. These findings will help researchers choose appropriate methods in different applications.

Expediting the Implementation of Closed-Loop Supply Chain Management: a Facilitated Case Study on Re-using Timber in Construction Projects

An increasing number of firms are aiming to implement closed-loop supply chain (CLSC) management to contribute to a more circular economy. However, for many of these firms, it is difficult to translate this strategic aim into fruitful operational decisions. They need to address many deep uncertainties and dynamic complexities in their supply chain system, which make their transition towards CLSC management challenging. This article aims to develop a better understanding of how supply chain actors taking steps towards CLSC management could be supported to reach higher levels of maturity in dealing with deep uncertainty and dynamic complexity. This is investigated in a single, facilitated, embedded case study: a future-oriented decision-making process regarding the use of timber with four real-world actors in the construction industry. The process is structured and supported with analyses, following a methodology based on the capability maturity approach. In this empirical context, the selected approach is shown to have positive effects on clarifying the potential impact of transitions to CLSC management. Furthermore, it stimulates important learning processes during the transition, and as such supports actors to achieve higher levels of maturity and to take further steps towards CLSC management. In this context, a conceptual distinction is made between 'situational maturity' and 'mental maturity', which enriches double-loop learning theory in the context of transitions.

Decision-making under the deep uncertainty of climate change: The psychological and political agency of narratives

Fossil fuel-based development has resulted in climate change and biodiversity loss, threatening the ability of the biosphere to sustain civilization. However, despite the transformative change needed to address climate change, the complexity inherent in dynamic, coupled social-ecological systems can create challenges that stifle mitigation and adaptation efforts. For example, increasing urbanization can mask information about the local and distal ecological impacts of unsustainable consumption patterns. Diverse actors, powerful vested interests in the status quo, and differential impacts of climate change create inevitable tradeoffs and conflicts among stakeholders. The multitude of plausible future scenarios and their dependence on actions taken today create challenges for planning, governance, and collective action. While there is a long history in psychology and economics of studying decision-making under uncertainty, we argue that the deep uncertainty inherent in climate change cannot be easily understood using these same paradigms. In this context, narratives-stories about how the world works, what the future will look like, and our own role in this process-can extend cognition, creating shared knowledge across space and time, and shape our beliefs, values and actions in the face of tremendous uncertainty. Narratives thus have political and psychological agency and can reinforce or challenge existing power relations and trajectories. Here, we review some of this literature in the context of climate change.

A perspective on the role of uncertainty in sustainability science and engineering

The Trans-Atlantic Research and Development Interchange on Sustainability Workshop (TARDIS) is a meeting on scientific topics related to sustainability. The 2019 workshop theme was "On the Role of Uncertainty in Managing the Earth for Global Sustainability." This paper presents the perspectives on this topic derived from talks and discussions at the 2019 TARDIS workshop. There are four kinds of uncertainties encountered in sustainability ranging from clear enough futures to true surprises. The current state-of-the-art in assessing and mitigating these uncertainties is discussed.

Deep Uncertainties in Sea-Level Rise and Storm Surge Projections: Implications for Coastal Flood Risk Management

Sea levels are rising in many areas around the world, posing risks to coastal communities and infrastructures. Strategies for managing these flood risks present decision challenges that require a combination of geophysical, economic, and infrastructure models. Previous studies have broken important new ground on the considerable tensions between the costs of upgrading infrastructure and the damages that could result from extreme flood events. However, many risk-based adaptation strategies remain silent on certain potentially important uncertainties, as well as the tradeoffs between competing objectives. Here, we implement and improve on a classic decision-analytical model (Van Dantzig 1956) to: (i) capture tradeoffs across conflicting stakeholder objectives, (ii) demonstrate the consequences of structural uncertainties in the sea-level rise and storm surge models, and (iii) identify the parametric uncertainties that most strongly influence each objective using global sensitivity analysis. We find that the flood adaptation model produces potentially myopic solutions when formulated using traditional mean-centric decision theory. Moving from a single-objective problem formulation to one with multiobjective tradeoffs dramatically expands the decision space, and highlights the need for compromise solutions to address stakeholder preferences. We find deep structural uncertainties that have large effects on the model outcome, with the storm surge parameters accounting for the greatest impacts. Global sensitivity analysis effectively identifies important parameter interactions that local methods overlook, and that could have critical implications for flood adaptation strategies.

Optimal adaptation pathway for sustainable low impact development planning under deep uncertainty of climate change: A greedy strategy

Robustness and cost effectiveness are major concerns for sustainable stormwater management under deep uncertainty of climate change. Given that many traditional static planning strategies are not working with unpredictable future conditions, the possibility of system failure, and the lock-in effects, the Adaptation Pathway (AP) approach was adopted for dynamically robust and cost-effective planning in this paper. In order to increase optimization accuracy of multi-staged planning, a continuous definition of the AP optimization problem was raised by improving the simplified versions in existing studies. A case study in Suzhou, a provincial pilot Sponge City in China undergoing increasing annual rainfall and severe water environment deterioration, was included by integrating Long-Term Hydrologic Impact Assessment-Low Impact Development model with optimization methods, aiming to persistently control the non-point source total phosphorus loading below an acceptable amount in the following unforeseen 20 years via multi-staged low-impact development (LID) construction. A novel optimization method developed by the authors in a companion paper, namely marginal-cost-based greedy strategy (MCGS), was successfully applied to efficiently solve the continuous version of the AP optimization problem. The popular genetic algorithm (GA) was used as a contrast. A weather generator was elaborated based on four Representative Concentration Pathway scenarios and 17 spatial downscaled general circulation models to simulate the unforeseen future annual rainfalls that helped with evaluating cost effectiveness of each prospective LID plan. Results showed that the adaptation pathways optimized by MCGS could save the whole life net present cost of an LID plan by 1%-60% compared with those optimized by GA, and the computational efficiency of MCGS was over 13 times faster than GA.