INTRODUCTION

National Wildlife Refuges (NWRs) play an important role in helping social-ecological systems adapt to global-change processes such as climate change, sea-level rise, and changing land use. NWRs constitute a national network of lands and waters owned or leased by the federal government of the United States whose principal mission is conservation of fish, wildlife, plant resources, and their habitats, as well as the provision of recreational opportunities. The challenges imposed by global change are daunting for any conservation organization, and available personnel and fiscal resources for individual refuges are rarely adequate to meet expectations. Thus, it is imperative that scarce conservation resources be used as efficiently as possible as refuge managers and personnel struggle with a mission that has gradually become more difficult. Conservation triage is a term referring to the allocation of scare resources to maximize the effectiveness of conservation (Bottrill et al. 2008). The term is derived from medical crisis management, where limited resources must be allocated to do the most good. We note that conservation triage has sometimes been seen as reflective of a defeatist attitude (Pimm 2000). We argue, however, that conservation triage is a constructive framework with which to make decisions about how to allocate limited resources over spatial, temporal, and organizational scales to best meet conservation objectives. This framework is the basis of traditional ecological-economic models that recognize and incorporate limited budgets when considering actions intended to maximize the net benefit to society.

In 2013 we initiated collaborations among managers and researchers with the intent of helping NWRs make more effective planning and adaptation decisions in response to climate change and other global-change processes. In a complex and uncertain world, good decisions do not guarantee good outcomes, but a systematic decision process, in which decision makers, i.e., managers, and scientists are both engaged in the development of potential solutions to complex problems, should enhance the likelihood of favorable outcomes (Keeney 2004). In pursuing this management-research collaboration, we worked with NWR staff along the Atlantic Coast to understand the problems they face and how decision science might help address them. Our initial focus was on (1) exploring how the managers’ ability to meet their objectives is influenced by the larger social-ecological system in which the NWRs are embedded; (2) understanding how to account for stakeholders’ values, perceptions of trade-offs, and risk tolerance (we define a stakeholder as anyone who has an interest in the outcomes of refuge decisions); and (3) developing decision-analytic tools appropriate for problems with “deep uncertainty,” conflicting values, and multiple decision makers.

Decision analysis has been widely used in business and government decision making, but its application to problems in natural resource conservation has become increasingly common in the last two decades (McFadden et al. 2011). Traditional approaches to decision making, which tend to focus mostly on alternatives and predicted outcomes, can be distinguished from modern methods that emphasize multiple values and the trade-offs inherent in natural resource management (Gregory et al. 2012). The emphasis on values rather than outcomes helps decision makers and stakeholders understand whether the source of disagreement over policy is related to predicting outcomes or how those outcomes are valued, i.e., subjective preferences. It also helps promote a role for analysts and scientists in conservation decision making as “honest brokers” (Pielke 2007) rather than as advocates for a particular course of action. Multicriteria decision analysis that accounts for outcomes and values is now widely used in conservation and is seen as contributing to better decisions through a formal structuring of decision problems that acknowledges and accommodates conflicts in values among stakeholders (Huang et al. 2011).

We note that an important aspect of the trend toward formal decision analysis in natural resource management has been the increasing application of methods to analyze dynamic decisions, often within an adaptive-management framework (Williams 1989, Possingham 1997). The dynamic management problem involves a temporal sequence of decisions, with the optimal action at each decision point depending on time and/or system state. The goal is to provide managers with a decision rule or strategy that prescribes management actions for each time and system state that are optimal with respect to the decision maker’s objective(s). A key consideration in dynamic decisions for natural resource problems is the uncertainty attendant to decision outcomes, which adds to the demographic and environmental variation of stochastic resource changes (Williams and Johnson 2013). Multiple hypotheses and associated models are used to characterize this uncertainty, and comparisons of model predictions with observations from a monitoring program are used to derive model-specific probabilities that indicate model credibility (Walters 1986). Important advances have followed from the recognition that these probabilities are not static, but evolve over time as new observations of system behaviors are accumulated from the management process, thus constituting a formalization of learning. Indeed, the defining characteristic of adaptive management is the attempt to account for the temporal dynamics of uncertainty in making management decisions (Johnson and Williams 2015).

Despite its conceptual appeal, however, the application of decision science to complex, real-world problems can be quite challenging. In formulating, evaluating, and modifying environmental policies, difficult questions arise about how to design processes that nurture and sustain the engagement of stakeholders, scientists, and decision makers, and that enable governing bodies and institutions to promote discourse, transparency, accountability, learning, and a shared stewardship of the environment. Indeed, much of the recent literature in natural resource management focuses on the need for so-called “double-loop” and “triple-loop” learning (Pahl-Wostl 2009), in which extant problem formulations, laws and regulations, institutional norms, and power relationships are called into question and revisited. Part of the difficulty is that many conservation problems involve multiple decision makers, often acting more or less independently in pursuit of their own goals. Add to this complexity the presence of various sources and degrees of uncertainty in outcomes, and it is perhaps easy to understand why some have questioned whether decision science can be successfully applied to such wicked problems (Ludwig 2001). The challenge for our project was to frame coastal resource conservation in a way that promotes a shared understanding of the problem among diverse decision makers; to understand the values and objectives of key stakeholders and to integrate these into the decision-making framework; to understand how key socioeconomic and environmental drivers and outcomes are linked across a range of spatial and temporal scales; and to develop approaches appropriate for coping with deep uncertainty and our collective ability to influence the future.

METHODS

In an effort to help bridge the research-implementation gap in conservation (Knight et al. 2006, 2008), our team conducted a number of face-to-face meetings with coastal NWRs on the Atlantic and Gulf coasts, as well as two workshops involving managers, scientists, and decision analysts. Several of us first visited Alligator River (North Carolina), Cape Romain (South Carolina), Savannah River (South Carolina/Georgia), Lower Suwannee (Florida), and St. Marks (Florida) NWRs during the autumn of 2013. Following these “listening tours,” our team conducted a two-day workshop involving staff from four coastal refuges or refuge complexes: Eastern North Carolina, Cape Romain, Chesapeake Marshlands (Maryland), and Parker River (Massachusetts) NWRs (Fig. 1). The objectives of the two-day workshop were (1) to develop a shared understanding of the adaptation problems faced by coastal refuges and (2) explore the utility of structured decision making (SDM), a.k.a., decision analysis or decision science, as a methodological approach for addressing these problems.

Initially, our focus was on gaining a better understanding of the perspectives of refuge staff and what they see as desired futures and key challenges in dealing with global change processes. In particular, we believed that the issues of identifying conservation objectives and the scales at which they are pursued are central to any systematic analysis of decision making. Using the decision context of a NWR to address these issues proved to be highly relevant and informative given the refuge system’s organizational structure and broad spatial scale, but these matters are germane to many conservation problems. Examples of the scale-related questions that arose during early discussions and the 2-day workshop are provided in Table 1.

Following the two-day workshop, we conducted a week-long Structured Decision Making Workshop at the National Conservation Training Center (NCTC; Shepherdstown, West Virgina, USA). NCTC has pioneered development of a curriculum to help build capacity in applied decision science within the U.S. conservation community (Johnson et al. 2015; http://nctc.fws.gov/courses/programs/decision-analysis/). This development includes both instructor-led courses and hands-on experiential workshops that address real conservation problems. NCTC also has a mentoring program designed to build core competencies in the delivery of decision science and to provide opportunities for in-depth training in applying decision-analytic methods.

SDM (Gregory et al. 2012) is a method for analyzing decisions by breaking them into their essential components and then reassembling them to identify a preferred course of action. The SDM process typically includes eliciting stakeholder values and formulating management objectives, identifying potential management actions, i.e., alternatives, developing relevant system models that predict the outcomes of those actions, selecting an optimization method to identify optimal choices with respect to the objectives, and designing a monitoring program to keep track of the state of the system and evaluate progress (Williams et al. 2002, Skinner 2009, Conroy and Peterson 2013). SDM and adaptive management are grounded in modern decision theory, which provides a rigorous framework for making decisions about the management of complex systems under uncertainty (Williams et al. 2002, Burgman 2005).

We followed general guidelines for the framing of decision problems provided by Keeney (1992), meaning that decision problems are explicitly structured in terms of alternatives, outcomes, and values to identify the management action that is most likely to meet the stated objectives. Decisions involve both predicting outcomes, generally with mathematical models, from alternative actions and valuing those outcomes. Predicting the outcomes of management actions falls under the purview of environmental science, whereas assigning value to those outcomes is the role of the decision maker and, ultimately, of society. This distinction is useful for, among other things, providing guidance on who should be involved with each stage of the decision framing and analysis. Discussions were thus values focused (Keeney 1992) in the sense that the ecological, social, and economic values the refuge supports were recognized as the key to developing and evaluating adaptation choices. Values and objectives were discussed first and drove the rest of the decision analysis. Emphasis was placed on the recognition that “the decision context and the fundamental objectives that frame a decision situation must be compatible” (Keeney 1992:35). In other words, objectives should be sufficient to fully evaluate all the alternatives and alternatives should be sufficient to describe all the various ways in which the objectives could be achieved. This type of approach to decision making focuses on explicit identification of objectives and then development of creative actions to achieve these objectives. This approach differs from the common strategy of starting with a set of obvious alternatives and then evaluating them, often against unspecified or assumed objectives (Keeney 1992).

The NCTC workshop focused on the Cape Romain NWR as a case study. This NWR has been highlighted by the Secretary of the U.S. Department of the Interior as an alarming example of the impacts of sea-level rise on the country’s public lands. As expected, two topics dominated workshop discussions: (1) values and objectives and (2) the issue of scale. Questions concerning values and objectives serve as a basis for exploring decision opportunities in addressing the impacts of global change on NWRs. Sometimes identifying the decision problem is obvious. For example, do we continue to manage this freshwater impoundment with increasing saltwater intrusion? However, in many contexts decision opportunities must be created from identification of one or more concerns, e.g., erosion of barrier islands, and an examination of the values or objectives in play, e.g., the aesthetic or existence value of sea turtles. Because of the many and diverse objectives embraced by coastal NWRs, a large portion of the workshop was devoted to identifying and structuring objectives, with attendant consideration of the issue of scale mismatch between management objectives and the decision context. In particular, there was a focus on both strategic and fundamental objectives and how they shape particular decision contexts (Keeney 1992). By decision context we simply mean the specific nature of the decision, and the set of suitable alternatives to consider, for addressing particular management concerns, e.g., what lands should be prioritized for conservation.

RESULTS AND DISCUSSION

The workshops involving scientists, NWR managers, and decision analysts demonstrated that refuge staff and scientists can work together successfully to identify objectives, develop viable and creative management alternatives, predict at least qualitatively the consequences of those alternatives, and identify preferred strategies for the set of traditional objectives being considered by the refuge. A key insight from the NCTC workshop was the realization that an individual refuge’s efforts to meet its conservation mission can be viewed as an opportunity to scale up resource objectives to complement the values of a broader collection of constituents to promote the value of the refuge to society at large. This insight led to an increased understanding of scale issues, the importance of how objectives are specified, and the added complexity of problems with multiple decision makers. The principal challenge is to broaden management objectives to include other ecosystem goods and services that traditionally have not been explicitly considered by refuges in their decisions, e.g., protection of coastlines from storm surge, carbon sequestration, and economic contribution of fin and shellfish nurseries. This broadening of objectives complicates the decision-making process by necessitating the search for additional alternative actions, but it also provides opportunities for collaboration, as well as identifying conflicts, with stakeholders who may have agendas different from those of the refuge.

Effective engagement with stakeholders will be critical to this broader refuge-society collaboration. However, important considerations remain, including (1) identifying the stakeholders with an interest in the outcomes of this project; (2) assessing their current levels of knowledge, interest, and engagement with respect to sea-level rise on coastal refuges; (3) specifying whose objectives should be considered; and (4) predicting the associated effects on management decisions. Resolving these issues could be achieved through a combination of techniques including face-to-face interviews, small group meetings, focus groups, surveys, literature review, media (e.g, print, broadcast, social) analysis, and expert opinion. The assessment would provide the foundation on which to build a stakeholder engagement strategy (e.g., Meadow et al. 2015), as well as baseline information against which to measure progress.

Physical scientists will also need to work closely with social scientists, in particular environmental economists, to help value the provision of ecosystem goods and services under current and future refuge designs. Many management actions that impact the quantity and quality of ecosystem services provided by refuges have benefits and costs whose values accrue outside traditional markets, such as increased flood protection, decreased nutrient cycling, and increased recreational quality for visitors. Nonetheless, these values are critical to understanding how scarce management dollars should be allocated among competing demands. A host of economic valuation tools exists to quantify these nonmarket benefits and costs, such as choice experiments, contingent valuation, hedonic property value models, and recreational demand analysis (Champ et al. 2003, Freeman et al., 2014). Benefits-transfer techniques provide a reasonable and efficient approach to quantify the value of a great many different ecological services (Bergstrom and Taylor 2006).

However stakeholder values are ultimately characterized, refuge staff thus far have articulated a different problem frame and associated scale considerations depending on whether the focus is on the short or long term. The short-term problem focuses on efficient allocation of limited staff time and budgets for management of existing programs and resources under the current refuge design. Acknowledged as a near-term solution, refuge managers must make informed resource-allocation decisions to minimize loss in the capacity to meet the refuge mission within the current refuge footprint. Impacts of global change processes like sea-level rise are contributing to this decline. Assessment of the impacts of alternative management actions on achieving these goals are implemented largely at the refuge level, constrained by an annual operating budget; i.e., objectives are primarily determined by local refuge leadership, although possibly informed by stakeholder input.

Over the long term, refuge managers recognize that impacts from global-change processes will continue to erode their ability to achieve social and ecological values derived from the existing refuge configuration. Ultimately, refuge managers must decide when and where to acquire or protect new habitat to supplement or replace the existing refuge footprint and sustain refuge values as the system evolves over time. Creating and implementing a strategy for expanding refuge capacity to sustain the refuge mission are likely to involve capital-allocation decisions out of the direct control of the staff of an individual refuge. Because of the disparity between federal funding for land acquisition and the high market prices for coastal properties, it is unrealistic to expect that a major refuge expansion could be accomplished solely through fee-simple land purchase using federal funds. Therefore, efforts to expand refuges to meet these changing needs will require creative problem solving, including the creation of partnerships and the identification of a set of common objectives and funding sources that the partners, including the refuge, are willing to bring to bear in a collaborative manner.

The theme common to both the short- and long-term problems is the desire to minimize the cumulative loss of value over time as conditions change in the refuge and surrounding landscape. However, each time frame suggests a unique set of alternatives to represent differences in decision context, the identity of the decision maker(s), and spatial, temporal, and governance scales of the decision problem. For example, the decision context of the short-term resource allocation problem is limited to the set of management activities that can be implemented on existing refuge lands, plus partnership building, with decision-making authority principally residing at the individual refuge level. This context helps to define the spatial and governance scales, while the temporal scale may reflect short-term planning (e.g., 10- to 15-year Comprehensive Conservation Plans; National Wildlife Refuge System Improvement Act 1997) or budgetary cycles. The long-term expansion plan or reserve-design problem will need to include broader scales. Needed is an expanded but explicit spatial scale (i.e., beyond the current refuge boundary but constrained to evaluate a reasonable set of alternative refuge designs) that accounts for more complex governance structures (e.g., multiple decision makers and their values, laws, and institutional policies) and takes into consideration a longer planning horizon (e.g., multidecadal).

There is interest in combining both short-term and long-term decision-making problems into a common analytical framework. For example, a decision maker might be faced with the problem of allocating varying resources to adaptive (e.g., relocation) or defensive (e.g., protection of an existing investment) actions via a management portfolio. Both actions incur an immediate cost, but defensive actions have an immediate payoff and may depreciate at an unknown rate, whereas the payoff of adaptation actions occurs in the future and, therefore, may be subject to temporal discounting. As global change proceeds, how should the investment mix of the portfolio change over time to maximize the temporal sum of returns? What if the decision maker is highly uncertain about how fast and how much the environment will change? Modern portfolio theory, which is based on the idea that both the expected return and the probability of deviation from the expectation (i.e., risk) are important (Ando and Mallory 2012, Hoekstra 2012), might provide a useful analytical framework. A portfolio diversifies risk onto a variety of assets, making it less sensitive to deviations in expectations. Key elements are the expected return of individual assets, their deviations (uncertainty), and the correlation in expected returns among assets. Low or even negative correlation among assets reduces risk. We suggest a useful analytical framework is that provided by Marinoni et al. (2011). What is missing from their approach is the fact that asset allocation can change over time; i.e., asset allocation can be dynamic. In particular, returns will depend on both current and future conditions, and although current conditions are known, future conditions are uncertain. In a dynamic setting, risk decreases as the future is realized, i.e., becomes more certain.

A productive line of inquiry would be to determine how to optimize a state/time-specific portfolio in response to changing system state and to our understanding of how that state may change in the future, i.e., adaptive portfolio optimization. For example, this approach might be demonstrated using investment decisions in tidal wetlands. Tidal wetlands and associated submerged aquatic plant beds are important spawning nurseries and shelter areas for fish and shellfish, including commercially important species like the blue crab (Callinectes sapidus). Investment decisions over time to maintain tidal wetlands and their associated benefits might include barrier-island renourishment, repairing impoundment levees, deepening freshwater wells, plugging ditches, installing or repairing water control structures, securing new property upslope, building new impoundments, raising wetlands (dredge and fill), or purchasing rolling easements. We would expect the portfolio of assets, i.e., management actions, to change over time as both the future and our understanding of the consequences of actions became more certain. This is the essence of adaptive management (Johnson and Williams 2015).

Another avenue for integrating short-term resource-allocation decisions with long-term reserve-design planning may be to include a full complement of possible alternatives in the time-space sequence of policy selection. Just as we view currently unprotected parcels as potentially contributing value to any of several objectives, we might also consider that the existing NWR comprises similarly defined spatial units, i.e., distinguished by resource type such as wetland, impoundment, or forest. Building on the concepts outlined above, the time-specific treatment of a dynamic reserve-design problem could include (1) alternative mechanisms used to conserve currently unprotected parcels (e.g., purchase, easement, none); (2) parcel-specific management options for land units within the current refuge footprint that account for predicted changes and benefits over time; and (3) decisions to divest from currently owned or managed refuge parcels and redirect the resources (time, money) gained to those actions inside or outside the refuge evaluated as having a greater cumulative benefit over the planning horizon. An approximate analogy exists for the static reserve-design problem (e.g., Watts et al. 2009), but we are unaware of any attempt to develop a dynamic solution that includes multiple classes of alternatives, as well as the ability to consider both investment and divestment decisions. Such an algorithm would constitute a significant advance in decision tools for conservation application by representing more realistically the complete set of management alternatives commonly available to decision makers, as opposed to treating decision problems separately and then integrating them as a subsequent step.

CONCLUSIONS

Refuge resource-allocation decisions are often complicated by a mismatch between the scale of environmental variation (processes) and the spatial and temporal scale at which management actions can influence the attainment of refuge objectives (Iguchi 2011). A significant number of the challenges and, presumably, failures in natural resource management are attributable to poor understanding of and response to the interaction of social-ecological processes across scales and levels (Cumming et al. 2006, Cumming and Norberg 2008, Guerrero et al. 2013). Throughout our workshops, refuge managers emphasized to us that they have limited control, authority, and resources to fully realize conservation objectives for wide-ranging migratory species that may only spend a portion of their lifecycles in refuge habitats and whose migratory paths may eventually be altered by changing climate and habitats. Although the governance structure of the regional NWRs may result in an appropriate matching of scales for some decisions (e.g., resource allocation at the regional or flyway scale), many decisions are made at the refuge level for which the likelihood of scale mismatch is increased (e.g., desiring to ensure adequate breeding habitat for sea turtles but, because of beach and barrier island erosion, lacking the operational capacity to maintain sufficient habitat). NWRs face a considerable challenge in defining achievable yet meaningful objectives while at the same time expanding the scale of the decision context by considering a broader set of stakeholders and expressing the significance of the refuge in terms that engender wide societal support.

Although partnerships are an important component of current refuge operation and sustainability, partnerships with, and public support from, other decision makers and stakeholders are critical for any expansion of a refuge. Examples of the kinds of actors whose interests and decisions may directly or indirectly influence the outcomes for fundamental resource objectives include commercial fishers, large private or public landowners, nongovernmental organizations, public agencies that enforce state or federal laws/regulations that affect habitat, Friends of the Refuge groups, volunteers, state/federal departments of transportation, local and county governments, and small businesses such as lodging and restaurant owners who benefit from ecotourism. Decisions and actions by these actors may influence the availability of habitat in a positive or negative way. Thus, broad-based political support is a crucial component for implementation of any expansion plan, regardless of the availability of land and willingness of potential sellers. Engendering support will likely be most effective if the objectives of these decision makers are also considered when quantifying the value of the refuge or refuge system. This understanding presents an opportunity to reframe the metrics used for appraising the value of the refuge and incorporating these expanded metrics and values in the decision framework; i.e., scaling the evaluation of benefits to match the decision context of a broader set of stakeholders whose interests are focused on those goods and services that meet greater public benefit.

Griffith et al. (2009) pointed out the overwhelming challenges that climate and other global-change processes present to NWRs and highlighted the need for better communication and collaboration among managers, scientists, and the broader public that is interested in the societal values provided by refuges. More pointedly, Iguchi (2011) described how climate change will render some refuge purposes obsolete and how conflicts among the purpose of an individual refuge, the various purposes of other refuges in the system, and the broad statutory mission of the NWR system are likely to increase. It now seems likely that refuge governance and institutional arrangements will have to adapt in significant ways if the ecosystem goods and services offered by refuges are to persist in the face of global change. Although we believe that application of decision science can certainly help refuges make better adaptation decisions (Table 2), perhaps its most enduring contribution will be in its capacity to motivate managers to periodically re-examine their purposes and rules of operation. This critical self-examination is usually precipitated by the recognition that current operating premises and protocols are inadequate to address emerging ecological or sociological issues (Pahl-Wostl 2009). This type of social learning is difficult because institutions have to acknowledge deficiencies in their processes and policies, and because the search for solutions usually involves conflict. However, we believe that working closely with individual refuges to structure their decision problems in terms of choices, outcomes, and values can best help motivate and organize this larger discussion.

LITERATURE CITED

Ando, A. W., and M. L. Mallory. 2012. Optimal portfolio design to reduce climate-related conservation uncertainty in the Prairie Pothole Region. Proceedings of the National Academy of Sciences of the United States of America 109:6484-6489. http://dx.doi.org/10.1073/pnas.1114653109

Bergstrom, J. C., and L. O. Taylor. 2006. Using meta-analysis for benefits transfer: strengths and weaknesses. Ecological Economics 60:351-360. http://dx.doi.org/10.1016/j.ecolecon.2006.06.015

Bottrill, M. C., L. N. Joseph, J. Carwardine, M. Bode, C. Cook, E. T. Game, H. Grantham, S. Kark, S. Linke, E. McDonald-Madden, R. L. Pressey, S. Walker, K. A. Wilson, and H. P. Possingham. 2008. Is conservation triage just smart decision making? Trends in Ecology & Evolution 23:649-654. http://dx.doi.org/10.1016/j.tree.2008.07.007

Burgman, M. 2005. Risks and decisions for conservation and environmental management. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/cbo9780511614279

Champ, P., T. Brown, and K. Boyle. 2003. A primer on nonmarket valuation.. Springer Science+Business Media, New York, New York, USA.

Conroy, M. J., and J. T. Peterson. 2013. Decision making in natural resource management: a structured, adaptive approach. John Wiley & Sons, West Sussex, UK. http://dx.doi.org/10.1002/9781118506196

Cumming, G. S., D. H. M. Cumming, and C. L. Redman. 2006. Scale mismatches in social-ecological systems: causes, consequences, and solutions. Ecology and Society 11(1):14. [online] URL: http://www.ecologyandsociety.org/vol11/iss1/art14/

Cumming, G. S., and J. Norberg. 2008. Scale and complex systems. Pages 246-276 in J. Norberg, and G. S. Cumming, editors. Complexity theory for a sustainable future. Columbia University Press, New York, New York, USA.

Freeman, A. M., III, J. A. Herriges, and C. L. Kling. 2014. The measurement of environmental and resource values: theory and methods. Third edition. RFF Press, Abingdon, UK.

Gregory, R., L. Failing, M. Harstone, G. Long, T. McDaniels, and D. Ohlson. 2012. Structured decision making: a practical guide to environmental management choices. John Wiley & Sons, West Sussex, UK. http://dx.doi.org/10.1002/9781444398557

Griffith, B., J. M. Scott, R. Adamcik, D. Ashe, B. Czech, R. Fischman, P. Gonzalez, J. Lawler, A. D. McGuire, and A. Pidgorna. 2009. Climate change adaptation for the US National Wildlife Refuge System. Environmental Management 44:1043-1052. http://dx.doi.org/10.1007/s00267-009-9323-7

Guerrero, A. M., R. R. J. McAllister, J. Corcoran, and K. A. Wilson. 2013. Scale mismatches, conservation planning, and the value of social-network analyses. Conservation Biology 27:35-44. http://dx.doi.org/10.1111/j.1523-1739.2012.01964.x

Huang, I. B., J. Keisler, and I. Linkov. 2011. Multi-criteria decision analysis in environmental sciences: ten years of applications and trends. Science of the Total Environment 409:3578-3594. http://dx.doi.org/10.1016/j.scitotenv.2011.06.022

Hoekstra, J. 2012. Improving biodiversity conservation through modern portfolio theory. Proceedings of the National Academy of Sciences of the United States of America 109:6360-6361. http://dx.doi.org/10.1073/pnas.1205114109

Iguchi, J. 2011. Improving the Improvement Act: climate change management in the National Wildlife Refuge System. Environs 34:247-269.

Johnson, F. A., M. J. Eaton, J. Williams, G. H. Jensen, and J. Madsen. 2015. Training conservation practitioners to be better decision makers. Sustainability 7:8354-8373. http://dx.doi.org/10.3390/su7078354

Johnson, F. A., and B. K. Williams. 2015. A decision-analytic approach to adaptive resource management. Pages 61-84 in C. R. Allen and A. S. Garmestani, editors. Adaptive management of social-ecological systems. Springer, Dordrecht, Netherlands. http://dx.doi.org/10.1007/978-94-017-9682-8_5

Keeney, R. L. 1992. Value-focused thinking: a path to creative decisionmaking. Harvard University Press, Cambridge, Massachusetts, USA.

Keeney, R. L. 2004. Making better decision makers. Decision Analysis 1:193-204. http://dx.doi.org/10.1287/deca.1040.0009

Knight, A. T., R. M. Cowling, and B. M. Campbell. 2006. An operational model for implementing conservation action. Conservation Biology 20:408-419. http://dx.doi.org/10.1111/j.1523-1739.2006.00305.x

Knight, A. T., R. M. Cowling, M. Rouget, A. Balmford, A. T. Lombard, and B. M. Campbell. 2008. Knowing but not doing: selecting priority conservation areas and the research-implementation gap. Conservation Biology 22:610-617. http://dx.doi.org/10.1111/j.1523-1739.2008.00914.x

Ludwig, D. 2001. The era of management is over. Ecosystems 4:758-764. http://dx.doi.org/10.1007/s10021-001-0044-x

Marinoni, O., P. Adkins, and S. Hajkowicz. 2011. Water planning in a changing climate: joint application of cost utility analysis and modern portfolio theory. Environmental Modelling & Software 26:18-29. http://dx.doi.org/10.1016/j.envsoft.2010.03.001

McFadden, J. E., T. L. Hiller, and A. J. Tyre. 2011. Evaluating the efficacy of adaptive management approaches: is there a formula for success? Journal of Environmental Management 92:1354-1359. http://dx.doi.org/10.1016/j.jenvman.2010.10.038

Meadow, A. M., D. B. Ferguson, Z. Guido, A. Horangic, G. Owen, and T. Wall. 2015. Moving toward the deliberate co-production of climate science knowledge. Weather, Climate, and Society 7:179-191. http://dx.doi.org/10.1175/WCAS-D-14-00050.1

National Wildlife Refuge System Improvement Act. 1997. Public Law no. 105-57 §7, 111 Stat. 1252 (1997).

Pahl-Wostl, C. 2009. A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes. Global Environmental Change 19:354-365. http://dx.doi.org/10.1016/j.gloenvcha.2009.06.001

Pielke, R. A., Jr. 2007. The honest broker: making sense of science in policy and politics. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/cbo9780511818110

Pimm, S. L. 2000. Against triage. Science 289:2289. http://dx.doi.org/10.1126/science.289.5488.2289

Possingham, H. P. 1997. State-dependent decision analysis for conservation biology. Pages 298-304 in S. T. A. Pickett, R. S. Ostfeld, M. Shachak, and G. E. Likens, editors. The ecological basis of conservation: heterogeneity, ecosystems, and biodiversity. Chapman & Hall, New York, New York, USA. http://dx.doi.org/10.1007/978-1-4615-6003-6_30

Skinner, D. C. 2009. Introduction to decision analysis. Probabilistic Publishing, Sugar Land, Texas, USA.

Walters, C. J. 1986. Adaptive management of renewable resources. MacMillan, New York, New York, USA.

Watts, M. E., I. R. Ball, R. S. Stewart, C. J. Klein, K. Wilson, C. Steinback, R. Lourival, L. Kircher, and H. P. Possingham. 2009. Marxan with Zones: software for optimal conservation based land- and sea-use zoning. Environmental Modelling & Software 24:1513-1521. http://dx.doi.org/10.1016/j.envsoft.2009.06.005

Williams, B. K. 1989. Review of dynamic optimization methods in renewable natural resource management. Natural Resource Modeling 3:137-216.

Williams, B. K. and F. A. Johnson. 2013. Confronting dynamics and uncertainty in optimal decision making for conservation. Environmental Research Letters 8:025004. http://dx.doi.org/10.1088/1748-9326/8/2/025004

Williams, B. K., J. D. Nichols, and M. J. Conroy. 2002. Analysis and management of animal populations. Academic Press, San Diego, California, USA.