INTRODUCTION

The Glen Canyon Dam Adaptive Management Program was established in 1997, to manage the Colorado River through Glen Canyon National Recreation Area and Grand Canyon National Park (Fig. 1, Hamill and Melis 2012). Glen Canyon Dam is operated by the Bureau of Reclamation mainly to store water to ensure downstream water deliveries. Annual flood peaks were drastically reduced after dam closure in 1963; the river's base flow nearly doubled (Topping et al. 2003), and cold, clear dam releases replaced seasonally varied water temperatures and highly turbid natural flows (Voichick and Topping 2014, Vernieu 2013). Initially, Glen Canyon Dam's hydroelectric peaking energy production created wide daily river fluctuations, but the dam was eventually re-operated to modified low fluctuating flows (Table 1) in 1996 (U.S. Department of the Interior and Office of the Secretary of the Interior 1996). Modified low fluctuating flows was the preferred alternative identified in a 1995 environmental impact statement (Bureau of Reclamation 1995), and was implemented as an adaptive strategy for improving downstream resources of the Colorado River ecosystem, defined as the river segment from Glen Canyon Dam to the westernmost boundary of Grand Canyon National Park.

Since 1996, management experiments have included additional flow and nonflow treatments ranging from alteration of daily-to-seasonal patterns of dam releases. These include high flow experiments intended to rebuild and maintain sandbar areas; low summer steady flows, fall steady flows, and high steady flows intended to benefit native fish and/or deliver water; and trout management flows that are higher winter fluctuations to disadvantage non-native rainbow trout (Oncorhynchus mykiss). Nonflow treatments have included non-native fish removal in Grand Canyon National Park, regulation of trout fishing, translocation of native fish within and between tributaries, and regulation of recreational raft trips to prevent overcrowding of limited campsites and damage to cultural resources (see Table 1 for details on flow treatments).

A federal advisory committee, the Adaptive Management Program evaluates performance of dam operations and nonflow treatments on Colorado River ecosystem resources by using monitoring and research results to make recommendations to the U.S. Department of the Interior (Gloss et al. 2005). The Adaptive Management Program is a complex, multi-objective program, in the sense that it aims to use not one, but an array of simultaneous treatments to improve downstream resources ranging from the Glen Canyon National Recreation Area's non-native rainbow trout tailwater fishery, to sandbar campsites (Figs. 2 and 3), endangered fish (humpback chub, Gila cypha) in Grand Canyon National Park (Fig. 4), the riparian terrestrial ecosystem (vegetation, birds, mammals), and cultural resources. The Adaptive Management Program has been criticized for its lack of quantifiable targets, for not advancing long-term changes in management in response to learning (Susskind et al. 2012), for not taking adequate account of particular stakeholder perspectives (Dongoske et al. 2010, 2015), and for perhaps being collaborative to a fault (Feller 2008, Camacho 2008). However, the program has produced valuable insights about the efficacy of some policy options—particularly the influences of hydropower peaking, steady flows for native fish, and high flow experiments to rebuild sandbars—and will likely continue to do so well into the future. Resolving resource trade-offs in managing the many diverse Adaptive Management Program objectives (Berkley 2013) is further complicated by a lack of clear resource prioritization, which in turn presents another challenge for stakeholders (Scarlett 2013)..

Experience with the Adaptive Management Program supports our view that misunderstandings about the basic aims of adaptive management often exist, with scientists asserting that adaptive management is conducted to improve scientific understanding of ecosystem function, and managers often asserting that such programs provide the monitoring information required to take corrective action if policy outcomes differ from predictions. The original definition of adaptive management (Walters and Hilborn 1976, Holling 1978) had no aim for gaining understanding of ecosystem function per se. Instead, the basic purpose of adaptive management was to learn how to better manage complex and uncertain systems, i.e., to discover policy options for improving management performance without regard to whether such discoveries might entail improved understanding. In the face of high uncertainty about the efficacy of various policy options, early adaptive management proponents viewed each option as an experimental treatment choice, and they viewed the conduct of adaptive management to be a large-scale experiment in which each treatment represents “probing for opportunity” to improve management performance. In this view, the main help needed from science is in identifying potentially effective policies, in the experimental design process, and in the design of monitoring that measures performance.

After 40 years of adaptive management, it now seems that such a position may have been too extreme. Inferences based on a comparison of alternate policies are most often quite weak because field experiments are often confounded by other priorities, such as prior water delivery agreements or necessary decisions made in response to changes in hydrology that govern water transfers. Field treatments also usually lack adequate replication or controls, and essential monitoring data are often limited, noisy, or lacking altogether. In such situations, at best a weight-of-evidence approach may lead to improved understanding for some aspects of ecosystem function, and this then may be combined with the basic policy comparison to help identify the most likely causal factors, strengthen the overall inferences from the experiments, and identify and screen new policy options. Therefore, we believe it is important for scientists involved in adaptive management programs to recognize that these projects are not inherently science endeavors, but are often quite complex societal collaborations where managers must identify effective management strategies under varying uncertainty and limited resources, including time (Pulwarty and Melis 2001, Walters 1997).

We first review the status of the Adaptive Management Program, emphasizing the role of ecosystem modeling and surprise learning from monitoring that has influenced continued Colorado River ecosystem experimentation (Table 1, Appendix 1). From this history, we see the likelihood for continued surprise learning, particularly under climate change (Brekke et al. 2009, Cook et al. 2015), challenges in forecasting long-lead streamflow volumes (Werner and Yeager 2013); increasingly variable year-to-year basin hydrology (Jain et al. 2005); and slower changes in key ecosystem attributes such as sandbar and riparian vegetation trends, native and non-native fish population trends, and the river's thermal regime and food web. Changes in river temperature may provide particularly fertile ground for surprise learning in the Colorado River ecosystem. We then outline some strategic science directions on the basis of our experience working with the Adaptive Management Program, which we suggest managers consider as inputs for long-term experimental designs aimed at reducing management uncertainties. Our suggestions are made in consideration of the basic aims of adaptive management, and on the basis of surprise learning about Colorado River ecosystem resources that we believe has already been embraced by the Adaptive Management Program (Appendix 1).

EXPERIENCE OF THE VALUE OF ECOSYSTEM MODELING IN ADAPTIVE MANAGEMENT

We suggest that a key initial step in adaptive management is to bring scientists and resource managers together to construct an ecosystem model focused on key resources. The aim of this type of modeling is not to make precise predictions, but rather to: (1) gain consensus about policy options to be tested; (2) identify key biophysical linkages that are likely to determine responses to the policies (and indirect interactions between options); (3) identify key uncertainties that prevent prediction of policy responses based on available data and experience; and (4) “screen” policy options to eliminate those that are very unlikely to benefit resources, but which may be costly in time and funds. In the Adaptive Management Program, initial modeling began during the 1990–1995 preparation of the Operation of Glen Canyon Dam, Colorado River Storage Project, Arizona: Final Environmental Impact Statement (Bureau of Reclamation 1995), which provided qualitative and quantitative predictions about hydropower and downstream resource responses to seven flow-only alternatives, plus spring controlled floods to mimic natural disturbance. Lovich and Melis (2007) later evaluated available Colorado River ecosystem monitoring data relative to Environmental Impact Statement predictions and concluded that only about half of the predictions had been correct.

A more formal Adaptive Management Program ecosystem modeling effort from 1997 to 2003, followed the Environmental Impact Statement. Ecosystem modeling workshops were coordinated by the Grand Canyon Monitoring and Research Center in collaboration with Adaptive Management Program stakeholders and cooperating scientists, and resulted in the Grand Canyon Ecosystem Model (Walters et al. 2000). The Grand Canyon Ecosystem Model is a large and complex spatial model representing key ecosystem indicators within unique geomorphic segments of the 470 km-long Colorado River ecosystem. Each segment is divided into a set of vertical layers from the river bottom to the upper part of the riparian zone. Predicted changes in food web structure are attempted by simulating a set of interacting indicator animal species (e.g., native and non-native fishes, swallows, ducks, falcons) using age-structured population dynamics models with recruitment rates linked to physical habitat factors, food availability, and predator-prey interactions (e.g., predation by non-native fish, particularly trout on endangered humpback chub). The Grand Canyon Ecosystem Model built on historical flow, sediment, and temperature monitoring data from the Colorado River ecosystem, and on an existing hydrologic operations model used by the Bureau of Reclamation, to schedule water deliveries throughout the Colorado River basin. However, as with the Environmental Impact Statement, most Grand Canyon Ecosystem Model biotic response predictions are considered to be highly uncertain (Table 2 in Walters et al. 2000), and some are now known to be wrong on the basis of new monitoring information provided to the Adaptive Management Program by its science provider, the Grand Canyon Monitoring and Research Center.

From the perspective of resource managers, it may seem reasonable to hope that models like the Grand Canyon Ecosystem Model will eventually be improved enough to correctly predict efficacy of various treatments. Such predictive power would then allow development of “best practice” policies without costly and time-consuming field tests (Olden et al. 2014) of management options (as argued by Van Winkle et al. 1997). Since development of the Grand Canyon Ecosystem Model more than a decade ago, this has led some Adaptive Management Program stakeholders to support funding of continuing updates and “calibration” of the ecosystem model, but apparently without recognition that such efforts must ultimately fall short of their expectations (as argued by Castleberry et al. 1996).

We believe that such hoped-for modeling outcomes are typically dashed because these types of ecosystem models are inevitably used to predict responses to management treatments that take the ecosystem into states for which there are no historical data or experiences to draw upon. Any such extrapolation is likely to produce highly uncertain predictions for any ecosystem as complex as the highly altered Colorado River ecosystem. This was the case with the Grand Canyon Ecosystem Model, and earlier predictions regarding operation of selective withdrawal structures (formerly proposed for the Glen Canyon Dam powerplant, but never built) to warm dam releases as an engineering technique for increasing native fish recruitment in Grand Canyon National Park. Moderate downstream warming achieved through use of selective withdrawal structures at the dam is generally predicted to increase both native and non-native fishes, and their interactions (see Table 2 of Walters et al. 2000 and Table 4 of Schmidt et al. 1998). Also, some ecosystem modeling predictions in the Environmental Impact Statement were prone to fail owing to incorrect assumptions about physical processes, such as sediment transport in river settings such as the Colorado River ecosystem, as shown by Rubin et al. (2002), and uncertain changes in future basin hydrology and streamflow under continued warming of the southwestern United States (Jain et al. 2005, Milly et al. 2008, Vano et al. 2014). Since Environmental Impact Statement and Grand Canyon Ecosystem Model modeling was first attempted, hydrologic change (Georgakakos et al. 2014) and increasing water demand (Bureau of Reclamation 2012) have combined to present a particularly slow but critical ecosystem driver that is likely to affect hydroelectric energy (Hibbard et al. 2014) and downstream resources, and perhaps most importantly, the Colorado River ecosystem's thermal regime and aquatic species (Olden and Naiman 2010).

Despite the shortcomings of the Grand Canyon Ecosystem Model, we have seen the Adaptive Management Program's ecosystem modeling experience to be extremely valuable in focusing management awareness of key uncertainties about policies that may be very costly in terms of both funding and risk, but likely have little potential to perform. Also, ecosystem modeling increased understanding about the value of consistent monitoring (see King et al. 2015) in areas where data were either previously missing or not adequate to resolve uncertainties (Walters et al. 2000, Coggins and Walters 2009, Wright et al. 2010, Draut 2012, Korman et al. 2012, Walters et al. 2012, Cross et al. 2013, Grams et al. 2013, Sankey and Draut 2014).

Construction and initial testing with the Grand Canyon Ecosystem Model, other existing models, and independent analyses, including the Environmental Impact Statement, revealed a number of uncertainties about responses of key resources to Glen Canyon Dam policy options. Depending on particular quantitative (and highly uncertain) parameter settings, the Grand Canyon Ecosystem Model (and more recent analyses of particular resources using other various models) provides uncertain and/or widely divergent predictions about a number of management treatments, either those proposed or implemented to date, including the following.

• High flow experiments (tested in 1996, 1997, 2000, 2004, 2008, 2012, 2013 and 2014) to rebuild the river's eroded sandbars—yet their long-term fate under increasingly variable basin hydrology, highly limited and variable downstream tributary sand supply, hydropeaking operations, and water transfers remains quite uncertain.
• River warming (occurred 2003 to 2011 and again in 2014, from reduced Lake Powell storage) to benefit native fish—but the Grand Canyon Ecosystem Model predicts that non-native fish will also benefit, potentially at increased risk to humpback chub from likely increases in native and non-native fish interactions in critical river segments within Grand Canyon National Park.
• Seasonally adjusted steady flows (not yet tested) to mimic a more natural flow pattern by eliminating daily hydropeaking flow variation, combined with spring-timed high flow experiments—were predicted by the Environmental Impact Statement to favor native fish and sandbars, but the Grand Canyon Ecosystem Model and later models suggest that such flows are likely to erode sandbars unless extra sand is added below the dam, and might also impact humpback chub by increasing trout in the Glen Canyon National Recreation Area that leads to downstream migration into Grand Canyon National Park.
• Modified low fluctuating flows (1996 to present) were predicted by the Environmental Impact Statement to accumulate multiyear inputs of tributary sand in Grand Canyon National Park for sandbar building during occasional high flow experiments, and provide benefits to native fish and the Glen Canyon National Recreation Area's tailwater fishery—but monitoring and modeling revealed rapid export of new tributary sand inputs, and trout increases in Grand Canyon National Park under such operations.

The main responses by Adaptive Management Program resource managers to ecosystem modeling uncertainties have been to: (1) implement focused monitoring and research of aquatic and terrestrial resources, i.e., sandbars, native fish, trout, the river's food base, and water quality (particularly, suspended-sediment transport, turbidity, dissolved oxygen, and water temperature, see http://www.gcmrc.gov/discharge_qw_sediment/); and (2) implement two separate 10-year-long experiments (2012 to 2020). The first, being a high flow experiment protocol for consistently implementing one to two high flow experiments of varying duration and magnitude annually, following tributary sand inputs from the Paria River (Fig. 1) in the fall and/or spring to test uncertain predictions about gain/loss in river sandbars. And the second is a non-native fish control action plan under which a series of physical and biological triggering criteria are monitored to make decisions about implementation of complicated and costly fish removals to limit interactions between native and non-native fish in Grand Canyon National Park. The non-native fish control plan also mentions the possibility of using variations of the previously tested in 2003–2006 trout management flow dam operations to limit rainbow trout recruitment in the Glen Canyon National Recreation Area tailwater fishery.

The main Adaptive Management Program uncertainties associated with these two policy experiments are to determine whether repeated high flow experiments can rebuild and maintain sandbars at a faster rate than daily flow fluctuations erode them in order to achieve increased sandbar area, and to determine whether there is a feasible strategy for sustainable concurrent management of trout and native fish of the Colorado River ecosystem. Three previously mentioned experiments have also occurred that we consider as outcomes of either the Environmental Impact Statement, or of discussions during later ecosystem modeling and assessment workshops (Grand Canyon Monitoring and Research Center 2008, Table 1): (1) a short-term (one summer) test of low summer steady flows (Ralston 2011) for enhancing native fish survival by both stabilizing shoreline habitats and allowing warming of water 125 km downstream of the dam near the Little Colorado River confluence, where juvenile humpback chub reside in the Colorado River ecosystem after dispersing from tributary spawning areas (Fig. 5); (2) an unanticipated, extended period of moderately warmer water releases from the dam resulting from reduced Lake Powell storage since 2003 (Vernieu 2013); and (3) the fall steady flows experiment implemented annually from 2008 to 2012, consisting of normal modified low fluctuating flows operations most of the year, but with fall steady flows at various levels aimed at providing favorable river shoreline nursery habitat conditions for juvenile humpback chub once they enter the Colorado River ecosystem (Gerig et al. 2014, Dodrill et al. 2014, and Finch et al. 2013).

CRITICAL ROLE OF SURPRISE IN THE ADAPTIVE MANAGEMENT PROGRAM, AS AN OPPORTUNITY FOR ADAPTIVE LEARNING

Results from experimental manipulations that have been carried out by the Adaptive Management Program, and from assessments of documented ecosystem model predictions about downstream resource responses, provide an opportunity to review surprise resource responses in the Colorado River ecosystem since 1995. We provide a tally of these learning opportunities in Appendix 1, starting with sediment, but then emphasizing native and non-native fish resources where we believe the largest uncertainties persist relative to dam operations and nonflow treatments. By “surprise” we mean any outcome that was not widely recognized as plausible by scientists and Adaptive Management Program members (see Pine et al. 2009 for examples). We do not mean to imply that no one identified the response as a possibility; for any outcome that we can imagine, some scientist or stakeholder will surely have thought of it and considered it possible or even likely, but was not successful in promoting it as a hypothesis worth considering in ecosystem modeling and experimental planning.

A fundamental notion about adaptive processes in general, and adaptive management in particular, is that we actually learn the most from ecosystem models when they fail to predict what we later observe, i.e., when nature surprises us by behaving differently than we expected. When an ecosystem behaves just as expected (or as our ecosystem models have predicted, or as we predicted before conducting an experiment), we actually learn little, in the sense that the results simply conform to our expectations. In this case, costly monitoring merely provides confirmation of what is likely already known. Importantly, observing the same behavior as an ecosystem model has predicted (i.e., having the model “fit well” or appear to be “well-calibrated”) does not in any sense imply that it was based on correct structural assumptions or that the model will correctly predict outcomes of different treatments in the future. All we can say when an ecosystem model fits, i.e., when we do not see surprise in the form of false model predictions, is that it is likely one member of some set of alternative ecosystem models (each of which might make very different predictions about other policy options) that make the same predictions about manipulations conducted to date. In simpler terms, it is probably the case that when our ecosystem model makes correct predictions, that we have been lucky so far with it. However, we must not assume that this “performance” will continue when the ecosystem model is used to predict the efficacy of other, as yet untested, Adaptive Management Program treatments or future hydrologic scenarios that the Colorado River ecosystem has not yet experienced and we have not monitored.

On the basis of the Adaptive Management Program surprise learning summarized in Appendix 1, we suggest that a more effective adaptive management strategy, once ecosystem modeling is initially completed and monitoring is in place, may be to “embrace uncertainty”. This means that managers and scientists fully expect surprises in key resource responses; viewing them as inevitable, but also as valuable learning opportunities. This might mean not building more ecosystem models, or working harder to calibrate and improve existing ones, for they have likely already served their main intended purpose. Rather, it could mean looking carefully at surprise responses and then seeing each of them as possible “learning” opportunities that help identify options for guiding adaptation and promoting ecosystem resiliency under increasingly variable and changing environmental conditions.

Perhaps the most important question to ask, then, is why were we surprised; why were different outcomes expected? One example is the original, but ultimately false, assumptions tied to the Environmental Impact Statement sandbar conservation strategy proposed for Glen Canyon Dam operations. Monitoring and research results that refuted the Environmental Impact Statement assumptions were summarized by Grand Canyon Monitoring and Research Center scientists (Rubin et al. 2002, Wright et al. 2005; Appendix 1) and suggestions for an alternative experimental approach were eventually accepted by the Adaptive Management Program. Those findings and suggestions from scientists eventually resulted in the 2012–2020 high flow experiment protocol (Wright and Kennedy 2011) that appears, so far, to be improving sandbars (Grams et al. 2015). From such outcomes, we then find opportunities to develop alternative hypotheses about other important and sometimes related surprises, such as spring-timed high flow experiments dramatically increasing rainbow trout recruitment rates (Korman et al. 2011). Such surprise learning may then lead to the development of better experiments that test those new hypotheses and help identify other policy changes that the results may imply (Korman and Melis 2011). In adaptive management we believe that embracing uncertainty combined with this kind of critical review and assessment of surprise outcomes is likely the most efficient path for learning about complex systems.

DISCUSSION

Following after the information presented in Appendix 1, we know that the main native fishes of concern in Grand Canyon National Park—i.e., the humpback chub (Gila cypha), and bluehead and flannelmouth suckers (Catostomuus discobolus and Catostomuus latipinnis), are long-lived and have shown stable adult survival rates. This means that, barring some catastrophe like a toxic spill in the Little Colorado River, their future abundances will be determined largely by recruitment rates of fish to about age 2 years; survival rates of fish younger than 2 years old have been highly variable, and have shown a major increase over the last decade (Figs. A1.1-A1.3). Monitoring data and more recent humpback chub modeling research by Yackulic et al. (2014) suggest to us that this increase has been due largely or entirely to the reappearance of juvenile chub rearing in the Colorado River ecosystem, after a period in the l990s when we think that most juveniles entering the mainstem from the Little Colorado River did not survive (Fig. A1.1), and is correlated with increases in dam release temperatures (Fig. A1.4) and declining trout abundance in the Glen Canyon National Recreation Area and Grand Canyon National Park from 2000 to 2006 (Fig. A1.5). River warming from 2003 to 2011, resulted from reduced Lake Powell storage, peaked in 2005 and coincided with minimum rainbow trout abundances in 2005 to 2006. Causes of the trout decline are not fully understood, and it began well before a non-native fish removal experiment near the Little Colorado River confluence was implemented in 2003 to 2006 (Table A1.1, see Coggins et al. 2011). The 2000 low summer steady flows and 2008-to-2012 fall steady flows tests apparently did not measurably help juvenile humpback chub. In the fall steady flows case, researchers also report that reduced fall-season flow variations were coincident with reduced growth and survival of juvenile chub for reasons that remain unclear. The surprise learning about modified low fluctuating flows operations, and other flow and nonflow treatments described in Appendix 1, has informed the Adaptive Management Program well beyond what was known and reported about downstream resources in the 1995 Environmental Impact Statement. Frequent modeling assessment workshops have effectively informed stakeholders about unexpected outcomes and have helped managers focus discussions on experimental design, monitoring, and the critical need to avoid confounding treatments whenever possible.

So what are the possible implications of this learning for next-phase evaluation of management options over the next 20 years, which is the period recently proposed for a Long-Term Experimental and Management Plan for Glen Canyon Dam operations (see http://ltempeis.anl.gov/)? Here are some things that we can suggest to the Adaptive Management Program for consideration, on the basis of current information we believe is relatively certain:

• Influences of continued, and potentially more frequent high flow experiments (particularly in spring, but also possibly in any season), and/or seasonally adjusted steady flow tests (see Bureau of Reclamation 1995), would very likely result in high abundances of rainbow trout in the Glen Canyon National Recreation Area. And, many of these fish will likely find their way downstream to the Little Colorado River confluence area where chub are found, despite any food base/habitat enhancement that such flows may cause upstream in the Glen Canyon National Recreation Area.
• From prior experiments, it is very uncertain whether trout control measures alone (either trout management flows aimed at juvenile rainbow trout stranding, or removal programs either at the Little Colorado River confluence area or upstream) will result in high enough native fish recruitments to maintain the desired humpback chub population level, particularly during periods of cold water dam releases from Lake Powell (as occurred from 1990 to 2002, and again in 2012 to 2013). Hence, we believe that the most pressing experimental need is to determine whether low trout abundance in Grand Canyon National Park alone is enough to ensure adequate humpback chub recruitment, independent of Colorado River ecosystem water temperature.
• More than likely there is a “sweet spot”—in terms of modest downstream Colorado River ecosystem water temperature increase combined with relatively lower trout abundance—that will likely continue to produce strong native fish recruitment without stimulating increases in non-native warm-water species and which could also negatively impact native fish. Staying in this sweet spot may not even require low rainbow trout abundance near the Little Colorado River, but very likely does require lowered abundance of brown trout (see Yard et al. 2011). Hence, we suggest that the second-most pressing experimental need is to determine whether modest increases in the Glen Canyon Dam summer release temperatures can be achieved through some acceptable engineered system, or by low-flow summer dam releases in years when chub production is high in the Little Colorado River and large numbers of juveniles enter the Colorado River ecosystem.
• Should it prove necessary to reduce rainbow trout abundance at the Little Colorado River confluence, by the time this is clearly demonstrated the Adaptive Management Program hopefully will have already tested various means to limit trout abundance in Grand Canyon National Park, i.e., through methods ranging from direct removal to manipulations of flow to reduce juvenile trout survival in the Glen Canyon National Recreation Area. We believe that what is also most needed are replicated tests of a variety of trout control measures, in combination with ongoing high flow experiments, steady flows, or other high spring releases that are likely to result in high rainbow trout recruitment, by applying treatment options in short (1 year) tests distributed over at least a decade, but possibly longer, with continued monitoring to determine where in the trout life cycle these tests might be effective (or not).
• There is no good evidence from the Adaptive Management Program that steady flow regimes, such as the previously tested 2000 low summer steady flows and the 2008-to-2012 fall steady flows tests, would benefit species besides rainbow and, possibly, brown trout. In fact, recent synthesis research from outside the Colorado River ecosystem that relates flow regimes to life history strategies of various fish assemblages also supports the idea that needs of non-native trout are best met by steadier flow regimes like modified low fluctuating flows operations (Mims and Olden 2012, 2013).
• Average Colorado River ecosystem warming in the main humpback chub rearing area immediately downstream of the Little Colorado River confluence would be very modest when dam releases are cold, as they were in summer 2000, and more recently in 2012–2013, that is unless summer dam releases are further reduced below levels previously tested. We realize that this may not be an appealing option to some interests owing to resource trade-offs associated with such low summer flows (i.e., meeting electrical energy demands during peak use months, and achieving annual water transfer obligations by October 1st), but such operations are also known to protect sandbars and retain Paria River sand inputs typically delivered to the Colorado River ecosystem in summer months.

SPECULATION

Monitoring of the Colorado River ecosystem from 1995 to 2003 showed only half of the Environmental Impact Statement predictions about modified low fluctuating flows dam operations to be valid. Many Adaptive Management Program members anticipated benefit to downstream resources from the preferred alternative, but some maintained that only steady and warmed dam releases would achieve sandbar and endangered fish objectives. Although the Environmental Impact Statement hypothesized that high flow experiments might rebuild eroded sandbars, most modified low fluctuating flows operations did not allow multiyear accumulation of tributary sand inputs as predicted, and sandbar erosion continued. Scientists later showed that only high flow experiments released soon after tributary sand inputs could increase sandbars in ways needed to increase camping sites, but then open camping areas were reduced by expanding riparian vegetation. Steady flows have not been shown to benefit humpback chub, but modified low fluctuating flows, steady releases, and spring flooding have benefited non-native rainbow trout, possibly at the risk to native fish. Surprise learning about sediment and fish, and more integrated monitoring during an extended period of warmer dam releases not anticipated in the Environmental Impact Statement, resulted in two 10-year-long condition-dependent experiments that began in 2012. We believe that these two adaptive management treatments reflect a willingness of Colorado River ecosystem managers to embrace uncertainties and seek strategies for achieving resource goals. Embracing uncertainty has advanced the Adaptive Management Program.

As such, it is not hard to imagine that any other longer term experimental management plan for operating Glen Canyon Dam is likely to include feedback (contingency) rules for implementing all proposed management treatments (flow or otherwise). There are already triggering criteria for the high flow experiment protocol and non-native fish removals, and it seems reasonable that similar resource-dependent strategies be identified for trout management flows and alternative daily operating rules tied to hydropower as part of any long-term experiment. There would presumably also be need for contingency rules for when to abandon poor-performing treatments, i.e., for how many positive/negative replicates to evaluate before committing to any course of action. We suggest that Colorado River ecosystem resource managers, and others undertaking equally complex adaptive management programs, consider observing at least two to three results from each option, and seeing if at least one of the first two is positive, before deciding to abandon any policy. We also suggest that continuing to use existing Adaptive Management Program ecosystem models as planning tools to examine such contingency rules in terms of the odds of making each of the possible incorrect conclusions (drop an action that is working, accept an action that is not) would be appropriate. Further, it seems to us that there is a need for the Adaptive Management Program to clearly evaluate trade-offs associated with potentially implementing either of the following strategies (among others proposed) in terms of how to deal with the lingering uncertainty about river temperature and/or non-native trout effects on native fish populations:

• Passive temperature testing: focus initial experimentation on trout control measures, using whatever temperature pattern that Lake Powell and daily-to-seasonal dam operations provide to examine combined temperature and trout effects. A basic problem with this approach is that there is no assurance of seeing informative combinations (Table 2), and in particular of seeing warmed conditions for long enough to allow trout to increase so as to test whether warming alone is sufficient to cause strong native fish recruitment. This is basically the type of plan that scientists have suggested to the Adaptive Management Program in past assessment and modeling workshops (Grand Canyon Monitoring and Research Center 2008), with an option for long-term investment in selective withdrawal structures should river warming prove necessary;
• Active temperature testing: seek immediate implementation of some acceptable method for maintaining modest river temperature increases (some options discussed by scientists include a temporary lowering of Lake Powell storage, average summer releases below 226 m³/s, and forebay mixing impellers), then insure that trout control options do not prevent observation of the warmer river/higher trout condition, i.e., initially avoid treatments that are very likely to cause substantial trout reduction.

In terms of the long-run objectives of maintaining desired sandbar area, a viable Glen Canyon National Recreation Area tailwater fishery, limits on non-native fish below Glen Canyon National Recreation Area, and Grand Canyon National Park humpback chub population sizes large enough to meet desired future conditions, we find the second option to be preferable for learning unless costs/risks associated with viable options for river warming are unacceptable to decision makers.

The Long-Term Experimental and Management Plan process has been underway since 2011, and has revealed at least one productive alternative to the earlier 1997-2003 adaptive management ecosystem modeling approach of developing a single large model. A key reason the Grand Canyon Ecosystem Model was not used extensively by the Adaptive Management Program for policy exploration and screening was that it was just too complex; requiring specification of many parameters to define each policy run. In more recent experimental planning we have observed much more extensive use of focused submodels or “mini-models” as tools for making particular predictions, and some of these appear to be quite reliable for answering particular policy questions.

For example, there was an initial emphasis in long-term experimental design scoping workshops on identification of a wide range of basic seasonal/diurnal dam operating options, ranging from “load-following” flows to optimize power production, to seasonally adjusted steady flow aimed at restoring a more natural seasonal hydrograph pattern (advocated by supporters of the “natural flow paradigm” for river management, Poff et al. 1997). Initial comparison of suspended-sand transport simulations for these alternatives, using the relatively simple but well-calibrated sediment transport model of Wright et al. (2010), quickly revealed that the seasonally adjusted steady flow alternative (as originally described in the 1995 Environmental Impact Statement) would likely result in substantial losses of limited Colorado River ecosystem sand required to rebuild and maintain sandbars, i.e., that restoring flow pattern without the pre-dam sand supply would make the system less rather than more natural (Wright and Grams 2010). This might support an informed option by managers to screen out seasonally adjusted steady flow as a very poor experimental strategy for benefitting sandbars, and to focus on a narrower set of long-term alternatives (http://ltempeis.anl.gov/documents/docs/LTEMP_Alternatives_April_2014.pdf). Apparently, screening of treatments has omitted sediment augmentation and more robust, active temperature management of Glen Canyon Dam releases, and long-term design for experimental alternatives remains focused on a restricted set of treatments that might benefit terrestrial sediment-related resources, but might still not generate the substantial benefits for aquatic ecosystem and endangered fish objectives that some stakeholders required in 1995.

LITERATURE CITED

Berkley, J. 2013. Opportunities for collaborative adaptive management progress: integrating stakeholder assessments into progress measurement. Ecology and Society 18(4):69. http://dx.doi.org/10.5751/es-05988-180469

Brekke, L. D., J. E. Kiang, J. R. Olsen, R. S. Pulwarty, D. A. Raff, D. P. Turnipseed, R. S. Webb, and K. D. White. 2009. Climate change and water resources management: a federal perspective. U.S. Geological Survey Circular 1331. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1331/Circ1331.pdf

Bureau of Reclamation. 1995. Operation of Glen Canyon Dam, Colorado River storage project, Arizona: final environmental impact statement. Bureau of Reclamation, Upper Colorado Regional Office, U.S. Department of the Interior, Salt Lake City, Utah, USA. [online] URL: http://www.usbr.gov/uc/library/envdocs/eis/gc/pdfs/Cov-con/cov-con.pdf

Bureau of Reclamation. 2012. Colorado River Basin water supply and demand study—study report. Bureau of Reclamation, U.S. Department of the Interior, Boulder City, Nevada, USA. [online] URL: http://www.usbr.gov/lc/region/programs/crbstudy/finalreport/Study%20Report/StudyReport_FINAL_Dec2012.pdf

Camacho, A. E. 2008. Beyond conjecture: learning about ecosystem management from the Glen Canyon Dam experiment. Nevada Law Journal 8(3):942-963.

Castleberry, D. T., J. J. Cech, Jr., D. C. Erman, D. Hankin, M. Healey, G. M. Kondolf, M. Mangel, M. Mohr, P. B. Moyle, J. Nielsen, T. P. Speed, and J. C. Williams. 1996. Uncertainty and instream flow standards. Fisheries 21(8):20-21.

Coggins, L. G., Jr., W. E. Pine, III, C. J. Walters, and S. J. D. Martell. 2006a. Age-structured mark-recapture analysis: a virtual population-analysis-based model for analyzing age-structured capture-recapture data. North American Journal of Fisheries Management 26(1):201-205.

Coggins, L. G. Jr., W. E. Pine, III, C. J. Walters, D. R. Van Haverbeke, D. Ward, and H. C. Johnstone. 2006b. Abundance trends and status of the Little Colorado River population of humpback chub. North American Journal of Fisheries Management 26:233-245.

Coggins, L. G., Jr., and C. J. Walters. 2009. Abundance trends and status of the Little Colorado River population of humpback chub: an update considering data from 1989-2008. U.S. Geological Survey Open-File Report 2009-1075. Grand Canyon Monitoring and Research Center, U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. http://dx.doi.org/10.1577/m05-075.1

Coggins, L. G., M. D. Yard, and W. E. Pine. 2011. Non-native fish control in the Colorado River in Grand Canyon, Arizona: an effective program or serendipitous timing? Transactions of the American Fisheries Society 140(2):456-470.

Cook, B. I, T. R. Ault, and J. E. Smerdon. 2015. Unprecedented 21st century drought risk in the American Southwest and Central Plains. Science Advance. 12 February 2015, Vol 1:e1400082.

Cross, W. F., C. V. Baxter, K. C. Donner, E. J. Rosi-Marshall, T. A. Kennedy, R. O. Hall, Jr., H. A. Wellard-Kelly, and R. S. Rogers. 2011. Ecosystem ecology meets adaptive management: food web response to a controlled flood on the Colorado River, Glen Canyon. Ecological Applications 21(6):2016-2033. http://dx.doi.org/10.1890/10-1719.1

Cross, W. F., C. V. Baxter, E. J. Rosi-Marshall, R. O. Hall, Jr., T. A. Kennedy, K. C. Donner, H. A. Wellard Kelly, S. E. Z. Seegert, K. Behn, and M. D. Yard. 2013. Foodweb dynamics in a large river discontinuum. Ecological Monographs 83(3):311-337.

Dodrill, M. J., C. B. Yackulic, B. Gerig, W. E. Pine, III, J. Korman, and C. Finch. 2014. Do management actions to restore rare habitat benefit native fish conservation? Distribution of juvenile native fish among shoreline habitats of the Colorado River. River Research and Applications 1-15. http://dx.doi.org/10.1002/rra.2842

Dongoske, K. E., L. Jackson-Kelly, and C. Bulletts. 2010. Confluence of values: the role of science and native Americans in the Glen Canyon Dam Adaptive Management Program. Pages 133-140. in T. S. Melis, J. F. Hamill, G. E. Bennett, L. G. Coggins, Jr., P. E. Grams, T. A. Kennedy, D. M. Kubly, and B. Ralston, editors. Proceedings of the Colorado River Basin Science and Resource Management Symposium, November 18-20, 2008, Scottsdale, Arizona. Coming Together: Coordination of Science and Restoration Activities for the Colorado River Ecosystem. U.S. Geological Survey Scientific Investigations Report 2010-5135. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA.

Dongoske, K. E., T. Pasqual, and T. F. King. 2015. The National Environmental Policy Act (NEPA) and the silencing of native American worldviews. Environmental Practice 17(1):36-45.

Draut, A. E. 2012. Effects of river regulation on aeolian landscapes, Colorado River, southwestern USA. Journal of Geophysical Research 117:1-22. http://dx.doi.org/10.1029/2011JF002329

Feller, J. M. 2008. Collaborative management of Glen Canyon Dam: the elevation of social engineering over law. Nevada Law Journal. 8(3):101-151.

Finch, C. G., W. E. Pine, III, and K. E. Limburg. 2013. Do hydropeaking flows alter juvenile fish growth rates? A test with juvenile humpback chub in the Colorado River. River Research and Applications 31(2):156-164. http://dx.doi.org/10.1002/rra.2725

Finch, C. G., W. E. Pine, III, C. B. Yackulic, M. J. Dodrill, M. Yard, B. S. Gerig, L. G. Coggins, and J. Korman. 2015. Assessing juvenile native fish demographic responses to a steady flow experiment in a large regulated river. River Research and Applications 12 March 2015. http://dx.doi.org/10.1002/rra.2893

Georgakakos, A., P. Fleming, M. Dettinger, C. Peters-Lidard, T. C. Richmond, K. Reckhow, K. White, and D. Yates. 2014. Chapter 3: water resources. Pages 69-112 in J. M. Melillo, T. C. Richmond, and G. W. Yohe, editors. Climate Change Impacts in the United States: The Third National Climate Assessment. U.S. Global Change Research Program, U.S. National Climate Assessment, Washington, D.C., USA. [online] URL: http://s3.amazonaws.com/nca2014/low/NCA3_Climate_Change_Impacts_in_the_United%20States_LowRes.pdf?download=1

Gerig, B., M. J. Dodrill, and W. E. Pine, III. 2014. Habitat selection and movement of adult humpback chub in the Colorado River in Grand Canyon, Arizona, during an experimental steady flow release. North American Journal of Fisheries Management 34(1):39-48. http://dx.doi.org/10.1080/02755947.2013.847880

Gloss, S., J. E. Lovich, and T. S. Melis, editors. 2005. The state of the Colorado River ecosystem in Grand Canyon: a report of the Grand Canyon Monitoring and Research Center 1991-2004. U.S. Geological Survey Circular 1282. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1282/c1282.pdf

Grams, P. E., J. E. Hazel, J. C. Schmidt, M. Kaplinski, S. A. Wright, D. J. Topping, and T. S. Melis. 2010a. Geomorphic response of sandbars to the March 2008 high-flow experiment on the Colorado River downstream from Glen Canyon Dam. In Proceedings of the 4th Federal Interagency Hydrologic Modeling Conference and the 9th Federal Interagency Sedimentation Conference, Las Vegas, NV, June 27 – July 1, 2010. Hydrology and Sedimentation for a Changing Future: Existing and Emerging Issues. Water Information Coordination Program, Advisory Committee on Water Information, U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://acwi.gov/sos/pubs/2ndJFIC/Contents/5D_Grams.pdf

Grams, P. E., J. C. Schmidt, and M. E. Andersen. 2010b. 2008 High-flow experiment at Glen Canyon Dam: morphologic response of eddy-deposited sandbars and associated aquatic backwater habitats along the Colorado River in Grand Canyon National Park. U.S. Geological Survey Open-File Report 2010-1032. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2010/1032/of2010-1032.pdf

Grams, P. E., J. C. Schmidt, S. A. Wright, D. J. Topping, T. S. Melis, and D. M. Rubin. 2015. Recent controlled floods contribute to sandbar gains in Grand Canyon. Eos 96, June 2015. [online] URL: https://eos.org/features/building-sandbars-in-the-grand-canyon

Grams, P. E., D. J. Topping, J. C. Schmidt, J. E. Hazel Jr., and M. Kaplinski. 2013. Linking morphodynamic response with sediment mass balance on the Colorado River in Marble Canyon: issues of scale, geomorphic setting, and sampling design. Journal of Geophysical Research: Earth Surface 118:1-21. http://dx.doi.org/10.1002/jgrf.20050

Grand Canyon Monitoring and Research Center. 2008. USGS workshop on scientific aspects of a long-term experimental plan for Glen Canyon Dam, April 10-11, 2007, Flagstaff, Arizona. U.S. Geological Survey Open-File Report 2008-1153. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2008/1153/

Hamill, J. F., and T. S. Melis. 2012. The Glen Canyon Dam Adaptive Management Program: progress and immediate challenges. Pages 325-338 in P. J. Boon and P. J. Raven, editors. River Conservation and Management. John Wiley and Sons, Ltd., West Sussex, UK. http://dx.doi.org/10.1002/9781119961819.ch26

Hazel, J. E., Jr., P. E. Grams, J. C. Schmidt, and M. Kaplinski. 2010. Sandbar response in Marble and Grand Canyons, Arizona, following the 2008 high-flow experiment on the Colorado River. U.S. Geological Survey Scientific Investigations Report 2010-5015. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/sir/2010/5015/

Hazel, J. E., Jr., D. J. Topping, J. C. Schmidt, and M. Kaplinski. 2006. Influence of a dam on fine-sediment storage in a canyon river. Journal of Geophysical Research 111(F01025):1-16. http://dx.doi.org/10.1029/2004jf000193

Hibbard, K., T. Wilson, K. Averyt, R. Harriss, R. Newmark, S. Rose, E. Shevliakova, and V. Tidwell. 2014. Chapter 10: energy, water, and land use. Pages 257-281 in J. M. Melillo, T. C. Richmond, and G. W. Yohe, editors. Climate Change Impacts in the United States: The Third National Climate Assessment. U.S. Global Change Research Program, U.S. National Climate Assessment, Washington, D.C., USA. [online] URL: http://s3.amazonaws.com/nca2014/low/NCA3_Full_Report_0a_Front_Matter_LowRes.pdf?download=1

Holling, C. S., editor. 1978. Adaptive environmental assessment and management. International Series on Applied Systems Analysis. John Wiley and Sons, Inc., Chichester, New York, USA.

Holling, C. S. 2001. Understanding the complexity of economic, ecological, and social systems. Ecosystems 4(5):390-405. http://dx.doi.org/10.1007/s10021-001-0101-5

Jain, S., M. Hoerling, and J. Eischeid. 2005. Decreasing reliability and increasing synchroneity of western North American streamflow. Journal of Climate 18:613-618. http://dx.doi.org/10.1175/JCLI-3311.1

Kenney, D., S. Bates, A. Bensard, and J. Berggren. 2011. The Colorado River and the inevitability of institutional change. Public Land & Resources Law Review 32:103-152.

Kennedy, T. A., W. F. Cross, R. O. Hall, Jr., C. V. Baxter, and E. J. Rosi-Marshall. 2013. Native and Non-native Fish Populations of the Colorado River are Food Limited-Evidence from New Food Web Analyses. U.S. Geological Survey Fact Sheet 2013-3039. Grand Canyon Monitoring and Research Center, Southwest Biological Science Center, U.S. Geological Survey, Flagstaff, Arizona, USA. [online] URL: http://pubs.usgs.gov/fs/2013/3039/fs2013-3039.pdf

Kennedy, T. A., and B. E. Ralston. 2011. Biological responses to high-flow experiments at Glen Canyon Dam. Pages 93-125 in T. S. Melis, editor. Effects of three high-flow experiments on the Colorado River ecosystem downstream from Glen Canyon Dam, Arizona. U.S. Geological Survey Circular 1366. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1366/c1366.pdf

King, A. J., B. Gawne, L. Beesley, J. D. Koehn, D. L. Nielsen, and A. Price. 2015. Improving ecological response monitoring of environmental flows. Environmental Management 55(5):991-1005.

Korman, J., M. Kaplinski, and T. S. Melis. 2011. Effects of fluctuating flows and a controlled flood on incubation success and early survival rates and growth of age-0 rainbow trout in a large regulated river. Transactions of the American Fisheries Society 140(2):487-505.

Korman, J., S. J. D. Martell, C. J. Walters, A. S. Makinster, L. G. Coggins, M. D. Yard, and W. R. Persons. 2012. Estimating recruitment dynamics and movement of rainbow trout (Oncorhynchus mykiss) in the Colorado River in Grand Canyon using an integrated assessment model. Canadian Journal of Fisheries and Aquatic Sciences 69(11):1827-1849. http://dx.doi.org/10.1139/f2012-097

Korman, J., and T. S. Melis. 2011. The Effects of Glen Canyon Dam Operations on Early Life Stages of Rainbow Trout in the Colorado River. U.S. Geological Survey Fact Sheet 2011-3002. Grand Canyon Monitoring and Research Center, Southwest Biological Science Center, U.S. Geological Survey, Flagstaff, Arizona, USA. [online] URL: http://pubs.usgs.gov/fs/2011/3002/fs2011-3002.pdf

Lemos, M. C., C. J. Kirchhoff, and R. Ramprasad. 2012. Narrowing the climate information usability gap. Nature Climate Change 2:789-794. http://dx.doi.org/10.1038/nclimate1614

Limburg, K. E., T. A. Hayden, W. E. Pine, III, M. D. Yard, R. Kozdon, and J. W. Valley. 2013. Of travertine and time: otolith chemistry and microstructure detect provenance and demography of endangered humpback chub in Grand Canyon, USA. PLOS ONE 8(12):e84235. http://dx.doi.org/10.1371/journal.pone.0084235

Lovich, J. E., and T. S. Melis. 2007. The state of the Colorado River ecosystem in Grand Canyon: lessons from 10 years of adaptive ecosystem management. International Journal of River Basin Management 5(3):207-221.

Makinster, A. S., W. R. Persons, and L. A. Avery. 2011. Status and trends of the rainbow trout population in the Lees Ferry Reach of the Colorado River downstream from Glen Canyon Dam, Arizona, 1991-2009. U.S. Geological Survey Scientific Investigations Report 2011-5015. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/sir/2011/5015/pdf/sir2011-5015.pdf

Makinster, A. S., W. R. Persons, L. A. Avery, and A. J. Bunch. 2010. Colorado River fish monitoring in Grand Canyon, Arizona: 2000 to 2009 summary. U.S. Geological Survey Open-File Report 2010-1246. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2010/1246/of2010-1246.pdf

McKinney, T., D. W. Speas, R. S. Rogers, and W. R. Persons. 2001. Rainbow trout in a regulated river below Glen Canyon Dam, Arizona, following increased minimum flows and reduced discharge variability. North American Journal of Fisheries Management 21(1):216-222. http://dx.doi.org/10.1577/1548-8675(2001)021<0216:rtiarr>2.0.co;2

Melis, T. S., J. Korman, and T. A. Kennedy. 2012. Abiotic and biotic responses of the Colorado River to a March 2008 controlled flood experiment at Glen Canyon Dam, AZ. River Research and Applications 28(6):764-776.

Melis, T. S., S. J. D. Martell, L. G. Coggins, W. E. Pine, III, and M. E. Andersen. 2006. Adaptive management of the Colorado River ecosystem below Glen Canyon Dam, Arizona: using science and modelling to resolve uncertainty in river management. In proceedings of American Water Resources Association Summer Specialty Conference 2009: Adaptive Management of Water Resources II, Snowbird, Utah, USA, 29 June –1 July 2009. American Water Resources Association, Middleburg, Virginia, USA. [online] URL: http://www.gcmrc.gov/library/reports/Synthesis/Melis2006.pdf

Milly, P. C. D., J. Betancourt, M. Falkenmark, R. M. Hirsch, Z. W. Kundzewicz, D. P. Lettenmaier, and R. J. Stouffer. 2008. Stationarity is dead: whither water management? Science 319(5863):573-574.

Mims, M. C., and J. D. Olden. 2012. Life history theory predicts fish assemblage response to hydrologic regimes. Ecology 93(1):35-45. http://dx.doi.org/10.1890/11-0370.1

Mims, M. C., and J. D. Olden. 2013. Fish assemblages respond to altered flow regimes via ecological filtering of life history strategies. Freshwater Biology 58: 50-62. http://dx.doi.org/10.1111/fwb.12037

Olden, J. D., C. P. Konrad, T. S. Melis, M. J. Kennard, M. C. Freeman, M. C. Mims, E. N. Bray, K. B. Gido, N. P. Hemphill, D. A. Lytle, L. E. McMullen, M. Pyron, C. T. Robinson, J. C. Schmidt, and J. G. Williams. 2014. Are large-scale flow experiments informing emerging challenges in freshwater management? Frontiers in Ecology and the Environment. 12(3):176-185.

Olden, J. D., and R. J. Naiman. 2010. Incorporating thermal regimes into environmental flows assessments: modifying dam operations to restore freshwater ecosystem integrity. Freshwater Biology 55:86-107. http://dx.doi.org/10.1111/j.1365-2427.2009.02179.x

Peterson, J. H., and C. P. Paukert. 2005. Development of a bioenergetics model for humpback chub and evaluation of water temperature changes in the Grand Canyon, Colorado River. Transactions of the American Fisheries Society 134:960-974. http://dx.doi.org/10.1577/T04-090.1

Pine, W. E., III, B. Healy, E. Omana-Smith, M. Trammell, D. W. Speas, R. A. Valdez, M. D. Yard, C. Walters, R. Ahrens, D. R. Van Haverbeke, D. M. Stone, and W. Wilson. 2013. An individual-based model for population viability analysis of humpback chub in the Colorado River in Grand Canyon, Arizona. North American Journal of Fisheries Management. 33(3):626-641.

Pine, W. E. I., S. J. D. Martell, C. J. Walters, and J. F. Kitchell. 2009. Counterintuitive responses of fish populations to management actions: some common causes and implications for predictions based on ecosystem modelling. Fisheries 34(4):165-180. http://dx.doi.org/10.1577/1548-8446-34.4.165

Poff, N. L., J. D. Allan, M. B. Bain, J. R. Karr, K. L. Prestegaard, B. D. Richter, R. E. Sparks, and J. C. Stromberg. 1997. The natural flow regime: a paradigm for river conservation and restoration. BioScience 47(11):769-784. http://dx.doi.org/10.2307/1313099

Pulwarty, R. S., and T. S. Melis. 2001. Climate extremes and adaptive management on the Colorado River: lessons from the 1997-1998 ENSO event. Journal of Environmental Management 63:307-324. http://dx.doi.org/10.1006/jema.2001.0494

Ralston, B. E. 2011. Summary report of responses of key resources to the 2000 low steady summer flow experiment, along the Colorado River downstream from Glen Canyon Dam, Arizona. U.S. Geological Survey Open-File Report 2011-1220. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2011/1220/of2011-1220.pdf

Robinson, C. T., and U. Uehlinger. 2008. Experimental floods cause ecosystem regime shift in a regulated river. Ecological Applications 18(2):511-526. http://dx.doi.org/10.1890/07-0886.1

Rosi-Marshall, E. J., T. A. Kennedy, D. W. Kincaid, W. F. Cross, H. A. W. Kelly, K. A. Behn, T. White, R. O. Hall, Jr., and C. V. Baxter. 2010. Short-term effects of the 2008 high-flow experiment on macroinvertebrates in the Colorado River below Glen Canyon Dam, Arizona. U.S. Geological Survey Open-File Report 2010-1031. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2010/1031/of2010-1031.pdf

Rubin, D. M., J. M. Nelson, and D. J. Topping. 1998. Relation of inversely graded deposits to suspended-sediment grain-size evolution during the 1996 flood experiment in Grand Canyon. Geology 26(2):99-102.

Rubin, D. M., D. J. Topping, J. C. Schmidt, J. Hazel, M. Kaplinski, and T. S. Melis. 2002. Recent sediment studies refute Glen Canyon Dam hypothesis. Eos, Transactions, American Geophysical Union 83(25):273, 277-278. http://dx.doi.org/10.1029/2002eo000191

Sankey, J. B., and A. E. Draut 2014. Gully annealing by aeolian sediment: field and remote-sensing investigation of aeolian-hillslope-fluvial interactions, Colorado River corridor, Arizona, USA. Geomorphology 220:68-80. http://dx.doi.org/10.1016/j.geomorph.2014.05.028

Scarlett, L. 2013. Collaborative adaptive management: challenges and opportunities. Ecology and Society 18(3):26. http://dx.doi.org/10.5751/es-05762-180326

Schmidt, J. C., and P. E. Grams. 2011. The high flows: physical science results. Pages 53-91 in T. S. Melis, editor. Effects of three high-flow experiments on the Colorado River ecosystem downstream from Glen Canyon Dam, Arizona. U.S. Geological Survey Circular 1366. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1366/c1366.pdf

Schmidt, J. C., R. H. Webb, R. A. Valdez, G. R. Marzolf, and L. E. Stevens. 1998. Science and values in river restoration in the Grand Canyon. BioScience 48(9):735-747. http://dx.doi.org/10.2307/1313336

Susskind, L., A. E. Camacho, and T. Schenk. 2012. A critical assessment of collaborative adaptive management in practice. Journal of Applied Ecology 49(1):47-51. http://dx.doi.org/10.1111/j.1365-2664.2011.02070.x

Topping, D. J., D. M. Rubin, P. E. Grams, R. E. Griffiths, T. A. Sabol, N. Voichick, R. B. Tusso, K. M. Vanaman, and R. R. McDonald. 2010. Sediment transport during three controlled-flood experiments on the Colorado River downstream from Glen Canyon Dam, with implications for eddy-sandbar deposition in Grand Canyon National Park. U.S. Geological Survey Open-File Report 2010-1128. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2010/1128/of2010-1128.pdf

Topping, D. J., D. M. Rubin, J. M. Nelson, P. J. Kinzel, III, and J. P. Bennett. 1999. Linkage between grain-size evolution and sediment depletion during Colorado River floods. Pages 71-98 in R. H. Webb, J. C. Schmidt, G. R. Marzolf, and R. A. Valdez, editors. The controlled flood in Grand Canyon. Geophysical Monograph Series, Vol. 110. American Geophysical Union, Washington, D.C., USA. http://dx.doi.org/10.1029/gm110p0071

Topping, D. J., D. M. Rubin, J. C. Schmidt, J. E. Hazel, Jr., T. S. Melis, S. A. Wright, M. Kaplinski, A. E. Draut, and M. J. Breedlove. 2006. Comparison of sediment-transport and bar-response results from the 1996 and 2004 controlled-flood experiments on the Colorado River in Grand Canyon. In Proceedings of the Eighth Federal Interagency Sedimentation Conference, April 2–6, 2006, Reno, Nevada, USA. Subcommittee on Sedimentation, Advisory Committee on Water Information, U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/misc/FISC_1947-2006/pdf/1st-7thFISCs-CD/8thFISC/8thFISC.pdf

Topping, D. J., J. C. Schmidt, and L. E. Vierra. 2003. Computation and analysis of the instantaneous-discharge record for the Colorado River at Lees Ferry, Arizona—May 8, 1921, through September 30, 2000. U.S. Geological Survey Professional Paper 1677. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/pp/pp1677/pdf/pp1677.pdf

U.S. Department of the Interior and Office of the Secretary of the Interior. 1996. Appendix G, record of decision, operation of Glen Canyon Dam: final environmental impact statement, October 1996. Office of the Secretary of the Interior, and Bureau of Reclamation, U.S. Department of the Interior, Washington, D.C., USA. [online] URL: http://www.usbr.gov/uc/rm/amp/pdfs/sp_appndxG_ROD.pdf

Van Haverbeke, D. R., D. M. Stone, L. G. Coggins, Jr., and M. J. Pillow. 2013. Long-term monitoring of an endangered desert fish and factors influencing population dynamics. Journal of Fish and Wildlife Management 4(1):163-177. http://dx.doi.org/10.3996/082012-jfwm-071

Van Winkle, W., C. C. Coutant, H. I. Jager, J. S. Mattice, D. J. Orth, R. G. Otto, S. F. Railsback, and M. J. Sale. 1997. Uncertainty and instream flow standards: perspectives based on hydropower research and assessment. Fisheries 22(7):21-22.

Vano, J. A., B. Udall, D. R. Cayan, J. T. Overpeck, L. D. Brekke, T. Das, H. C. Hartmann, H. G. Hidalgo, M. Hoerling, G. J. McCabe, K. Morino, R. S. Webb, K. Werner, and D. P. Lettenmaier. 2014. Understanding uncertainties in future Colorado River streamflow. Bulletin of the American Meteorological Society 95(1):59-78. http://dx.doi.org/10.1175/bams-d-12-00228.1

Vernieu, W. S. 2013. Historical physical and chemical data for water in Lake Powell and from Glen Canyon Dam releases, Utah-Arizona, 1964-2012. Revised February 2015. Version 3.0. U.S. Geological Survey Data Series 471. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/ds/471/pdf/ds471.pdf

Vernieu, W. S., and C. R. Anderson. 2013. Water temperatures in select nearshore environments of the Colorado River in Grand Canyon, Arizona, during the low steady summer flow experiment of 2000. U.S. Geological Survey Open-File Report 2013-1066. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2013/1066/of2013-1066_text.pdf

Voichick, N., and Topping, D. J. 2014. Extending the turbidity record-making additional use of continuous data from turbidity, acoustic-Doppler, and laser diffraction instruments and suspended-sediment samples in the Colorado River in Grand Canyon. U.S. Geological Survey Scientific Investigations Report 2014-5097. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/sir/2014/5097/pdf/sir2014-5097.pdf http://dx.doi.org/10.3133/sir20145097

Voichick, N., and S. A. Wright. 2007. Water-temperature data for the Colorado River and tributaries between Glen Canyon Dam and Spencer Canyon, northern Arizona, 1988-2005. U.S. Geological Survey Data Survey Series 251. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/ds/2007/251/

Walters, C. 1997. Challenges in adaptive management of riparian and coastal ecosystems. Conservation Ecology 1(2):1.

Walters, C. 1986. Adaptive management of renewable resources. The Blackburn Press, Caldwell, New Jersey, USA.

Walters, C., J. Korman, L. E. Stevens, and B. Gold. 2000. Ecosystem modelling for evaluation of adaptive management policies in the Grand Canyon. Conservation Ecology 4(2):1-38.

Walters, C. J., and R. Hilborn. 1976. Adaptive control of fishing systems. Journal of Fisheries Research Board of Canada 33:145-159. http://dx.doi.org/10.1139/f76-017

Walters, C. J., B. T. van Poorten, and L. G. Coggins. 2012. Bioenergetics and population dynamics of flannelmouth sucker and bluehead sucker in Grand Canyon as evidenced by tag recapture observations. Transactions of the American Fisheries Society 141(1):158-173. http://dx.doi.org/10.1080/00028487.2012.654891

Werner, K., and K. Yeager. 2013. Challenges in forecasting the 2011 runoff season in the Colorado River basin. Journal of Hydrometeorology 14:1364-1371. http://dx.doi.org/10.1175/JHM-D-12-055.1

Wright, S. A., and P. E. Grams. 2010. Evaluation of water year 2011 Glen Canyon Dam flow release scenarios on downstream sand storage along the Colorado River in Arizona. U.S. Geological Survey Open-File Report 2010-1133. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/of/2010/1133/OFR_2010-1133.pdf

Wright, S. A., and T. A. Kennedy. 2011. Science-based strategies for future high-flow experiments at Glen Canyon Dam. Pages 127-147 in T. S. Melis, editor. Effects of three high-flow experiments on the Colorado River ecosystem downstream from Glen Canyon Dam, Arizona. U.S. Geological Survey Circular 1366. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1366/c1366.pdf

Wright, S. A., T. S. Melis, D. J. Topping, and D. M. Rubin. 2005. Influence of Glen Canyon Dam operations on downstream sand resources of the Colorado River in Grand Canyon. Pages 17-31 in S. P. Gloss, J. E. Lovich, and T. S. Melis, editors. U.S. Geological Survey Circular 1282. U.S. Geological Survey, U.S. Department of the Interior, Reston, Virginia, USA. [online] URL: http://pubs.usgs.gov/circ/1282/c1282.pdf

Wright, S. A., D. J. Topping, D. M. Rubin, and T. S. Melis. 2010. An approach for modelling sediment budgets in supply‐limited rivers. Water Resources Research 46:1-18. http://dx.doi.org/10.1029/2009WR008600

Yackulic, C. B., M. D. Yard, J. Korman, and D. R. Van Haverbeke. 2014. A quantitative life history of endangered humpback chub that spawn in the Little Colorado River: variation in movement, growth, and survival. Ecology and Evolution 4(7):1006-1018. http://dx.doi.org/10.1002/ece3.990

Yard, M. D., L. G. Coggins, C. V. Baxter, G. E. Bennett, and J. Korman. 2011. Trout piscivory in the Colorado River, Grand Canyon: effects of turbidity, temperature, and fish prey availability. Transactions of the American Fisheries Society 140(2):471-486. http://dx.doi.org/10.1080/00028487.2011.572011