Can sense-making tools inform adaptation policy? A practitioner’s perspective

Insight, part of a special feature on Making Sense of Climate Change, Orientations to Adaptation

INTRODUCTION

I explore from a government practitioner’s perspective whether sense-making studies can inform adaptation policy. As a Canadian climate adaptation policy analyst and the only nonacademic to participate in the research reported in this special issue, I and my policy unit are uniquely positioned to assess the relevance of the research to climate adaptation policy practice and to speculate on how other government policy analysts in the field might respond. However, what might seem at first sight to be a straightforward question promising an equally straightforward answer is not that simple and is limited to one analyst’s experience and perspective alone. The particular application of “SenseMaker” and of its findings as set out in the essays in this volume will no doubt provoke legitimate debate over its promise and weakness as a policy aid. I have anticipated and explored this matter, but any judgments made are not themselves conclusive on the ultimate utility of the tool.

The title of this piece also implicitly asks how receptive the government policy community might be to this kind of approach. How might other government climate analysts and advisers in Canada and abroad use SenseMaker to develop smart adaptation plans, policies, and programs? Would they value its worth? The answer requires some reckoning with the prevailing paradigms in the policy profession and how open and flexible they might be. At the moment, policy makers, a catchall term I use to describe all government analysts and advisers involved in climate adaptation policy, are themselves divided about how best to support adaptation to environmental change. Most are heavily preoccupied with traditional risk management and engineering approaches to climate preparedness, relying heavily on empirical data and rational policy thinking for strategy development (Dessai et al. 2009, Eakin et al. 2009). A small but growing minority are looking instead to psychology and sociology research for insights on how to enable the kinds of behavioral changes that will better protect communities from climate impacts and build their capacity to adapt. Policy makers who take a more nuanced social science approach to adaptation may be less married to hard scientific and expert-led research and practice and more open to new disciplines, methods, and tools. Clearly, the fate of SenseMaker will depend in part on how these two policy paradigms or archetypes within the policy community play out.

The Climate Change Directorate, my unit, within the provincial government of Nova Scotia in Canada is itself at a crossroads. As the government body responsible for coordinating and supporting adaptive action across the civil service and across local governments and businesses more broadly, the directorate must deliver research that not only informs decision makers but also inspires them to act. A lot of time is devoted to supporting climate impact and risk assessment planning in various sectors and departments, but these assessments have not on their own galvanized wide social change or led to new adaptive behaviors; policy change requires more than just evidence-based research to make that shift. Climate impact assessment research can also sometimes box decision makers into simplistic and maladaptive strategies if the scope of the study is overly narrow or ignores important cultural, psychological, and socioeconomic factors (Eakin et al. 2009, Kennedy et al. 2010). The directorate recognizes these limitations and the need to think outside the traditional policy and research box. There is a curiosity for what wider social science approaches in complexity thinking, in social psychology, might offer decision makers and policy makers that would help them “make sense” of issues that risk management studies cannot, such as how to engage their stakeholders in discussions about acceptable levels of risk or how to create institutions that work collaboratively across department silos on climate issues that affect them all.

This is why in 2011 the directorate participated in the research reported in this special issue and joined the Australian research team. We were interested in exploring the usefulness of complexity and social psychology tools in informing adaptation policy. We were interested in what they might reveal about the many different people with whom we worked and how different groups in Canada make sense of and act in the face of climate change. We were also interested in evaluating the effectiveness of this new online SenseMaker “survey” tool for its ability to collect and contrast opposing views and experiences of climate adaptation and to engage policy analysts and decision makers alike in the kind of collective inquiry that complexity approaches advocate.

I reflect on our experience and thinking on the value of this research in informing climate adaptation policy and practice. First, the policy challenge is outlined as it appears to policy makers, exploring why climate adaptation is such a “wicked” problem and what might be gained by applying complexity tools, such as SenseMaker. I then reflect on our role in the research, as well as its limitations and findings. I conclude with an evaluation of the usefulness of this work in informing policy and with recommendations for improvements for greater policy impact.

The policy challenge

Adaptation to climate change is highly complex, involving multiple players across temporal and spatial scales, facing different stressors, at different times, with unique vulnerabilities and capacities to adapt (Berkes et al. 2003, Adger et al. 2009, Smithers and Smit 2009). What enables some groups to thrive and others to decline is often not the result of a single factor but of multiple interacting factors (Folke et al. 2002, Diamond 2005). It is difficult to capture these dynamics a priori, let alone affect them through policy (Adger et al. 2007).

There are also many tools in the policy toolbox, such as regulations, market-based incentives, and behavior-change campaigns that can be used to steer behavior (Halpern et al. 2004). Identical policy instruments may have positive and negative applications and unintended as well as intended outcomes in different policy contexts (Fiorino 2006). Policy makers cannot, therefore, rely on off-the-shelf solutions or naively assume that by adopting so-called best practices they will evoke the positive responses experienced elsewhere. Achieving desired policy outcomes is particularly complex because of this context sensitivity. The factors that lead to success cannot always be predicted.

Adaptation policy is daunting too because of the diversity of views about adaptation held by different groups. These preferences often influence what policies get supported and later adopted. However, frequently what people think and say may not always align with how they act or behave, particularly if factors change or policies shift (Kahneman 2011). Policy makers cannot assume, for instance, that public support for proposed initiatives will continue once they are implemented or that stated preferences would dictate actual behavior on the ground. This exacerbates political uncertainty about how people will respond to different policy regimes and renders decision making difficult.

In this sense, climate adaptation fits many of the characteristic traits of what Rittel (1972) called “wicked problems.” The causes and the solutions to the problem are difficult if not impossible to define. There is no one actor responsible for the problem or the solution. Actors hold different views about what should be done, and unpredictable social behaviors determine to a great extent policy success (Rittel 1972, Conklin 2006, Australian Public Service Commission 2007, Head and Alford 2013). These and other characteristics of wicked problems help analysts to distinguish them from more benign challenges where the problem definition and solution is clear and where there are already best practices in place to guide decision making.

Standard rational policy planning is not designed to solve wicked problems (Rittel 1972). It is designed to solve “simple” or “complicated” problems where the causes and effects are known, linear, and empirically grounded (Kurtz and Snowden 2003). Managing wicked problems requires a different strategy, and this poses a great challenge to policy makers who tend to default into traditional policy thinking, even when it comes to complex issues (Conklin 2006, Australian Public Service Commission 2007, Head and Alford 2013). As Einstein once said, “We can’t solve problems by using the same kind of thinking we used when we created them” (Mielach 2012). However, changing how policy makers approach wicked problems is tough because it requires reconciling two very opposing modes of thinking (http://www.homerdixon.com/2010/05/05/complexity-science-and-public-policy).

The archetypal “traditional policy maker” is trained to distill from complex problems logical rational solutions, using evidence to support political decision making (Rittel 1972, Howlett and Ramesh 2009). These analysts rely on experts and independent advisers for objectivity and tend to place great faith in the scientific method (Rittel 1972, Kurtz and Snowden 2003, Howlett and Ramesh 2009, Head and Alford 2013). They are geared toward providing tangible, efficient, and one-time policy solutions.

The “complexity analyst” archetype, by contrast, captures and tackles problems in their inherent complexity. These analysts rely on the multiple perspectives of nonexperts for insights on how to navigate, not necessarily solve, wicked problems (Rittel 1972, Conklin 2006). People on the ground are thought to be more knowledgeable because they interact in the problem environment every day, as opposed to experts who may be removed from the environment and only able to understand it through abstraction. Through creative conversation and engagement with peers, complexity policy makers believe that people come to identify and negotiate ways of interacting that eventually lead to solutions that are more effective than ones that are imposed.

We use this dichotomy between traditional and complexity policy makers to help illustrate the tension between these two opposing approaches to policy making and how it influences policy-makers’ impressions of SenseMaker and the value of this kind of work. In reality, policy makers are neither one of these extremes and may exhibit any range of perspectives and behaviors along the continuum. Like people, they are not constrained to a single, consistent position. Their views can change as the policy context, goals, and problems shift. That said, policy makers, like anyone else, are still vulnerable to habitual modes of thinking, bias, and preference that can color the way they approach policy issues.

Both traditional and complexity approaches have merit provided they suit the problem at hand. If the policy problem is tangible and simple, then a traditional approach is the most efficient and effective (Rittel 1972). It is when traditional approaches get wrongly applied to complex problems that troubles arise (Snowden and Boone 2007). The criticisms of risk assessments in climate adaptation planning are a case in point. Risk assessments involve identifying impacts, ranking risks, and finding cost-effective measures to reduce those risks (Willows and Connell 2003, Carter et al. 2007, Burton et al. 2009, Smith and Petley 2009, Kennedy et al. 2010). Risk analysts believe that there are “known knowns,” or at least “known unknowns,” in the way that climate change will impact society and that these factors can be drawn out with the help of expert modeling or analysis (Snowden and Boone 2007, Smith and Petley 2009). Adaptation is treated as a “simple” or “complicated” domain issue, not a complex or wicked one (Snowden and Boone 2007). In reality, climate adaptation is often more complex. As argued previously, it requires addressing cause-and-effect relationships that are sometimes only retrospectively knowable, which makes identifying the problem and the solution difficult if not impossible. Applying linear rational policy approaches may not work in this case because there is no clear policy problem or solution to work toward. If we force a problem diagnosis where it is not clear, simply to adhere to the steps of our approach, we risk oversimplifying and misdiagnosing the problem (Rittel 1972, Eakin et al. 2009, Kennedy et al. 2010).

Risk management approaches also tend to ignore the psychology of different players, the connection between their cognitive frames, and their subsequent actions. There is little emphasis on complexity, psychology, and culture, despite the important role they play in enabling or hindering adaptation (Adger 2003, Adger et al. 2004; K. Brown, unpublished manuscript, http://www.youtube.com/watch?v=FdnJ52VkQLQ). If we reduce climate adaptation to the mere study of climate hazards, we may miss the important cultural and social change elements that adaptation requires (Eakin et al. 2009, Kennedy et al. 2010).

As complexity scholars argue, “Wise executives tailor their approach to fit the complexity of the circumstances they face” (Snowden and Boone 2007:69). The same logic can and should be applied to policy making. Clearly, the task goes beyond how governments deal with the unpredictable nature of climate hazards and the complex environment in which adaptation policy making takes place. Policy makers must also reconcile traditional and complexity approaches to policy making and be able to discriminate between different types of problems and the kind of analytical tools and thinking they require (Rittel 1972, Kurtz and Snowden 2003).

Why sense-making tools

Some complexity-oriented policy makers and scholars find standard analytical tools or at least the epistemology behind those tools, i.e., multiple-choice surveys and reductionist models, ineffective, at least on their own, at adequately representing social complexity, behavior, and preference (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). As one Nova Scotian government analyst stated:

Traditional tools tend to categorize and break things down into simple constructs and apply logic to what are sometimes illogical problems. Policymakers need tools that complement but also challenge existing models that have been relied on for years. The tools must honor the nature of the problems that governments are asked to solve. (A. Poirier, Halifax, Nova Scotia, 15 September 2011, personal communication)

SenseMaker is a type of survey tool that aims to capture and analyze complexity in a way that reductionist surveys supposedly cannot (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). It can demonstrate how different groups perceive and relate to one another and to their environment through the capture of “narrative fragments,” allegedly a richer way of eliciting experiences, perceptions, and knowledge than would otherwise be possible from generic question-and-answer surveys (Lynam and Fletcher 2015, M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). The narrator chooses a story relating to a particular topic and signifies to the analyst the importance or meaning behind it. This allegedly reduces misinterpretations, allows respondents to communicate more freely, and widens the scope of what can be observed to reveal the complexity (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication).

In this way, SenseMaker is more than just a data collection tool: it is a new complexity-oriented approach to policy. It may support policy makers to become familiar with, and to be informed by, the patterns of their own environment as expressed by the people in it. It may force policy makers to suspend judgment on what is the problem or the solution until after the experiences, perceptions, motivations, and behaviors of people are better understood (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). It asks policy makers to move beyond seeing people as simply supporters or dissenters of preconceived policy goals and instead as legitimate experts in their own right, with information to tell about the system and experiences within it. Policy makers must withhold judgment to see all perspectives as equally valid and as opportunities for learning, recognizing that no one perspective holds the “truth” (Snowden and Boone 2007). This approach promises to reveal social patterns of interaction and beliefs that policy makers would not otherwise see, including “outlier” patterns that may impede policy goals. It may also provide clues about what policies work for different people in different contexts, serving as a critical supplement to standard risk assessments in informing, in this case, adaptation policy.

Another sense-making tool used in the research in this special issue was word association analyses. Although not normally part of a SenseMaker survey design, its inclusion in this case allowed the research team to solicit additional useful information. The team was interested in what words respondents were most likely to associate with climate change with the aim of helping those in policy practice in Canada and Australia to get to know and understand people’s mental framing of climate change (Moloney et al. 2014). Such methods may reveal contradictions between people’s representations of climate change and their experiences of climate change. This could give policy makers the ability to identify and modify those patterns of thinking and behavior by actively working to influence the mental representations of those they wish to effect. For example, in better understanding their audiences, policy makers could, the research team argued, shape consultations and messaging more strategically for different audiences.

A challenge in this, and similar, research projects is whether these tools can live up to the traditional policy-makers’ standards of scientific rigor without compromising the complexity and broader patterns these tools try to illuminate. Traditional policy makers, even those sympathetic to complexity, still prefer or require “certainty” or, at the very least, scientifically defensible and logical grounds for a policy direction particularly where major resource investments are at stake. Policy makers are therefore more likely to rely on conventional tools because, unlike complexity methods, these tools emphasize reducing uncertainty and finding statistically supported relationships that will meet evidence-based decision-making demands. Word association analysis may be better able to meet these demands than would SenseMaker, as the findings are based on statistical significance. Although SenseMaker software is capable of doing some traditional statistical analysis, it is more so designed to detect and make sense of emerging “patterns” in the data set that may or may not be statistically relevant. The research in this special issue helped us to anticipate whether tools like SenseMaker could meet those demands and sway both traditional and complexity types to see the value in this kind of approach.

THE STUDY

As described by Lynam and Fletcher (2015), social psychology and climate adaptation scholars at the Commonwealth Scientific and Industrial Research Organization (CSIRO) in Australia initiated the research in this special issue. They partnered with the Australian-based company, Emerging Options, which had staff trained in complexity thinking and had the license to use the SenseMaker software. Together they designed and launched two separate online surveys in 2010 and 2011 targeting climate change and adaptation scholars at a conference in Australia in 2010 and an Australian state government department involved in environmental sustainability. Our Climate Change Directorate learned about the study in early 2011 and joined the research team just before the third and final survey launch. This allowed us to solicit Canadian responses to the third survey and to broaden the initial geographic scope of the pilot for greater cross-cultural study and comparison Lynam and Fletcher (2015), for a full methodological description).

Because we were late to join the study and merely piggybacking on the original process, we did not have the opportunity to devise the initial probing question in the SenseMaker survey. Not having a rigorous set of policy questions or problems in mind at the outset likely had an impact on how relevant the data will be to current policy issues. The survey question of what enables and constrains climate change, for example, was broad enough to be relevant but also rather vague. Early immersion in the design and planning would also have assisted the directorate in achieving a better assessment of the survey results and a more rigorous process to draw out practical policy implications and consequences. In the course of applying the survey, several other methodological and epistemological problems arose. The following is a synthesis of the most prominent challenges and surprises. I and the Climate Change Directorate critiqued the research and the SenseMaker tool in light of those challenges.

The challenge with data collection

Data collection proved more difficult with SenseMaker than for traditional analytical surveys, thus hindering researchers’ ability to get sufficient data. The Australian survey collector for the general public panel, for instance, noted higher dropout rates with SenseMaker than with other surveys that had been launched. This was in part attributed to the unusualness of the survey instrument. It could also be that SenseMaker requires more engagement on the part of respondents than a typical “question-answer” survey. Reflecting on Canadian responses to the survey, some respondents felt that the opening question and following prompts were confusing, and in frustration with the tool’s design, they dropped out. Whatever the reasons, these weaknesses posed a barrier, particularly for more traditional policy makers who understandably wanted sufficient data to give confidence to any conclusions drawn.

Assumptions about narratives

A repeated assertion made by SenseMaker proponents is, “Humans convey complex knowledge through narratives [and] that people are able to write or tell narratives easily” (K. S. Ghee, unpublished manuscript). The term “narrative” is often thought of in a strict or narrow sense requiring “at least three elements: an original state of affairs, an action or an event, and the consequent state of affairs” (Czarniawska-Joerges 1998:2). It is precisely SenseMaker’s ability to elicit and analyze such narrative fragments that differentiates it from traditional opinion surveys (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication).

For this reason, it was assumed that survey responses would be in narrative form. However, as was the case in a separate SenseMaker study in Singapore, “it was observed that most respondents more naturally shared opinions rather than narratives” (K. S. Ghee, unpublished manuscript). “Experiences provided mainly talk about climate change—what it is, [or] is not and, to a lesser extent, [a] focus on how to adapt” (Fletcher, unpublished manuscript). Respondents did not follow a plot or narrative, involving characters, place, or time markers describing their own adaptation experiences.

This could be because of the style of the opening survey question, which solicited a perspective rather than a narrative about climate change. It was designed this way because the research team worried that respondents might not have had direct experiences managing the impacts of climate change:

Imagine you are in a lift (elevator) with 2 people who are discussing how people and institutions are reacting to climate change. One person mentions that several obstacles constrain the extent to which people are able to prepare for impacts and/or adapt. The other person says that he knows of a few examples in which people and institutions are already responding. They turn to you and ask for your perspective on what makes preparation/reaction possible or difficult. How would you respond?

Respondents’ confusion with the survey may have in part been caused by the fact that the follow-up prompting questions did not seem to match the opening one. For instance, one of the generic prompts asks: “Is your response about you or someone you know?” Or, “Who are the main characters in your story?” This would be an irrelevant or confusing question for the respondents to answer if their initial response was an opinion about how to adapt rather than an experience of adapting. It could also be a problem with online surveys generally where respondents may not take the time to read the instructions that emphasize the importance of relaying experiences and stories around climate adaptation. It could also be that narratives are just more difficult to conjure up than SenseMaker proponents had anticipated. Those who think strict narrative form is critical to the success of SenseMaker may find the lack of narrative form a problem, even a contradiction, in the tool’s application. It may undermine the “richness of the data collected since opinions unlike narratives do not encapsulate real experiences which provide important contextual data” (K. S. Ghee, unpublished manuscript). Others, however, are less concerned about the structural form of the responses and more concerned with the message. Although the directorate was initially surprised at the content of the responses, given the repeated emphasis on narrative, we concluded that all the data could prove insightful regardless of the form.

SenseMaker seems equipped to study these patterns of opinion and has been useful for such analysis, although it is debatable whether it is superior to or even different from traditional surveys in this regard (K. S. Ghee, unpublished manuscript). This is why applying different analytical tools beyond those of SenseMaker, such as narrative or text analysis using QDAMinor, for instance, can be useful, because researchers and policy makers alike can begin to compare and assess how well different software tools interpret and draw insights from the data.

Involving policy makers in the analysis

One of the critical questions to be faced in this kind of undertaking is how best to involve policy makers in the SenseMaker process and investigation of the data. SenseMaker proponents assert that policy analysts in the field will detect patterns that other general analysts will not see and should therefore be involved in the analysis process (Fletcher, unpublished manuscript). Such involvement moves many policy makers out of the realm of practice and into the world of academia. Most nonresearch-based civil servants including many policy analysts tend to outsource this analysis, partly because of time constraints but also to get independent expert advice. They look at data and make decisions but do not often crunch the data or analyze the patterns themselves. A major culture shift is required especially for traditional, nonresearch-based policy makers to work with SenseMaker consultants and academics to draw out conclusions from the data together.

Our main finding from being involved in the data analysis phase was that there was little difference in the ability to detect patterns in the data between policy makers versus individuals with limited knowledge of adaptation. However, we found that policy makers were better able to hypothesize why those patterns might be forming because of prior knowledge of the policy environment.

Making use of the findings

The word association analysis was moderately helpful in giving policy makers insight into how different respondents contextualized climate change. The most frequently cited words in Canada, i.e., “weather,” “sea-level rise,” “melting,” “flooding,” “greenhouse gases,” and “oil,” seem to reflect the way that media and governments currently contextualize climate change in Canada and suggest that popular media is a key contributor to people’s mental framing of the issue (Lorenzoni et al. 2006, Lorenzoni and Hulme 2009; T. Lynam, 2011, personal communication). People also seem to resonate more with impacts they see in their own back yard, hence the reason why most respondents, being northern coastal inhabitants, chose “sea-level rise,” “melting,” and “flooding” as impacts. This prompted the Climate Change Directorate to think about the potential of using familiar images that resonate with people to steer communication strategies, but the data were still perceived to be too vague to inform a detailed strategy or propel a reluctant traditional policy maker to invest in such a strategy.

Traditional analysts are likely to have trouble finding clear-cut explanations for the mental representations that people hold. Moreover, traditional policy makers will likely want to know whether the prevailing mental anchoring will be good news or bad news for specific climate policies. What does the evidence suggest about people’s behavior in the face of climate change, and how are governments going to work with these groups? The depth of the data analysis is less useful, permitting researchers and policy makers only to speculate.

A traditional policy maker, for instance, might take a positive interpretation of the way Canadians associate climate change. The policy maker may think that Canadians are correctly associating climate change with “weather,” climate is after all a long-term weather trend; with causes, i.e., “greenhouse gases” and “oil”; and with the consequences of the problem, i.e., “sea-level rise,” “melting,” and “flooding.” The policy maker may think that because Canadians have this factual anchoring, they may be more rational and reasonable in their response to proposed adaptation policies.

A complexity policy maker, on the other hand, may find this mental anchoring too narrow and too simple in its sole focus on images that resonate, marginalizing and forgetting other important climate impacts that are not so prominent. The question then becomes whether communication strategies should build principally on the known target associations or purposely target other neglected issues to broaden people’s knowledge and support for other adaptation efforts. It is not immediately clear which policy response is best.

Such is the real world of sense making, which presents complexity but no clear direction on where policy makers should necessarily go. Traditional policy makers will likely retreat from this uncertainty, whereas complexity theorists may be more ready to try small and safe experiments to test policies for wicked problems in complicated times. However, even getting small experimental projects up and going is difficult when resources and capacity are strapped and the evidence to warrant experiments is slim.

Patterns in the narratives can also show how receptive or fearful respondents are to certain policy tools. Canadian respondents, for instance, saw regulations as more strongly helping adaptation than did Australians, who saw technology as more strongly associated with enabling adaptation than were regulations (T. Lynam, personal communication). Respondents from both countries saw money as the greatest hindrance to adaptation.

These can be useful observations for policy makers to understand their audience and the extent to which policy can address public feelings and concerns. However, the data likely do not provide sufficient evidence for a traditional policy maker to warrant further exploration unless there is a specific reason to do so. The data would not be a critical consideration in the day-to-day policy decisions because there are so many other subsidiary technical questions to be answered for the data to be useful, such as what kind of technology and why it is preferred, what kind of regulations, and so forth. The data allow you to probe a little further, but the interrogation appears endless.

DISCUSSION

The directorate’s expectations going into the study were that the application of the SenseMaker tool would enable us to assess public perceptions as well as the applicability of the SenseMaker tool. The reality was that the project was able to give us a flavor for the potential of SenseMaker and to reflect on our experiences in using it, but it did not enable us to draw definite conclusions about the quality of the tool and its impact on policy.

The findings from the study are a little underwhelming, in terms of their ability to have much policy impact. They do provide good initial direction to local climate adaptation policy but lack specifics. However, they have prompted the directorate to think about how multiple groups interpret and respond to climate change and to different adaptation tools. Even the more traditional policy makers in the group have started creative conversations with openness to strategies around these concepts, which was of course one of the intentions of the study. We just need now to provide a deeper analytical rigor to win more confidence from these and other traditional policy makers.

This said, whatever the limits of the tool or the study approach, the research that the directorate undertook has broken ground on several levels. It was one of only a few Canadian studies looking at public perceptions and experiences of climate adaptation, as opposed to climate mitigation, and the only one to use SenseMaker software for the analysis. It has forged new relationships between policy makers and researchers, which created new opportunities to learn how different disciplines would approach and go about analyzing and handling different climate change problems. There are doubtless further policy clues to be drawn from the SenseMaker data with further analysis.

Recommendations

Researchers need to acknowledge that a majority of policy makers in climate adaptation are not yet thinking about more nuanced social science research. If this is to change, aspects of these sense-making approaches must be more defensible from a rigorous “evidence-based” point of view and more directive in what policy makers must do. Problems with research design, data collection, and dropout rates must be tackled if research of this kind is to produce the volume and quality of data to ensure reliability and validity.

Researchers also need to be clear about what they mean by “narrative” and be careful not to expect that all responses to a SenseMaker survey will elicit strict narrative content. They must be prepared to use other analytical tools beyond SenseMaker to assess this content, especially given the diversity in the way responses are structured, because some tools may be better than others at assessing different aspects of the response. Finally, the initial probing question must be carefully designed to prompt exactly the kind of responses policy makers are interested in, so that there is some control over the kind of responses policy makers get.

Our experience suggested that policy makers might not need to be involved in all aspects of pattern detection or analysis. Researchers can still probe the data based on policy-maker reactions to it, so policy engagement is not compromised but is also not diverted to endless investigations. Nonresearch-based policy makers should be consulted on instrument questions and design at the outset and then left to be involved afterward in interactive analysis of data patterns with investigators. The goal should be to identify what factors are most engaging from the policy practitioners’ perspectives and how best to draw out the implications and consequences for both the policy process and public policy.

Conclusions

Many policy makers are looking for certainty, or at least to reduce uncertainty, despite the fact that social psychology and complexity approaches suggest that we cannot, at least currently, provide that certainty. Under such circumstances, we might follow Conklin’s (2006:10) advice: “You don’t so much ‘solve’ a wicked problem as you help stakeholders negotiate shared understanding and shared meaning about the problem and its possible solutions. The objective of the work is coherent action not final solution.”

This is wise advice for these kinds of problems, but the dilemma is that politicians want, and traditional policy analysts are trained to provide, a clear problem definition and decisive solution that they can work toward. SenseMaker and word association analysis gives analysts a rich data set in which to probe and sense for answers, but they do not always give policy makers definitive answers or certainty on the direction to be followed. Instead, the evidence raises yet more hypotheses and investigations. This can be an excellent starting point for creative policy discussions, but it is frustrating for policy makers or politicians, who want action, are short on time, and cannot get definitive expert guidance.

As an analytical tool, we consider SenseMaker strong in its ability to gather large suits of data at the collection stage, yet weak in the wide room left for bias and misinterpretation when trying to make sense of the findings. There is also a traditional policy tendency to want to make judgments about the data before fully absorbing the patterns themselves and their meanings. In this way, it is easy to see how a top-down traditional policy focus might end up reframing the narrative and hijacking its policy consequences.

In our view, the greatest value of SenseMaker is the epistemology and complexity theory behind it. It reminds policy makers that there is a role for government in helping to shape public discourse on climate change, to augment or dampen patterns of thinking around adaptation, and to play more of a hand in directing cultural adaptation for climate change. It is these gaps in current practice and policy that sense making can begin to fill. Overall, SenseMaker permits a more nuanced understanding of public knowledge and opinion, particularly that of specific target groups, and it prepares policy makers with a lens through which to engage in intensive analysis and consultations and frame climate policy.

LITERATURE CITED

Adger, N. W. 2003. Social capital, collective action, and adaptation to climate change. Economic Geography 79:387-404. http://dx.doi.org/10.1111/j.1944-8287.2003.tb00220.x

Adger, N. W., S. Agrawala, M. M. Q. Mirza, C. Conde, K. O’Brien, J. Pulhin, R. Pulwarty, B. Smit, and K. Takahashi. 2007. Assessment of adaptation practices, options, constraints and capacity. Pages 717-743 in M. L. Parry, O. F. Canziani, J. P. Palutikof, P. J. van der Linden, and C. E. Hanson, editors. Climate change 2007: impacts, adaptation and vulnerability: contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK.

Adger, N. W., N. Brooks, G. Bentham, M. Agnew, and S. Eriksen. 2004. New indicators of vulnerability and adaptive capacity. Technical Report 7, Tyndall Centre for Climate Change Research, East Anglia, UK.

Adger, N. W., I. Lorenzoni, and K. L. O’Brien. 2009. Adaptation now. Pages 1-22 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.

Australian Public Service Commission. 2007. Tackling wicked problems: a public policy perspective. Australian Government, Canberra, Australian Capital Territory, Australia.

Berkes, F., J. Colding, and C. Folke. 2003. Navigating social–ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK.

Burton, I., S. Huq, B. Lim, O. Pilifosova, and E. L. F. Schipper. 2009. From impacts assessment to adaptation priorities: the shaping of adaptation policy. Pages 377-393 in E. L. F. Schipper and I. Burton, editors. The Earthscan reader on adaptation to climate change. Routledge, London, UK.

Carter, T. R., R. N. Jones, X. Lu, S. Bhadwal, C. Conde, L. O. Mearns, B. C. O’Neill, M. D. A. Rounsevell, and M. B. Zurek. 2007. New assessment methods and the characterisation of future conditions. Pages 133-171 in M. L. Parry, O. F. Canziani, J. P. Palutikof, P. J. van der Linden, and C. E. Hanson, editors. Climate change 2007: impacts, adaptation and vulnerability: contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK.

Conklin, J. 2006. Wicked problems and social complexity. Pages 3-40 in J. Conklin. Dialogue mapping: building shared understanding of wicked problems. John Wiley and Sons, Chichester, UK.

Czarniawska-Joerges, B. 1998. A narrative approach to organization studies. Sage, London, UK.

Dessai, S., M. Hulme, R. Lempert, and R. Pielke. 2009. Climate prediction: a limit to adaptation? Pages 64-78 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.

Diamond, J. 2005. Collapse: how societies choose to fail or succeed. Viking Press, New York, New York, USA.

Eakin, H., E. L. Tompkins, D. R. Nelson, and J. M. Anderies. 2009. Hidden costs and disparate uncertainties: tradeoffs in approaches to climate policy. Pages 212-226 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.

Fiorino, D. J. 2006. The new environmental regulation. MIT Press, Cambridge, Massachusetts, USA.

Folke C., J. Colding, and F. Berkes. 2002. Synthesis: building resilience for adaptive capacity in social-ecological systems. Pages 352-387 in F. Berkes, J. Colding, and C. Folke, editors. Navigating social-ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511541957.020

Halpern, D., C. Bates, G. Mulgan, and S. Aldridge. 2004. Personal responsibility and changing behaviour: the state of knowledge and its implications for public policy. Cabinet Office, Prime Minister’s Strategy Unit, London, UK.

Head, B. W., and J. Alford. 2013. Wicked problems: the implications for public management. Administration & Society, in press. http://dx.doi.org/10.1177/0095399713481601

Howlett, M., and M. Ramesh. 2009. Approaches to public policy. Pages 18-47 in M. Howlett and M. Ramesh. Studying public policy: policy cycles and policy subsystems. Oxford University Press, Oxford, UK.

Kahneman, D. 2011. Thinking fast and slow. Doubleday Canada, Toronto, Ontario, Canada.

Kennedy, D., L. Stocker, and G. Burke. 2010. Australian local government action on climate change adaptation: some critical reflections to assist decision-making. Local Environment: The International Journal of Justice and Sustainability 15:805-816. http://dx.doi.org/10.1080/13549839.2010.514602

Kurtz, C. F., and D. J. Snowden. 2003. The new dynamics of strategy: sense-making in a complex and complicated world. IBM Systems Journal 42(3):462-483.

Lorenzoni, I., and M. Hulme. 2009. Believing is seeing: laypeople’s views of future socio-economic and climate change in England and in Italy. Public Understanding of Science 18:383-400. http://dx.doi.org/10.1177/0963662508089540

Lorenzoni, I., A. Leiserowitz, M. D. F. Doria, W. Poortinga, and N. F. Pidgeon. 2006. Cross-national comparisons of image associations with “global warming” and “climate change” among laypeople in the United States of America and Great Britain. Journal of Risk Research 9:265-281. http://dx.doi.org/10.1080/13669870600613658

Lynam, T., and C. Fletcher. 2015. Sensemaking: a complexity perspective. Ecology and Society 20(1): 65. http://dx.doi.org/10.5751/ES-07410-200165

Mielach, D. 2012. We can’t solve problems by using the same kind of thinking we used when we created them. Business Insider, 19 April. [online] URL: http://articles.businessinsider.com/2012-04-19/strategy/31366385_1_business-lessons-success-business

Moloney, G., Z. Leviston, T. Lynam, J. Price, S. Stone-Jovicich, and D. Blair. 2014. Using social representations theory to make sense of climate change: what scientists and nonscientists in Australia think. Ecology and Society 19(3): 19. http://dx.doi.org/10.5751/ES-06592-190319

Rittel, H. 1972. On the planning crisis: systems analysis of the ‘first and second generations.’ Bedriftskonomen 8:390-396.

Smith, K., and D. N. Petley. 2009. Environmental hazards: assessing risk and reducing disaster. Fifth edition. Routledge, New York, New York, USA.

Smithers, J., and B. Smit. 2009. Human adaptation to climate variability and change. Pages 15-33 in E. L. F. Schipper and I. Burton, editors. The Earthscan reader on adaptation to climate change. Routledge, London, UK.

Snowden, D. J., and M. E. Boone. 2007. A leader’s framework for decision making: wise executives tailor their approach to fit the complexity of the circumstances they face. Harvard Business Review 85:68-76.

Willows, R. I., and R. K. Connell, editors. 2003. Climate adaptation: risk, uncertainty and decision making. United Kingdom Climate Impacts Programme (UKCIP) Technical Report, UKCIP, Oxford, UK.