Pathways from research to policy: Implications for researchers and policymakers (part 1 of 2)

Professor Cynthia Coburn, Northwestern University, Chicago

This is the first blog in a two-part series that summarises Cynthia’s keynote address to the New Zealand Association for Research in Education (NZARE) annual conference 2017. The second blog in the series can be found here.

---

There is a lot of interest right now in how policy makers in various fields use research in their decision making. Researchers wonder why some research ends up influencing policy making while other research does not. Policy makers and funders want their investment in research to be more influential. And, with the recent move toward evidence-based policy making in many countries, advocates argue that policy makers and educational leaders should be using the best information available to inform decisions, especially those that affect children and youth. 

The vast majority of writing on this topic is normative, as people argue about how policy makers should use research in their policy making (for example, see here and here). However, my colleagues and I are trying to understand the patterns by which personnel in public agencies actually use research. We are also interested in intentional efforts to promote research use.

We argue that it is only by understanding the patterns of decision making in public agencies—and when and how research enters into this process—that we can begin to understand the promise and possibility, and pitfalls and limitations of using research to inform decision making. Our goal is to leverage research to foster a better understanding of research use and more appropriate policy solutions.

Traditional assumptions about decision making in public agencies

Most efforts intended to foster research use are rooted in the following image of how decision making occurs (source):

Scholars call this the instrumental use of research. This model is widespread in a lot of writing about research use—especially the normative writing. However, empirical research on actual research use raises several questions about this model.

First, existing research suggests that there is rarely a shared understanding of the problem to solve. Given the complexity and multi-dimensionality of the social world, there are always multiple potential causes for any problem. Further, any way of understanding a problem inevitably involves highlighting some aspects of the situation, while de-emphasizing or ignoring others.

Second, decision making is not a single event. Rather, as Carol Weiss taught us, decisions accrete. They emerge through a series of conversations and actions where ways of thinking about problems emerge. Small steps add up to a policy decision, whether formal action is taken or not.

Third, decision making in school districts is focused on a wide range of issues. The current conversation about research use, at least in the US, seems to be focused on a narrow range of decisions: mainly decisions related to selecting or adopting curriculum, materials, and programs. Thus, there’s been an enormous effort to improve the quality of efficacy studies such as randomized controlled trials of various curricular materials and instructional programs. However, in our 2.5-year study of instructional decision making across three US school districts, we identified only one decision—one!—that involved the adoption of particular approaches. Instead, we found that district officials were involved in a much broader range of decision making, including:

• designing their own policies and programs,
• creating implementation plans for those initiatives,
• problem solving when issues emerged,
• writing standards and frameworks,
• developing professional development plans,
• deciding how to support low performing schools,
• deciding where to build new schools and where to close schools, and
• discussing whether and how to end programs or terminate providers.

This suggests that research on a much broader range of issues may be necessary to inform the kinds of decisions that practitioners need to make.

Finally, the diagram above—and the conversation more broadly about decision making—is missing a really crucial element: the organizational context within which decision making unfolds. The nature of decision making is profoundly shaped by organizational conditions, including time, money and staffing. Our research documented two consequences of these types of constraints:

• First, decision making became increasingly interrupted and drawn out. Conversations about a particular issue or concern would surface, be discussed, and then recede from view in the press of competing priorities, only to resurface later in the face of a new crisis or an impending deadline. Many issues remained chronically unresolved.
• Second, decision making became more conservative. The solutions that were chosen borrowed heavily from pre-existing district practices. In the crush of impending deadlines, district personnel tended to reach to the familiar as the basis for solutions.

Alternative ways of understanding decision making

Overall, the findings of our studies on how decisions are actually made within school districts challenge the traditional instrumental view of research. In particular, our findings challenge the assumption that the meaning of research is self-evident to practitioners, and the assumption that the main role that research plays is filling a gap in knowledge right at the moment of decision.

Our research and also that of other scholars has repeatedly shown that research findings do not speak for themselves. Research use is, at root, an interpretive process. When people engage with research findings, they must construct their understanding of what they mean. It is these understandings that guide decision making and action … not the research itself. Furthermore, available research does not always point directly to an appropriate solution in a given context. So, there is a space between a given finding and appropriate action—and interpretive processes play a central role as people draw on their existing beliefs, knowledge, and values to make meaning of research and draw implications for solutions.

Implications

Given the diverse nature of the decisions made in schools, there is a need to work to improve the quality of a wide range of research – not just research on the efficacy of particular programs or products. Educational decision makers need high-quality research that they can draw on to inform their decisions in a diverse array of areas.

There also needs to be greater attention to fostering better conditions for decision making in districts and other policy making settings. Today’s resource constraints mean that educational leaders are working in settings that are frankly not at all conducive to learning and decision making. In order to improve the quality of decision making, and the role of research therein, it is important to attend to the conditions for decision making in public agencies as well.

Given the issues outlined in this post related to traditional instrumental understandings of research use, we need to consider research use in different ways. The second blog post in this two-part series outlines some alternative understandings of research use.

---

Cynthia Coburn is Professor at the School of Education and Social Policy, Northwestern University, Chicago. Cynthia has two related lines of research: 1) the relationship between instructional policy and teachers’ classroom practices in urban schools; and 2) the role of research and other forms of information in policymaking about instruction. In 2011, she was awarded the Early Career Award from the American Educational Research Association in recognition of her contributions to the field of educational research in the first decade of her career. In 2015, she was elected Fellow of the American Educational Research Association, honoring “exceptional contributions to and excellence in educational research.”