Pathways from research to policy: Implications for researchers and policymakers (part 2 of 2)

Professor Cynthia Coburn, Northwestern University, Chicago

This is the second blog in a two-part series that summarises Cynthia’s keynote address to the New Zealand Association for Research in Education (NZARE) annual conference 2017. The first blog in the series can be found here.

---

The first blog post in this two-part series discussed the conventional image of the role of research in practitioners’ decision making. This traditional image focuses on district leaders consulting research to provide missing knowledge at the main moment of decision. There are certainly times when research is used in that way, but there are also other ways in which research is used. These alternative understandings of research use are the focus of this post.

Symbolic use of research

Symbolic use is when research is used to justify or legitimize decisions that have already been made. Although not ideal, this approach has been widely documented—we need to be aware that this is a way in which research is frequently used.

For example, in one of our studies, district leaders were redesigning their course sequence in response to the Common Core State Standards. The district leaders decided to take the opportunity to do away with the accelerated track in mathematics. This decision was sure to be controversial, and the district leaders described using research to support their case for the new plan:

“It [research] just gave us more evidence if we needed it to convince people … I found a little blurb that Maika Watanabe wrote in her book, and then I started looking at other places, and a lot of people point to the same research, which was this district in New York … We had Claude Steele’s argument around stereotype threat and how one of the things he talks about is how ability grouping, tracking, plays into stereotype threat … And Jo Boaler’s talks and research around all the causes of inequity and ability grouping.”

These district leaders were clearly using research to justify a decision that is already made, making this a symbolic use of research. Using research in this way also made them very effective advocates for the policy with teachers, parents, and others in the district, support that was crucial for implementation of the controversial policy.

Conceptual use of research

Conceptual use is when research provides decision makers with new ideas, concepts, or generalizations that influence how they view the nature of a problem or what they see as appropriate kinds of solutions. (Carol Weiss calls this the ‘enlightenment function‘ of research.) In this scenario, the research does not feed into a decision directly. Rather, it shapes the beliefs and understandings that decision makers draw on as they consider issues and evaluate potential solutions.

Conceptual use of research can have long-term consequences. Rather than influencing a single decision, conceptual use of research shapes how people see the world, how they respond to problems they encounter in their everyday work, and how they design and manage solutions. It touches not only policymaking but also policy implementation.

Conceptual use of research is not well understood, but my research suggests that it’s very important. My colleague Caitlin Farrell and I have identified four kinds of conceptual use of research:

First, engagement with research can introduce new concepts. For example, one district leader reported:

“[Researcher] makes me think conceptually. He wrote an article, which is what got me into multiplication as scaling … I’ve been thinking about what that means, how it could happen, how you could do that. That’s a huge mind-bender for me.”

Reading a research article caused this district leader to think differently about mathematics teaching and to incorporate these new ideas into district-level curriculum and professional development.

Second, engagement with research can enable people to see a problem they didn’t see before, or to see a problem in a new light. For example, I studied a school district that worked with an external researcher over five years in the area of teacher professional development (PD). As the district leaders engaged with research, they began to see their then-current approach to PD as problematic. Ultimately, they began to reframe their conceptions of what constituted high-quality PD. Engagement with research and the researcher influenced the district leaders to see a problem with their current way of working that they hadn’t seen before.

Third, engagement with research can help broaden or narrow understandings about the kinds of solutions that should be considered and are most appropriate to pursue. Returning to the example of high-quality PD, engagement with research shifted what district leaders saw as appropriate ways to address limitations in teachers’ pedagogy. They began to offer different types of PD offerings and changed school schedules to accommodate ongoing, situated training.

Finally, research use can provide a framework to guide action. For example, leaders in one school district read this book together. The book presents a framework of five essential supports for school reform. The district built this framework into their school improvement planning documents, used it to direct resources, and reported that it guided what they paid attention to when they visited schools. The research did not provide a program or a step-by-step guide for the district to adopt. Rather, it provided a framework to guide their strategic thinking.

Research habits of inquiry

We have also recently identified an additional way in which research can play a role in decision making—one that we don’t see discussed in the existing literature. We call this the development of research habits of inquiry. After several years of engagement with researchers, several of the districts we worked with started investigating decisions and challenges in ways that are more typical of the research world than the practitioner world. We have observed them routinely questioning the empirical evidence for each others’ claims, or discussing strategies that involve standard research techniques when they need to investigate issues. Extended work with researchers seemed to lead to an appropriation of some of the concerns, terms, and habits of inquiry of researchers.

The role of researchers

In traditional models of research use, the role of researchers is to provide access to research findings. However, we believe that researchers also have other roles to play.

First, direct interaction between researchers and district decision makers fosters greater access to and use of research. In our studies, research was much more likely to be invoked by district leaders—and used in substantive ways—when there was a researcher involved in the decision processes. As well as facilitating access to research, the direct involvement of a researcher made a difference to the way that research was used in the districts we studied.

Second, researchers can facilitate sensemaking about research findings (for example, see here and here). In school districts that were involved in intentional partnerships with researchers, we found that researchers did not simply hand over or provide a presentation on their research findings. They facilitated highly structured meetings with district leaders to assist them in making meaning of the findings. They asked questions like: What does this mean for teachers? What does this mean for school leaders? What does this mean for supervisors of school leaders? What does this mean for upper-level district leadership? This supported sensemaking appeared to contribute to deeper and more systemic efforts to address issues within the district.

Third, researchers can bring research concepts into co-design processes as they partner with schools. These co-design processes can then result in conceptual use of research as learnings from the particular co-design project are carried forward into future practice and decision making.

Finally, researchers can help interweave findings into ongoing district deliberations as they unfold. In our studies, we saw researchers helping district leaders to link research findings to policies, programs, and issues. This linking often happened either in one-on-one conversations about the problems that districts were facing or when the researcher sat in on district discussions. The researchers acted as thought partners and looked for strategic ways to bring research into the conversation and link it to the rhythms of the decision process itself.

Overall, our research has provided evidence that the above cluster of activities has been tremendously influential in fostering research use.

Implications

One of the key lessons that organizational design professionals have learned is that it is only by understanding the actual rhythms of people’s work—the nature of the challenges they face and how their work is organized—that we gain insight into windows of opportunity to foster change. One of the key lessons I’ve learned from studying school districts is that school district leaders want to make the best decisions possible. And they want the best information possible to make those decisions. But they are making decisions about very complex issues, and they are doing so in settings that are unstable, uncertain, and where there are many conflicting pressures and priorities. The questions at hand are: How can we provide the best information possible about the issues that district leaders are facing? How can we help create conditions that enable district leaders to use this information productively?

First, our research suggests that policy makers and researchers need to go beyond an emphasis on fostering access to research. In the U.S. context, most efforts to encourage greater research use have involved creating access to research – but access is a necessary but not sufficient condition for research use.

Second, to address research use we need to cultivate new roles for researchers. The new roles that my colleagues and I have identified mainly involve researchers directly interacting with district leaders—participating in the social process of meaning making about research findings; coming into decision process at multiple stages; and finding ways to link research to district concerns and issues. These new ways of working also involve creating new spaces and new routines (like the facilitated sensemaking or the co-design processes) and engaging district decision makers in these.  Finally, they involve asking questions and modeling habits of critically evaluating one’s own assumptions and the ways in which inferences are drawn from data.

Finally, there is a need to create conditions that enable decision makers to engage with new ideas and each other in ways that cause them to question and rethink their assumptions about the nature of the problem; and to open up to new ideas rather than discount them. In our research, this appeared to happen when deliberation involved:

• individuals with diverse points of view,
• enough safety for individuals to be able to voice their point of views,
• enough time and staffing to reach out and investigate new ideas and spend time discussing them with one another, and
• time to engage with research outside of a high stakes decision context.

I realize that this is a tall order. Not all of these ideas will be realistic all of the time or in all contexts, but my colleagues and I hope that our work will lead to more informed discussion of ways to help improve decision making in public education.

---

Cynthia Coburn is Professor at the School of Education and Social Policy, Northwestern University, Chicago. Cynthia has two related lines of research: 1) the relationship between instructional policy and teachers’ classroom practices in urban schools; and 2) the role of research and other forms of information in policymaking about instruction. In 2011, she was awarded the Early Career Award from the American Educational Research Association in recognition of her contributions to the field of educational research in the first decade of her career. In 2015, she was elected Fellow of the American Educational Research Association, honoring “exceptional contributions to and excellence in educational research.”