By Noah Weeth Feinstein, Rainer Bromme, Sarit Barzilai, and Ayelet Baram-Tsabari
Although there have been numerous attempts to categorize public engagement activities, we are not aware of a published account that focuses specifically on the different ways in which success in public engagement with science can be conceptualized, measured, or evaluated. In this blog post, we present a simple framework for classifying various dimensions of success, as well as related goals for public engagement activities. Our analysis is intended to be descriptive rather than normative: we are not arguing that successful public engagement should be measured in all of these different ways, merely that it might be measured thus. Readers will not find any measurable criteria for success below. On the other hand, our analysis may help them understand their own goals and criteria for success, and the ways in which their goals differ from others. In other words, clarifying the different dimensions of success can help readers reflect on the normative assumptions that underlie their current definitions of success, ultimately enabling more thoughtful research as well as more precisely targeted interventions.
Our framework consists of three dimensions :
1. Is success evaluated internally, by the people engaging with science, and/or externally, by observers or sponsors?
Relatively little research considers whether people engaging with science consider their own efforts successful or are satisfied with the outcomes of their engagement. Yet it is their evaluation of their own success, perhaps more than anything else, which determines whether they will continue engaging with science, whether their engagement with science will deepen or come to an abrupt end, and if/how they will engage with science in the future. We expect that individuals and groups have a wide range of criteria for evaluating their own success, from the detailed and explicit to the intuitive and implicit. These criteria may overlap with the criteria adopted by scientific communities or differ from them. For this dimension of our analysis, the key question is not how people evaluate their success, but whether and to what degree their own evaluation (vs. an observer’s or researcher’s evaluation) is identified and accepted as meaningful.
The processes and outcomes of public engagement with science are conceptualized in many different ways. For this dimension, the key question is not which process or outcome is chosen as a hallmark of success, but the degree to which the evaluation of success focuses on process and/or outcomes. People who place greater value on the process may judge a particular instance of public engagement to be good or successful regardless of the particular conclusions that participants form. People who place greater value on results may judge a particular instance of public engagement to be good or successful regardless of the reasoning or social processes that led to these results.
By characterizing this dimension as “epistemic and/or non-epistemic,” we have revealed something about our own inclinations and disciplinary biases. A behavioral theorist or health educator would probably put knowledge in the background and frame this dimension as “behavioral and/or non-behavioral.” This dimension, perhaps more than the others, could be characterized in many ways. We believe that our focus on knowledge is in keeping with the scholarly emphasis of many fields that study public engagement with science (as well as many of the agencies that promote it); an argument could be made that advancing knowledge has special importance in public engagement with science, compared to other spheres of public engagement (e.g., public engagement with art). That argument is beyond the scope of this blog post, however.
At its most simplistic level, this framework can be thought of as a 2x2x2 matrix that may help sort out the explicit as well as the implicit aims for public engagement with science. It could be applied to evaluate programs, plan new public engagement activities, or study the aims and goals of actors engaged in these types of activities. The three dimensions of the framework might also be useful for identifying aspects of success that have been ignored or overlooked in studies of people’s engagement with science. Yet the framework and its dimensions should be understood as analytic simplification with characteristic weaknesses. Other dimensions, or other ways of conceptualizing these dimensions, are easily imagined. There is vast heterogeneity within each dimension. Perhaps most importantly, we have outlined each dimension in terms of a contrast – between processes and outcomes, for example – but in no case are the two “ends” of a dimension mutually exclusive. Indeed, we believe that the richest and most fruitful evaluations of success in public engagement with science will mingle process and outcome, internal and external judgments, epistemic and non-epistemic outcomes.
Nevertheless, we offer this framework as a useful starting point, a tool that we believe will be valuable for revealing unspoken/ hidden aims of public engagement with science, for researching its live enactments, and for critiquing programs and policy documents. We hope that others will test, refine, dispute, and improve it further in the years ahead.
This blog post is a product of PESO2017 – Public Engagement with Science Online, an interdisciplinary, international research workshop of the Israel Science Foundation, exploring interactions between sciences, publics and social media. It took place at the Technion – Israel Institute of Technology June 25th to June 28th, 2017.
2. Is success conceptualized in terms of the quality of the engagement process and/or by the outcome or result of this process?
Some research that addresses public engagement with science emphasizes the quality or integrity of the engagement process, whether that process is understood as individual (e.g. evidence-informed decision-making) or social (e.g., seeking out particular sorts of expertise and assistance). Other research emphasizes the outcome: Does a person or group of people know more about a scientific topic, or accept a particular scientific consensus? Do they adopt or avoid a particular health-related behavior? Do they better understand a science-related issue, or have a more positive attitude toward scientists and scientific research? Although each of these outcomes can certainly be interpreted as part of a larger process, they are often interpreted to be the results of engagement.The processes and outcomes of public engagement with science are conceptualized in many different ways. For this dimension, the key question is not which process or outcome is chosen as a hallmark of success, but the degree to which the evaluation of success focuses on process and/or outcomes. People who place greater value on the process may judge a particular instance of public engagement to be good or successful regardless of the particular conclusions that participants form. People who place greater value on results may judge a particular instance of public engagement to be good or successful regardless of the reasoning or social processes that led to these results.
3. Is success evaluated in epistemic terms, related to the quality of knowledge and the processes that produce it, and/or in non-epistemic terms, related to things other than knowledge (e.g., advancing desirable behaviors, attitudes, or emotional states)?
Because science is often conceptualized in terms of advancing knowledge and knowing, public engagement with science is frequently evaluated with regard to quality of knowledge production, acquisition, and communication. Such evaluations focus on how people who engage with science attempt to know, what they learn, how they interact with scientific knowledge (whether they are contributing to it or questioning it), and how they understand the nature of scientific knowledge, as well as how the sponsors of public engagement attempt to communicate knowledge or engage people in scientific ways of knowing. These epistemic processes and outcomes differ from other non-epistemic ways one might characterize the quality of processes and outcomes of public engagement with science. In health and environmental contexts, for example, the value of knowledge is often eclipsed by actions. For example, the success of particular interventions might be measured in terms of increases in recycling or declines in drug use. Finally, epistemic and non-epistemic purposes can interact, both in the work of scientists and in lay engagement with science. For example, parents may seek valid knowledge about the effects of WiFi radiation in schools because they are concerned about their children’s health. Hence, public engagement with science may be evaluated based on both epistemic and non-epistemic criteria.By characterizing this dimension as “epistemic and/or non-epistemic,” we have revealed something about our own inclinations and disciplinary biases. A behavioral theorist or health educator would probably put knowledge in the background and frame this dimension as “behavioral and/or non-behavioral.” This dimension, perhaps more than the others, could be characterized in many ways. We believe that our focus on knowledge is in keeping with the scholarly emphasis of many fields that study public engagement with science (as well as many of the agencies that promote it); an argument could be made that advancing knowledge has special importance in public engagement with science, compared to other spheres of public engagement (e.g., public engagement with art). That argument is beyond the scope of this blog post, however.
At its most simplistic level, this framework can be thought of as a 2x2x2 matrix that may help sort out the explicit as well as the implicit aims for public engagement with science. It could be applied to evaluate programs, plan new public engagement activities, or study the aims and goals of actors engaged in these types of activities. The three dimensions of the framework might also be useful for identifying aspects of success that have been ignored or overlooked in studies of people’s engagement with science. Yet the framework and its dimensions should be understood as analytic simplification with characteristic weaknesses. Other dimensions, or other ways of conceptualizing these dimensions, are easily imagined. There is vast heterogeneity within each dimension. Perhaps most importantly, we have outlined each dimension in terms of a contrast – between processes and outcomes, for example – but in no case are the two “ends” of a dimension mutually exclusive. Indeed, we believe that the richest and most fruitful evaluations of success in public engagement with science will mingle process and outcome, internal and external judgments, epistemic and non-epistemic outcomes.
Nevertheless, we offer this framework as a useful starting point, a tool that we believe will be valuable for revealing unspoken/ hidden aims of public engagement with science, for researching its live enactments, and for critiquing programs and policy documents. We hope that others will test, refine, dispute, and improve it further in the years ahead.
This blog post is a product of PESO2017 – Public Engagement with Science Online, an interdisciplinary, international research workshop of the Israel Science Foundation, exploring interactions between sciences, publics and social media. It took place at the Technion – Israel Institute of Technology June 25th to June 28th, 2017.
Noah Weeth Feinstein is Associate Professor in the departments of Curriculum & Instruction and Community & Environmental Sociology at the University of Wisconsin- Madison. He studies how people understand and use science as they confront health and sustainability challenges in their personal, social, and political lives. His current projects focus on climate change adaptation and the capacity of science museums to enrich democratic societies by enhancing engagement with science. |
Rainer Bromme is Professor for Educational Psychology at the University of Munster. His research interests include: Communication and trust between Science and the Public, the psychology of interdisciplinary work and the development of knowledge and understanding in Science and Mathematics. |
Sarit Barzilai is a lecturer (assistant professor) at the Department of Learning, Instruction and Teacher Education at the University of Haifa’s Faculty of Education. Her primary research interests focus on learners’ epistemic thinking (i.e. thinking about knowledge and knowing) and digital literacy. She is particularly interested in studying learners’ epistemic thinking in digital media contexts such as learning from multiple online information sources and from digital games. |
Ayelet Baram-Tsabari is an Associate Professor at the Faculty of Education in Science and Technology at the Technion – Israel Institute of Technology, and head of the Science Communication research group. Her research interests include: Bridging science education and science communication scholarship; Identifying people’s interests in science; Building on these authentic interests to teach and communicate science in more meaningful and personally relevant ways. |
Comments
Post a Comment