Effective democracy, human rights, and governance (DRG) programming to respond to disinformation requires practitioners to make accurate inferences about the underlying causes of information disorders and about the effects of their interventions. Programs to counter disinformation often rely on a research component to identify problems, to identify potential targets or beneficiaries of an intervention, to develop and adapt program content, to monitor implementation, and to evaluate results. This chapter will survey a broad menu of research tools and approaches for understanding disinformation and potential responses, with the goal of supporting DRG practitioners in designing, implementing, and evaluating programs based on the best available data and evidence.
The sections that follow distinguish broadly between research approaches or designs and data collection methods.
Highlight
For the purposes of this guide, a research approach or research design refers to a method or set of methods that allow researchers or practitioners to make valid inferences about disinformation or programmatic responses. In other words, a research design is a method through which one can confidently and accurately answer specific research questions. On the other hand, data collection describes the ways in which researchers and practitioners collect the information needed to answer those research questions. For example, key informant interviews (KIIs) or in-depth interviews (IDIs) are data collection methods that may be used within several research designs.
To support DRG practitioners in developing evidence-based programs to counter disinformation, this chapter is structured according to stages in the program cycle – design, implementation, and evaluation. It provides examples of research approaches that can help answer questions for specific decisions at each stage. As a final note, the examples provided are suggestive, not exhaustive. Useful and interesting research and data collection methods, especially on information and disinformation, require thought, planning, and creativity. To develop a research approach that is most useful for a program, consider consulting or partnering early with internal experts including applied researchers and evaluators or external experts through one of many academic institutions that specialize in research on democracy and governance interventions.
Research Networks
EGAP: Evidence in Governance and Politics (EGAP) is a research, evaluation, and learning network with worldwide reach that promotes rigorous knowledge accumulation, innovation, and evidence-based policy in various governance domains, including accountability, political participation, mitigation of societal conflict, and reducing inequality. It does so by fostering academic-practitioner collaborations, developing tools and methods for analytical rigor, and training academics and practitioners alike, with an intensive focus in the Global South. Results from research are shared with policy makers and development agencies through regular policy fora, thematic and plenary meetings, academic practitioner events, and policy briefs.
J-PAL: The Abdul Latif Jameel Poverty Action Lab (J-PAL) is a global research center working to reduce poverty by ensuring that policy is informed by scientific evidence. Anchored by a network of 227 affiliated professors at universities around the world, J-PAL conducts randomized impact evaluations to answer critical questions in the fight against poverty. J-PAL translates research into action, promoting a culture of evidence-informed policymaking around the world. Their policy analysis and outreach help governments, NGOs, donors, and the private sector apply evidence from randomized evaluations to their work and contributes to public discourse around some of the most pressing questions in social policy and international development.
IPA: Innovations for Poverty Action (IPA) is a research and policy nonprofit that discovers and promotes effective solutions to global poverty problems. IPA brings together researchers and decision-makers to design, rigorously evaluate, and refine these solutions and their applications, ensuring that the evidence created is used to improve the lives of the world’s poor.
Political Violence FieldLab: The Political Violence FieldLab provides a home for basic and applied research on the causes and effects of political violence. The FieldLab provides students the opportunity to work on cutting-edge and policy-relevant questions in the study of political violence. Their projects involve close collaboration with government agencies and non-government organizations to evaluate the effects and effectiveness of interventions in contemporary conflict settings.
MIT GovLab: GovLab collaborates with civil society, funders, and governments on research that builds and tests theories about how innovative programs and interventions affect political behavior and make governments more accountable to citizens. They develop and test hypotheses about accountability and citizen engagement that contribute to theoretical knowledge and help practitioners learn in real time. Through integrated and sustained collaborations, GovLab works together with practitioners at every stage of the research, from theory building to theory testing.
DevLab@Duke: The DevLab@Duke is an applied learning environment that focuses on connecting social scientists at Duke who work in international development with the community of development practitioners to create rigorous programming, collect monitoring and evaluation data, and conduct impact evaluations of development projects. In addressing these goals, they bring together scholars and students attuned to the research frontier and with advanced capabilities in experimental and quasi-experimental impact evaluation designs, survey design and other data collection tools, and data analytics, including impact evaluation econometrics, web scraping and geospatial analysis.
Center for Effective Global Action (CEGA): CEGA is a hub for research on global development. Headquartered at the University of California, Berkeley, their large, interdisciplinary network–including a growing number of scholars from low and middle-income countries–identifies and tests innovations designed to reduce poverty and promote development. CEGA researchers use rigorous evaluations, tools from data science, and new measurement technologies to assess the impacts of large-scale social and economic development programs.
Citizens and Technology Lab: Citizens and Technology Lab does citizen science for the internet. They seek to enable anyone to engage critically with the tech tools and platforms they use, ask questions, and get answers. Working hand-in-hand with diverse communities and organizations around the world, they identify issues of shared concern (“effects”) related to digital discourse, digital rights and consumer protection. Their research methods can discover if a proposed effect is really happening, uncover the causes behind a systemic issue, and test ideas for creating change.
Stanford Internet Observatory: The Stanford Internet Observatory is a cross-disciplinary program of research, teaching, and policy engagement for the study of abuse in current information technologies, with a focus on social media. The Observatory was created to learn about the abuse of the internet in real time, to develop a novel curriculum on trust and safety that is a first in computer science, and to translate research discoveries into training and policy innovations for the public good.
Goals of Research
Description, Explanation, or Prediction? Applied research in the DRG program cycle can support programs by fulfilling one or more of the following scientific goals.
Description: Descriptive research aims to identify characteristics of research subjects at different levels of analysis (e.g., individual, group, organization, country, etc.). Descriptive research classifies or categorizes subjects or identifies general patterns or relationships. Examples of descriptive research in countering disinformation programs might include developing descriptive statistics in polling or survey data to identify key target groups, or analysis to identify key themes in media content.
Explanation: Explanatory research aims to identify cause and effect relationships; it helps answer “why?” questions. It establishes causation through sequencing (as causes must precede their effects) and/or eliminating competing explanations through comparisons. This category may also include evaluation research in the program cycle, to the extent an evaluation attempts to determine the “impact” of a program on an outcome of interest (i.e., whether a program causes a result), or to determine which of several potential program approaches is most effective.
Highlight
Predictive Research in DRG Programming
Several USAID-funded initiatives use predictive research to help DRG practitioners better anticipate and respond to changes in political context. For example, the CEPPS Democratic Space Barometer forecasts democratic opening and closing over a two -year window. The Internews-led INSPIRES Consortium uses media scraping and machine learning to forecast closing civic space on a monthly basis.
Prediction: Predictive research uses descriptive or explanatory methods to forecast what might happen in the future. At a basic level, predictive research in the DRG program cycle might involve using findings from a program evaluation to adapt approaches to the next cycle or to another context. More systematic predictive research uses qualitative or quantitative methods to assign specific probabilities to events over a designated time, as in a weather forecast.
Data sources and collection methods for Disinformation Research include Key Informant Interviews (KII), Focus Groups, Public Opinion Polls, Surveys, Audience Metrics (analog and digital), Web and Social Media Scraping, Administrative Data analysis (data collected and stored as part of the operations of organizations like governments, nonprofits, or firms). There are other methods but these are some key ones that will be explored further in this text.