Knowledge Systems: Evidence to Policy Concepts in Practice

In today’s world, the issues governments grapple with have become increasingly complex. Developing and articulating effective, multifaceted solutions require production and application of knowledge to support evidence-informed policies for socioeconomic development especially given the overwhelming deluge of data and information produced globally. However, connecting researchers to policy makers has proven to be a challenge. Researchers may not frame their inquiry in ways that meet policy makers’ expectations or needs, and policy makers may not communicate clearly to researchers what they want or need. This mismatch has led some to describe their relationship as “an unrequited love affair” (Clancy et al., 2012).

What makes for a happier match, and how to create or facilitate a mutually beneficial process that leads to the effective incorporation of knowledge into policy? Answers to these questions have been posed from a variety of viewpoints; among the more recent ones is what can be termed “knowledge systems.”

This policy brief reviews the evolution of knowledge-to-policy studies and the emergence of systems perspectives. Theoretical and analytic assessments of knowledge systems have identified their elements and interconnections, yet less well understood or documented has been how to grow and reinforce knowledge systems in settings where they are weak and underdeveloped. We offer a knowledge systems model that encapsulates current thinking and present an example of an effort to strengthen a knowledge system, drawn from a project managed by RTI in Indonesia. We conclude with some recommendations for knowledge systems strengthening.

Definition and Components of Knowledge Systems

A knowledge system, sometimes referred to as a knowledge sector, is a holistic conceptualization that specifies, for a given country, a set of knowledge institutions and actors and delineates the interconnections among them (Australia-Indonesia Partnership for Pro-Poor Policy, 2012; UK Department for International Development (DfID), 2017). The main components of a national knowledge system include the following, which are noted again throughout this brief:

• The supply of evidence or researchers (knowledge producers) consisting of universities, research centers, and think tanks (Australia-Indonesia Partnership for Pro-Poor Policy, 2012; Nugroho et al., 2018) ^[1]

• The demand for evidence by policy makers (knowledge users) consisting of government ministries or line agencies, and parliamentarians

• The debate about evidence through public discourse around policy issues consisting of civil society organizations, the private sector, and the media (knowledge intermediaries)

• The funding and regulations around generation of evidence consisting of regulatory authorities and public and private funding bodies (knowledge enablers)

The interactions and linkages among these components and the quality of their exchanges influence the extent to which knowledge can effectively inform policy making. These dynamic interchanges over time will have an impact on the functioning of the knowledge system. Positive interchanges can increase the likelihood of sustained uptake of research by policy makers; negative interchanges can lead to the sad “love affair” referred to above. This model is illustrated in Figure 1.

Although seemingly simple in its presentation, the knowledge system model presented in the figure is actually a model of contestation in which the different actors in the knowledge system debate policy informed by evidence, taking into consideration values, traditions, and political calculations (Eyben, 2013). By supporting the interaction between actors, contestation of evidence is increased, leading to more robust thinking and decision making.

Figure 1.Knowledge system model

Source: Developed by authors, building on UK Department for International Development (DFID) (2017) and Australia-Indonesia Partnership for Pro-Poor Policy (2012).

Knowledge Sector Frameworks in Historic Perspective

Concern with influencing policy makers through research has a long history. Numerous studies have examined how and why policy makers use research and have explored the dynamics of the application of policy analytic tools to socioeconomic problems facing politicians (Weiss, 1997; Wildavsky, 1979). This investigative stream has continued up to the present in the form of arguments for, and analysis of, evidence-based policymaking, bringing together researchers, practitioners, and policy makers (see, for example, the Africa Evidence Network, https://www.africaevidencenetwork.org/).

Building on the findings of early research utilization studies, later investigations sought to identify the factors that affect whether policy makers are likely to refer to research when making and implementing policy choices. Various knowledge-to-policy studies converged in identifying the following factors as central to research uptake: (1) political context (participatory structures and processes, power distributions, openness to change); (2) actors’ values, beliefs, and interests (willingness to consider policy alternatives, ideological orientations); (3) types of knowledge used in policy discussions (expert vs. citizen knowledge, research-based vs. practical); and (4) structures and processes shaping knowledge-policy interactions (linkage and intermediary functions) (Jones et al., 2013). These studies also identified factors associated with researchers that affect research uptake, such as incentives for researchers to pursue studies leading to academic and professional advancement rather than practical application, concerns with methodological debates that overshadow attention to policy relevance and practice, and the weak ability of researchers to communicate effectively with non-specialist policy makers. The findings from these studies exposed the complexities inherent in the production and utilization of policy-relevant knowledge and illuminated issues related to demand, supply, capacity, and intermediation to effectively link supply and demand (Carden, 2009).

Efforts to apply knowledge-to-policy models in practice, coupled with the adoption of systems thinking, led to conceptions of knowledge generation and utilization as a distinct sector and as an ecosystem (Valkokari, 2015). A key concept in systems thinking is that system outcomes emerge from interactions among the four factors listed above and can only be imperfectly identified or predicted in advance. This places a premium on tools such as network analysis to complement stakeholder analysis and political economy analysis. It also suggests that detailed planning and design are unlikely to yield anticipated results. Rather than the traditional project management model of predict-implement-control (Robertson, 2015), knowledge-to-policy models apply adaptive management principles that emphasize iterative trial-and-error approaches (Brinkerhoff et al., 2018).

Integrating a systems perspective leads practitioners to focus not simply on the capacities, interests, and roles of the actors in the knowledge system but on the linkages and interactions among them. Evidence for the value and contribution of knowledge to socioeconomic progress and effective policy making has grown (see, for example, Kaur & Singh, 2016). The Indonesia case that we explore in this brief builds on the lessons of research and practice briefly summarized here and illustrates the challenges in capitalizing on the potential of knowledge to effectively inform policy.

Knowledge Systems in Action: An Example from Indonesia

While many of the programs that RTI implements around the world contribute to various individual elements of a country’s knowledge system, here we focus on a program that incorporates all the elements of knowledge systems previously discussed. The Knowledge Sector Initiative (KSI)^[2] is a partnership between the Australian and Indonesian governments that seeks to improve public policy by strengthening systems to encourage the use of research and evidence in policy making (Australia-Indonesia Partnership for Pro-Poor Policy, 2012). Developing countries like Indonesia that have achieved middle-income status are increasingly seeing the importance of investing in knowledge systems in an effort to transition to a knowledge economy (Lawrence et al., 2020).

As outlined in the 2014 ADB report on knowledge-based economies in Asia (Asian Development Bank, 2014), Indonesia scored 3.11 on the knowledge economy index compared with an average 4.39 for countries in Asia and the Pacific and average 8.25 for OECD (Organisation for Economic Co-operation and Development) countries. For Indonesia to successfully transition to a knowledge economy (defined as an economy based on production, distribution, and use of knowledge and information), it will need to move away from extractive industries and natural resources and invest in the knowledge sector including science, technology, and innovation (Pellini, 2018a, 2018b). The passage of Law No. 11/2019 on a national system of science and technology in Indonesia provides for a national endowment fund for research as well as a new research and innovation agency tasked with coordinating Indonesia’s research agenda. These are positive steps toward the reforms needed to promote a knowledge economy but are only one part of the overall knowledge system.

Systems Approach to Building the Knowledge Sector

KSI’s original design reflected the systems perspective on evidence-based policy making and targeted interventions aimed at researchers (supply), policy makers (demand), public discourse (intermediaries), and funding and regulations (the enabling environment). The KSI team’s approach to implementation was two-pronged, using diagnostics to identify ways to strengthen individual components of the system, while also seeking to enhance linkages among these system components (see KSI diagnostic studies at https://www.ksi-indonesia.org/en/knowledge/detail/93-diagnostic-studies-on-the-knowledge-sector).

To improve the quality of research and policy influence (and increase the production of knowledge [supply]), KSI has provided capacity-building opportunities to 16 Indonesian policy research institutes selected to pilot this approach. These include university-based and independent think tanks, nongovernmental organizations (NGOs), and research centers with such sectoral focuses as HIV and public health, budget analysis and advocacy, foreign policy, and poverty and economic growth. Indonesian government officials did not consider many of these research centers to be reliable sources of policy-relevant analyses because their research was not focused on the immediate policy needs of the government. Further, government officials saw the products of the policy research institutes as of variable quality or not easily applicable. KSI initiated improvements in research quality, research communication, and organizational strengthening through core grants and capacity building. This support has resulted in the following:

• The 16 policy research institutes partnering with KSI are involved in over 50 different policy issues in Indonesia and have increased the breadth of their engagement with different ministries.

• The policy research institutes have directly contributed to multiple policy-relevant products including the revised government procurement regulations and the Policy Quality Index,^[3] as well as an inclusive economic development index, national competency standards for policy analysts, and local regulations, including promoting elderly-friendly measures.

To build demand for data and research among policy makers, KSI collaborated with the National Institute of Public Administration (LAN) on training modules for policy analysts as well as with the National Planning Ministry in support of a policy analysis center.

• LAN incorporated KSI-supported training modules into its own training curriculum for policy analysts. LAN applied these in training policy analysts from multiple ministries charged with translating data and research into policy for their respective divisions and made them publicly available to universities, with several large public universities incorporating the modules into their curriculum. Alumni of the KSI training created the Policy Quality Index, which national and subnational governments are now using to assess and measure the quality of policies under review in their jurisdictions.

• The National Planning Ministry’s analytical staff developed policy papers and briefs to respond to emerging policy issues through the Policy Analysis Center. KSI support for the Center included strengthening analytical tools, training in research and writing methods, and organizational management.

Strengthening the intermediary function of the knowledge system often involved engaging with the media, which plays an important role in bringing key research to the attention of both policy makers and the public.

• KSI provided training for both journalists and researchers to enable them to write about science and data in a compelling, articulate manner.

• KSI developed strategic partnerships between researchers in Indonesia and key media outlets that have extensive digital presence in Indonesia and internationally, such as The Conversation, New Mandala, and Indonesia at Melbourne. Blogs and articles in these outlets written by Indonesian researchers enable the public to both access data and use it to encourage evidence-based policy making. For example, The Conversation published a blog about the reforms to Indonesia’s procurement regulations.

Strengthening the enabling environment for Indonesia’s knowledge sector has been challenging. The government has a still-emerging culture of basing policy on rigorous independent research, and the regulatory framework in practice often inhibits rather than encourages evidence-based policy (for more detail on Indonesia’s knowledge sector, in particular the enabling environment, see Pellini et al., 2018). University researchers applying for government-funded research grants must adhere to the government budget cycle, which requires application, data collection, and reporting, all within a 12-month cycle, with limited options for multiyear research projects. In response to these conditions:

• KSI commissioned diagnostics to identify the bottlenecks in the regulatory framework. Two KSI white papers built on the analysis: one on bureaucratic barriers in the higher education system that inhibit knowledge-to-policy uptake, and a second on provision of knowledge in executive and legislative government (Jackson et al., 2020; Sherlock & Djani, 2015).

• These documents and subsequent KSI workshops with government informed revision of the national procurement regulations to allow multiyear research projects to be commissioned. In addition, KSI support to the Indonesian Academy of Science (AIPI) enabled AIPI to provide analysis and input into the Law on the National System for Research and Technology passed in 2019 (Akademi Ilmu Pengetahuan Indonesia [Indonesian Academy of Sciences], 2019).

Facilitating Collaboration Within the Knowledge System

After commencing work with individual components of the system, KSI began to connect the actors in the knowledge system through initiation of policy working groups. These combine researchers, policy makers, and intermediaries committed to a particular policy area. Each seeks to build a body of knowledge in that policy area, promote its use, and develop or revise required policies. KSI originally set up three policy working groups: one focused on higher education and the regulatory environment for research, another focused on village law, and a third on bureaucratic reform.

The three policy working groups took different paths, but all were demand-driven, led by policy makers, and built a sense of community that fostered collaboration between researchers and policy makers around a particular policy issue. Over time, the leadership role of policy makers within these working groups increased.

At present, the groups have refined and advanced their agendas by identifying critical policy challenges or issues, identifying relevant stakeholders to engage, identifying required knowledge products or data gaps, and developing activities to address issues. The working groups have become progressively more government-led and results-oriented, with objectives and deliverables, and allocating tasks and action items often beyond KSI’s programmatic scope, often mobilizing their own institutional resources.

KSI has emphasized the diversity of knowledge, rather than just availability of data, and engagement among various knowledge producers with each other and with policy makers. Indonesian policy makers, like those in many countries, are adept at cherry-picking data that reinforces their political positions and conveys an illusion of a sound evidence-to-policy process. To mitigate that tendency, KSI also provides knowledge exchange venues and events which enable NGOs, advocacy groups, researchers, and government officials with differing views to provide input and exert influence. Such exchanges have helped to address the power dynamics underlying evidence-based policy where data and knowledge that support policy makers’ predetermined preferences are often favored (Eyben, 2013).

KSI has also encouraged the inclusion of local knowledge (e.g., religious, customary, and indigenous) in informing policy. For example, local knowledge from villages in eastern Indonesia, where clans or church-based groups manage water supply effectively, informed government policy on water projects. In another example, drought management policy was influenced by a West Java farmers group that combined traditional methods with modern tools for monitoring and calculating rainfall, which were successful in avoiding a failed harvest in an area known for drought (Nugroho et al., 2018).

Policy Impact: Changing the Regulation on Procurement of Research

Among KSI’s results was a change in policy on government-sponsored research. In 2018 Indonesia’s president signed into law a decree allowing noncommercial entities (like NGOs and research centers) to bid on government tenders for research. Previously, the government could procure research only from commercial entities or private consultants. This restricted the quality of the data available for policy makers and limited funding for a large portion of knowledge producers. This problem was highlighted in diagnostics commissioned by KSI and presented in 2014 through meetings of KSI research partners.

In response, in 2015 KSI established a policy working group focused on the enabling environment for knowledge-to-policy uptake. Procurement reform quickly became a priority for this working group, which was led by the Ministry of Research, Technology, and Higher Education and included several top-tier universities as well as a group of KSI’s key policy research centers. KSI resourced the policy working group to convene several workshops with a legal drafter to work on the technicalities of redrafting the regulation. With the passing of the revised regulation, KSI has supported multiple meetings and presentations of the National Public Procurement Agency with AKATIGA (one of the policy research institutions supported by KSI) to explain the challenges that NGOs faced and receive input into the revisions to the regulations. KSI is now helping to mobilize a constituency of research institutes, policy makers, and intermediaries around a shared commitment to reform.

Recommendations

RTI has taken lessons learned from the KSI case, as well as from our evidence-to-policy initiatives in other parts of the globe,^[4] and distilled them into five recommendations that are relevant for knowledge systems beyond Indonesia. These recommendations are useful for donor agencies designing new projects related to knowledge systems; practitioners implementing knowledge systems related programs or elements of knowledge systems in sectoral programs; host country government ministries developing knowledge systems strategies or implementing existing strategies; universities, research institutes, or think tanks positioning themselves within a host country knowledge system; and host country civil society organizations and private sector interested in engaging in the knowledge system.

• Promote debate among a diversity of voices within the knowledge systems. Knowledge systems do not consist of mutually exclusive groups of researchers and policy makers encapsulated in neat Venn diagrams. Their reality is much messier, and the richness is precisely in the interactions among the multiple components of the system. In that respect, the greater the diversity of voices interacting, the better. We began KSI neatly distinguishing between researchers, policy makers, and intermediaries. However, as implementation progressed, we observed that often a researcher plays a key role in policy formation, not just knowledge production, and that when policy makers are involved in knowledge production processes from the outset, outcomes are often more robust and inputs more directly applicable. It is important to encourage a broad range of voices interacting in knowledge production and use. Some will be researchers commissioned by policy makers, others will be critical voices of dissent, and still others will be neutral media outlets and such. It is the interaction—the push and pull, the process of challenge and justification—between multiple sources of knowledge that produces rigorous and well-informed policy. Recognizing the power differentials in this push and pull is central to maintain the diversity of voices.

• Provide sustained stakeholder commitment to the systems approach. Knowledge system work is different from, and in fact methodologically contradictory to, most development projects involved in knowledge production. Many development projects provide direct technical assistance, often delivered by foreign experts embedded within ministries, or through analytics and research paid for by the project at the direct demand of policy makers. Building a country’s knowledge system, on the other hand, requires the slow, often painstaking work of building the systems (structures, regulations, budgets, and policies) to deliver these analytics without reliance on donors or foreign expertise. This is usually a deeply political effort of aligning interests, competing over resources, and building the sustainability and capacity of local research institutes. It is not a quick fix, nor does it always provide immediate policy impact. It requires patience, visionary leadership, and commitment from key stakeholders within the political apparatus. The following textbox illustrates the differences in the knowledge systems approach.

• Invest in knowledge system components as well as the interaction between components. A systems approach may need to be introduced gradually in new country contexts where the knowledge sector is still weak or nascent by investing in subcomponents of the knowledge system (supply, demand, intermediary, enabling) to build a foundation for interaction among stakeholders. Some donors may invest in one or more subcomponents of knowledge systems such as

  • improved research quality, research communication, or institutional development of research centers and think tanks (supply);

  • support to government ministries on identifying a research agenda, commissioning research, or extracting research findings and translating them into policy inputs (demand);

  • resources for independent media on investigative journalism for policy, evidence-based advocacy training for civil society organizations (intermediaries); or

  • analytics related to regulatory reform and support for multi-stakeholder groups on new approaches to research funding (enablers).

  It may take time to build the fundamentals of evidence-to-policy in each of these components before constructive interaction will start to bear fruit. In countries with highly functioning knowledge systems, key stakeholders in government, the research community, and civil society cooperate to manage all of these components as well as the interactions among them. These interactions build commitment to a national innovation or knowledge systems plan that outlines joint ideas, goals, and targets. Establishing a coordinating agency facilitates collaboration between different organizations during the implementation of the national plan and mitigates any turf wars (Lawrence et al., 2020).

• Foster a balance between government-led mechanisms and space for civil society perspectives. Sustainability of the knowledge sector depends upon investment in government-led governance and systems. However, providing spaces for independent voices and challenges from civil society is essential, both for influencing emerging policy agendas and for offering input to existing policies. The advantage of government-led governance and systems is ownership from policy makers who are directly involved in shaping discourse and responsible for promoting evidence-based policy within their ministries or line agencies. However, in establishing national priorities it is important to include spaces for civil society to raise issues that may not yet be recognized by government or minimized due to political reasons. Here again, attention to power dynamics is key.

• Explore how knowledge systems can engage the private sector. In knowledge systems, the private sector has not commonly been recognized as a major player, but that is changing. Private foundations and corporations are increasingly important funders, producers, and consumers of research for policy purposes, and they engage in knowledge systems in ways ranging from endowing university research chairs and funding scholarships to direct lobbying activities. The Gates Foundation, for example, supports research on HIV globally that is used to influence health policy in multiple countries.

Conclusion

The recommendations above summarize the principles in this policy brief related to successful knowledge systems development. Their emphasis on diversity of knowledge, recognition of the multiple and overlapping roles of knowledge system actors, and the highlighting of power dynamics in knowledge systems interactions deepen understanding of effective knowledge systems. For application in new country contexts, we recommend a political economy analysis be undertaken to determine areas with the largest potential for reform. Preconditions useful to ensure successful implementation include some degree of acknowledgment of the importance of evidence-based policy making and commitment to doing it (this does not need to be widespread, but change agents with influence somewhere within government are needed).

In addition, some researchers and associated institutions need to be willing to commit to policy research and policy engagement (not all research institutes or universities research centers may be interested, but a critical mass is needed). Finally, press freedom is needed to enable the provision of contrasting evidence or challenges to current policies (again, this does not need to be widespread but at least should offer an opening for presenting evidence that may stray from the status quo). These various facilitating elements can be supported in different ways to strengthen the foundation for knowledge system development and institutionalization.

These preconditions are not essential for knowledge systems to grow, but where they are absent, additional awareness-raising may be needed regarding the importance of evidence-based policy making and how it contributes to a knowledge economy. By demonstrating the benefits of the approach and taking small steps to test the process, the number of committed individuals and institutions can grow, as the KSI experience demonstrates. KSI spent significant time in the early years focused on raising awareness about what knowledge-to-policy means and how it can make a difference. Given the global threats of fake news, hoaxes, and populist distrust of expertise, robust knowledge systems continue to be relevant. This research brief offers some insights on how to develop and sustain them.

Acknowledgments

The authors gratefully acknowledge the financial support from an RTI independent research and development grant for this publication. We also extend our appreciation to the Australian Department of Foreign Affairs and Trade-(DFAT) funded Knowledge Sector Initiative team for support of knowledge systems reforms highlighted in this brief. We would also like to acknowledge the background research conducted by Kara Reeve; helpful comments from Eric Johnson, Anna Wetterberg, Sara Lawrence, and Budiati Prasetiamartati; constructive comments from the anonymous reviewer; fact checks from Irene Pingkan Umboh and Sugiyanto; graphics support from Sonja Douglas; and photo support from Benedicta Kusuma.

Cover photo credit: Hubertus Gorat, Knowledge Sector Initiative