Introduction
Stakeholder engagement is an important component in developing policies and regulations on critical issues such as the use and development of new methods of product innovation, such as biotechnologies and nanotechnologies. It offers an opportunity to engage with stakeholders and to address their concerns and answer their questions. More importantly, sound, transparent, and trusted policy recommendations can be developed by understanding various stakeholder groups’ perspectives, needs, and concerns. Furthermore, stakeholder engagement can identify areas of commonality and agreement or disagreement and reveal the underlying motivations and rationales for differences of opinion and perspective, aiding communication between policy makers and various stakeholder groups. Accounting for stakeholder needs and preferences is especially important during responsible development of new technologies to ensure that research and innovation processes are open, inclusive, and consider diverse perspectives, ultimately creating new technologies that align with societal values, needs, and expectations (Owen et al., 2013; RRI Tools, 2021; Stilgoe et al., 2013).
Stakeholder engagement is routinely called for in governance, risk governance, and policy development within food and agriculture sectors, including those that develop or utilize new food and agriculture technologies. Stakeholder engagement can help incorporate perspectives and concerns from diverse viewpoints to address complex environmental and public health challenges efficiently (Cummings & Kuzma, 2017; International Risk Governance Center, 2017, 2020). For instance, stakeholder engagement can help identify desirable and undesirable technology attributes and inform the problem formulation stage of risk analysis and regulatory decisions (Kokotovich et al., 2020; Kuzma, 2019). In food and agriculture, there are several types of stakeholder groups that are directly or indirectly involved and that can be impacted by decisions and policies. Typical stakeholders in food and agriculture include the general public (i.e., consumers, primary shoppers), academia, government, industry, international, and non-profit organizations.
Decision methods such as multi-criteria decision analysis (MCDA) can incorporate and weigh the different stakeholders’ views, perceptions, and preferences systematically and transparently. These methods have been used to help improve decision-making in several areas such as engineering, environmental health (Linkov & Moberg, 2012), and food safety (Food and Agriculture Organization of the United Nations, 2017; Ruzante et al., 2010). However, before starting any approach such as MCDA, stakeholder preferences need to be captured and characterized.
The first step in stakeholder engagement is to identify key stakeholders in the field of interest and determine how and when they can be involved in policy development (Lemke & Harris-Wai, 2015). The motivation for engaging stakeholders and potential challenges expected during engagement activities also need to be defined. Many methods exist to engage, gather, and quantify stakeholders’ perspectives and views. Focus groups are efficient in engaging participants in a targeted manner and are often used to gather information and identify themes concerning participants’ perceptions, beliefs, and behaviors on a topic in a short timeframe (typically 60–90 minutes); however, it is also more difficult to maintain anonymity, which might be a disadvantage when it comes to discussing potentially polarizing or controversial issues. In-person focus groups are generally used to bring together individuals from a similar geographic area. Therefore, several focus groups need to be conducted to obtain representative points of view and ensure that geographically diverse groups have a chance to discuss their different points of view about an issue, being potentially expensive. Furthermore, focus groups might also need to be hosted outside working hours to include a representative group of participants, which might add a burden to researchers. In-person interviews and mail or online questionnaires are also used to gather stakeholders’ opinions, values, and perceptions.
With mail and online methods, respondent identity can be protected; however, it is often not possible to interact or converse directly with stakeholders on the subject at hand. Community forums and public comments are other methods used to gather stakeholder input, but these may lack the level of active interaction, depth, and dynamic discussion of focus groups and suffer from nonrepresentative samples. Additionally, consensus conferences also capture perceptions and decision-making under “high information” circumstances and provide insights from a more diverse sample of participants (Einsiedel et al., 2001). Consensus conferences have usually been convened to assess how non-experts might respond to a variety of controversial topics. A desirable method to gather stakeholder input would likely include attributes from all of these methods: geographically diverse participation, interactive group discussion, ease of administration, and anonymous participation.
Over the past 10 to 15 years, innovative online research platforms have been developed as alternatives to typical stakeholder engagement approaches, which in many ways come closer to having all desirable attributes. These online stakeholder engagement platforms (OSEPs) overcome many of the limitations related to in-person focus groups, questionnaires, and other methods. They allow researchers to engage geographically diverse stakeholders using a wide range of methods, from virtual focus groups to digital diaries and message boards (Belpasso, 2019; Hussey & Zerfas, 2020). Moreover, stakeholders can participate at their leisure—within the parameters set by the researchers—and from wherever is most comfortable for them. Another advantage of OSEPs is that they allow stakeholders to engage with one another anonymously, which can facilitate more open and uninhibited engagement on issues where they might have different points of view. This anonymity can also help mitigate issues around power dynamics, which can reduce participation by certain stakeholder groups or individuals who may defer to others who “know best.” Most of all, these OSEPs offer flexibility. They can accommodate both fixed-duration projects and efforts to generate ongoing or long-term engagement. CMNTY, EngagementHQ, Incling, The HiVE, and VisionsLive are a few examples of OSEPs that have made stakeholder engagement increasingly convenient for both researchers and stakeholders. These OSEPs have also made it easier to navigate stakeholder engagement during the COVID-19 pandemic, when gathering in person was not possible. One potential disadvantage is that they require stakeholders to have access to the internet, which can unintentionally limit participants’ full availability or privilege certain participant groups.
In this paper we demonstrate the potential benefits (from researchers’ and participants’ points of view) and the application of commercially available OSEPs to elicit stakeholder perceptions on topics where there is disagreement and the views of stakeholders are not clearly defined. In the case study presented, we used an OSEP to explore stakeholder views of responsible innovation (RI) in the context of nanotechnology applications in food and agriculture (nano-agrifood).
Case Study: Responsible Innovation of Nanotechnology in Food and Agriculture
Background
The use of nanotechnology and engineered nanomaterials in food and agriculture sectors may provide potential health benefits, such as the use of nano-emulsions to create nutrient-fortified foods that are appealing to consumers (Aswathanarayan & Vittal, 2019), and potential environmental benefits, as the use of nano-pesticides and nano-fertilizers that may have less ecological impact compared with conventional agrochemicals (Fraceto et al., 2016). However, their full impacts on health, the environment, and society are not yet fully understood and therefore require further investigation on their environmental, health, and safety (EHS) aspects and ethical, legal, and societal implications (ELSI) (Cummings et al., 2021; Grieger, Hansen, et al., 2016; Grieger, Harrington, et al., 2016; Iavicoli et al., 2017; Steenis & Fischer, 2016). RI is a governance paradigm that promotes the principles of anticipation, inclusion, reflexivity, and responsiveness to incorporate not only aspects of EHS and ELSI in early innovation stages but also processes that researchers can utilize to reflect upon and modify their innovation practices to develop products that align with societal needs and expectations (Stilgoe et al., 2013). It is often presumed that when successfully practiced, RI also leads to better ground-up or “upstream” policymaking.
In this case study, we were interested in characterizing stakeholders’ views of RI for nano-agrifood applications across the farm-to-fork continuum. The outcomes from the case study selected here included identifying approaches and best practices that may help address stakeholders’ concerns surrounding nano-agrifoods, with the goal of ultimately improving RI for nano-agrifoods, and are reported elsewhere (Grieger et al., 2021).
Online Stakeholder Engagement Platform
In the present case study, we developed a series of stakeholder engagement surveys and discussion forums on the CMNTY platform (CMNTY Corporation, 2021) to allow for an open, inclusive, and effective means for deliberation on the topic of RI for nano-agrifoods. More specifically, it focused on applications of nano-agrifood across the food continuum because stakeholder perceptions and attitudes differ across various nano-agrifood applications as it has been shown by Steenis & Fischer (2016). Multiple platforms, including CMNTY, EngagementHQ, Incling, The HiVE, and VisionsLive all have very similar functionalities: flexibility for sharing platform content, options to solicit stakeholder feedback using multiple research methods (i.e., individual questionnaires and group discussions), ability to maintain user anonymity; however, we did not conduct a comparative analysis among these platforms, and CMNTY (2021) was selected for convenience because some of the researchers had previous experience with the platform. All study procedures were developed and submitted to the Institutional Review Board at North Carolina State University for approval. The following section outlines the specific steps taken to develop and conduct stakeholder engagement using CMNTY.
Platform Development
Recruitment. We identified stakeholder candidates based on their affiliations (i.e., government, nongovernmental organization [NGO] think tanks, advocacy organizations, industry, and academia) through the research team’s connections in food and agricultural industries, research consortia, and networks. We conducted a literature review to identify authors publishing articles in the fields of interest. We also reviewed the program and speaker lists of high-profile conferences in nano-agrifoods and conducted a broader internet search on relevant stakeholder groups in nano-agrifood innovations to obtain potential candidates to join the study. We did not recruit consumers and the general public directly, but we did include advocacy organizations representing these groups.
In total, we invited 442 stakeholders to participate. The study offered a $100 incentive upon completion of all activities. The benefits for participation included opportunities to contribute to stakeholder discussions that can influence recommendations on nano-agrifood policy, be involved in research that develops best practices for RI of agrifoods, voice key perceptions and concerns, and identify approaches to address concerns to support agrifood RI. Of the 442 invited, 62 individuals agreed to participate. The final group of participants represented a range of agrifood sectors, including government, NGO, industry, and academia.
Platform Setup. We set up the CMNTY platform to elicit responses from stakeholders using a mixed-methods approach that included independent activities where participants had no interaction with others (e.g., questionnaires) and a group activity where discussions between participants were expected (e.g., message board–style discussion forum). The home page included a welcome message that provided participants with clear instructions on how to complete the various activities and in what order to complete them. Figure 1 provides a screenshot of the welcome page. None of the activities were actively moderated by the research team; however, researchers were available to answer any clarifying questions via the chat feature.
We used two initial questionnaires to assess participant familiarity with nanotechnology and engineered nanomaterials in food and agriculture and to understand their own definition of RI and related practices. Following the initial questionnaires, participants individually reviewed five case studies based on existing uses of nanotechnology in food and agriculture. Each case study explored a different application and rationale for using nanomaterial. Questions following the description of each case study aimed to quantify how each stakeholder viewed the usefulness, safety, benefits, and other considerations associated with the case study, including whether they thought each case study was an example of RI. Figure 2 shows a screenshot of the questionnaire page that participants completed independently.
Following the case studies, we asked participants to rank each of the five case studies from most to least aligned with their own definition of RI. We also asked them to share the three most important factors that influenced their ranking. The study included a “Situating Worldviews” multiple-choice questionnaire to gauge participants’ general perceptions and beliefs on science and technology more broadly.
For the group activity, we created a message board discussion forum for participants to discuss topics with threaded comments. Participants could comment in response to the discussion prompt and view and respond to comments posted by other participants. The message board contained five discussion prompts, including what types of barriers exist concerning RI for nano-agrifoods and how they can be overcome.
A final insights questionnaire captured participants’ concluding views concerning RI for nano-agrifoods, and an exit survey asked participants about their experience using the platform—specifically, whether they thought it worked well—and to give them an opportunity to provide any additional feedback.
Table 1 summarizes platform activities, numbered in the order that participants progressed through the activities.
Stakeholder Participation
The discussion forum and in the consent form that participants signed before starting the activities clearly described expectations around the level and type of engagement. The consent form outlined that participants would regularly participate in the platform by (1) completing the individual questionnaires, (2) completing a group forum exercise, and (3) participating on the platform at least two times over 3 weeks. The expected level of effort (2 to 4 hours) was also noted. In the discussion forum, we asked participants to respond to each question by adding their own answers and by commenting on one or two responses made by other participants. We also encouraged participants to log in and review the discussion board multiple times over the course of the study. In terms of etiquette for engaging in online discussions, we asked participants to practice respectful communication and disagreement. It was clear that there would be no tolerance for disrespectful (i.e., offensive, hateful, or obscene) comments. Members of the research team monitored the platform for new activity and encouraged participation through regular reminder emails. However, there was no real-time moderation of the discussions, and research team members did not post comments on the message board.
Evaluation of the Platform
Of the 62 who accepted the invitation to participate, 55 completed all activities on the platform: government (16.4%; n = 9); NGO – think tank (12.7%; n = 7); NGO – advocacy (18.2%; n = 10); industry (18.2%; n = 10); and academia (34.5%; n = 19). This high completion rate (89%) coupled with the high quality of responses (i.e., thorough and considered responses) speaks to the effectiveness of OSEPs for online stakeholder engagement, as discussed below. Despite the instructions provided, the engagement level varied widely across participants. The discussion forum garnered 401 responses (170 original posts and 231 replies), and the number of responses per participant ranged from 1 to 17, with an average of 7.
The majority of study participants (80%, n = 44 of 55) agreed or strongly agreed that the OSEP worked well for this study. The most frequently cited advantage (n = 12 participants cited this advantage) of the platform was its message board–style discussion feature. One participant shared, “The [discussion] forum was very interesting, and it was very helpful to read and evaluate the cases individually and then engage in the discussion. It was a good exercise to make me think about RI and also learn from others." Other participants commented that they were “impressed by the quality of the discussion on the message boards” and that "it was eye-opening to see so many different points of view.” In fact, exposure to diverse viewpoints was cited as an advantage of the OSEP (n = 6 participants cited this advantage), speaking to OSEP’s ability to engage diverse stakeholder audiences effectively and efficiently.
Participants also appreciated that the OSEP was easy to use and navigate (n = 6), that it facilitated a mixed-methods research approach (n = 6), and that it allowed for self-pacing (n = 6) and anonymity (n = 3). One participant who strongly agreed that the platform worked well for the study noted, “Really liked this format. Easy to navigate, easy to pause and continue as my schedule and energy allowed. I also really enjoyed engaging with other subjects in the forum. Excellent hybrid of survey and focus group research methods.” Several other participants commented on the mixed-methods approach, specifically the benefits of independent assessment followed by group discussion. For instance, one participant shared:
I thought it worked well because it provided material for all to read, and questions for all to answer, to provide a common background for the discussion between participants whom I suspect have a wide range of familiarity with nanotechnology. The fact that the questions prior to the group discussion required rankings as well as written responses allowed positions on the topics to be identified and the reasons why explained. Having gone through this all “together” seemed to me to break the ice and allow a wide-ranging, and respectful discussion to follow.
This comment speaks to the notion of “leveling the playing field” by ensuring that all participants have an opportunity to read study materials and consider their stance before the group discussion, a benefit that was reiterated by another participant who appreciated the ability to self-pace: “I liked having time to field the questions and formulate my answers. I do not always think as well when I am placed in a group.”
Although the feedback on the platform was overwhelmingly positive, participants identified a few downsides to engaging stakeholders in this manner, namely the lack of “live interaction,” which created an inability to ask clarifying questions and to discuss perspectives in depth, which is particularly problematic for a complex topic such as RI in food and agriculture. For instance, one participant noted, “It was sometimes difficult to understand what was meant in the discussions. In a live situation, you could ask someone to clarify their statement or to check your understanding of [what] they said. You can’t easily do that in this format.”
Discussion
The inclusion of stakeholder perspectives is essential for responsibly designing and developing nano-agrifoods. In the case study described here, a commercially available OSEP was used to obtain stakeholders’ individual perspectives and their asynchronous interactions with other stakeholders assessing the same information. The goal was to connect with and learn from a wide range of stakeholders directly involved in nano-agrifoods, including members of NGOs, advocacy organizations, academia, industry, and government, using a single platform; however, it could have also been used to connect and engage the general public and consumers just as efficiently. Moreover, the platform supported a diverse array of research methods, which allowed the researchers to use a mixture of questionnaires and group discussions to obtain a rich dataset without overly burdening participants. As an example of this array of research methods, the researchers designed this study so that stakeholders first answered a series of questionnaires independently before engaging in a group discussion. Several study participants noted the advantage of this stepwise approach and felt that it facilitated a more robust discussion.
From the researchers’ point of view, using an OSEP allowed both individual questionnaires and a group discussion forum to be conducted on the same platform, increasing the ease of administration and tracking of participant progress. The platform also provided in-platform analysis capabilities that allowed researchers to identify emerging trends in the results from the multiple-choice questionnaires and an easy way to export results. Finally, the OSEP subscription included regular meetings with company employees who could answer questions about how to best integrate our research into the platform.
This study had several limitations that can serve as lessons learned for researchers planning to use OSEPs. First, according to participants, a weakness was that it lacked “live” or dynamic interaction. Although we used the discussion forum to engage stakeholders in a conversation, this forum is modeled after a message board in which participants respond at their leisure to a static prompt rather than in “real time” to moderated questions. The OSEP used here does offer a virtual focus group feature; however, we did not use this data collection method for the current study because the primary goal was to have all stakeholders participating in discussions together. Depending on the goals of the research study, it may be beneficial to utilize the “live” features of OSEPs to facilitate a more interactive engagement. Discussion boards could be timed for specific days or participants could be randomly assigned to a particular day to allow more time for exchange to facilitate engagement and restrict participants from posting all comments at one time.
Another limitation was that despite the instructions provided in the consent form and in message boards to describe what was expected of study participants, several individuals shared that they did not clearly understand the goals of the project or of the stakeholder engagement. Therefore, it is important to consider the use of multiple methods through which participants can engage in real time with researchers so that the goals of the engagement are better understood. Repetition of expectations throughout the activities might also be helpful when asking stakeholders to participate in an OSEP such as the one used here.
Although one advantage of OSEPs is that stakeholders can participate whenever is most convenient for them, future studies that utilize these platforms may find it beneficial to have predetermined, advertised windows of time (i.e., a specific time each day or week of the study) for research team members to engage with stakeholders in real time to answer any clarifying questions and facilitate conversations where needed to facilitate a “two-way” interaction between stakeholders and the research team as a way to address the previously discussed limitations. However, one disadvantage of ongoing researcher participation is that participants may be more reticent to express their views freely for fear of being incorrect or in deference to what they view as “experts” conducting the forum behind the scenes. There is likely an appropriate balance to strike in this regard.
Access to the internet was not an issue for the stakeholder groups targeted in this case study; however, depending on the groups that need to be involved in the discussion, reliable internet access needs to be considered so this does not become an excluding factor for participation and the engagement process can be truly inclusive. That would be an important factor to be considered if using these platforms to engage with the general public.
Overall, our use of an OSEP resulted in positive experiences for most participants, both for being part of research evaluating real-world applications of nano-food innovations and for furthering their own thinking about RI in the food sector. This particular OSEP provided a reliable and engaging environment for stakeholders to exchange and discuss ideas and could be further explored by policy makers and other decision makers as an innovative tool to engage different types of stakeholders.
Acknowledgments
The authors would like to state that there are no conflict of interest involving authors and CMNTY, the software tool used in the case study and cited in this paper.
The authors would like to acknowledge Christopher L. Cummings from North Carolina State University for his contributions to the development of the stakeholder platform activities and study design. This research was conducted with funding from the United States Department of Agriculture, National Institute for Food and Agriculture.