This Framework for RRI in ICT builds on and incorporates the broader EPSRC framework. It draws on research undertaken in the Framework for Responsible Research and Innovation in ICT to provide more detailed insights into how principles of RRI can be implemented.

This Framework is informed by characteristics that are typical for ICT, notably the high speed of innovation and diffusion, ubiquity and pervasiveness of ICT, the difficult distinction between applied and fundamental research, the logical malleability of ICT many artefacts and the problem of many hands that renders it particularly difficult to hold individuals accountable for the consequences of ICT use.

Anticipate

Is the planned research methodology acceptable?

The choice of research method and approach can have consequences that are problematic for participants of the research and the broader social and natural environment. Methodological choices also influence the possible findings and thus the relevance and consequences of the project. The following methods should be considered:

Justification / appropriateness of the methodology

Justification of the methodology is part of normal research work. There are differences between disciplines with regards to the perceived importance of methodology. From an RRI perspective it is important to specifically consider the consequences that choices of methodology have for the conduct and side effects of the research.

Lab health & safety

Ensuring the health and safety of researchers and others involved in research is paramount to ensure acceptability of the research process.
Links:
Responsible Research – Managing Health and Safety in Research

Risk assessment

Risk assessment and risk management are part of project management. The degree to which they are required depends on type of project and funder. Risk management can be done explicitly or implicitly. From the perspective of responsiveness in RRI, explicit risk assessment and mitigation strategies are desirable to allow for broad feedback on research approaches.

Data management

Data management strategies are crucial. Appropriate data management is required to ensure transparency of findings and conclusion. On the other hand, data may include personal information and therefore raise data protection and privacy issues, as well as questions of intellectual property. In addition there is a political push in the UK for open data to maximise the impact of research.
Links:
UK Data Archive

Ethical approval/Informed consent

Most research requires ethical approval by an appropriate university or other committee. Ethics approval tends to based on biomedical ethics with the core principle of informed consent. There is a debate about the extent to which the biomedical paradigm can be applied to ICT research. Ethics review can comprise all of the issues discussed in this section, in addition to questions concerned to the way in which respondents are treated.
Links:
Human Research Ethics Approval
Ethics and informed consent in online social network research
Digital technologies for supporting the informed consent of children and young people in research: the potential for transforming current research ethics practice
Rethinking informed consent

Will the products be socially desirable?

The social desirability of an intended research outcome can have consequences for the future uptake and use of products that may be developed by the commercial sector.  For this reason it is important to investigate whether or not the innova tion is consistent with existing cultural norms and customs. The following questions should be considered:

Foresight

Involves the application of critical thinking to long-term developments. Developing a clear statement of the vision for how the research might influence the future will assist in making explicit what may be otherwise implicit impacts. Additionally, the envisioning of alternative futures might provide alternative pathways for development. Finally, foresight can be used for short, mid and long-term planning and priority setting.

Vision assessment

Is the critical analysis of visions for the future. These act as guides in the identification of ‘grand challenges’ and research funding. These are ‘attainable futures’ that are considered desirable by a collection of stakeholders. Analysis of these shared visions can reveal underlying assumptions in the construction of the problem definition and solution assessments. (Roelofsen et al 2007).

Scenarios

These are stories that describe different but equally plausible futures. They are used to systematically gather perceptions about certainties and uncertainties. They can be used to formulate strategies as well as assist in the decision-making process (Selin 2005). Scenarios are tools that can be used in foresight exercises.

How sustainable are the outcomes?

The social desirability of an intended research outcome can have consequences for the future uptake and use of products that may be developed by the commercial sector.  For this reason it is important to investigate whether or not the innova tion is consistent with existing cultural norms and customs. The following questions should be considered:

Foresight

Involves the application of critical thinking to long-term developments. Developing a clear statement of the vision for how the research might influence the future will assist in making explicit what may be otherwise implicit impacts. Additionally, the envisioning of alternative futures might provide alternative pathways for development. Finally, foresight can be used for short, mid and long-term planning and priority setting.

Vision assessment

Is the critical analysis of visions for the future. These act as guides in the identification of ‘grand challenges’ and research funding. These are ‘attainable futures’ that are considered desirable by a collection of stakeholders. Analysis of these shared visions can reveal underlying assumptions in the construction of the problem definition and solution assessments. (Roelofsen et al 2007).

Scenarios

These are stories that describe different but equally plausible futures. They are used to systematically gather perceptions about certainties and uncertainties. They can be used to formulate strategies as well as assist in the decision-making process (Selin 2005). Scenarios are tools that can be used in foresight exercises.

Why should this research be undertaken?

Explain and justify the research for a broad audience using non-technical language.

Addressing grand challenges

Define the how your research might maximise the social and economic impact in the future.

  • Economic growth
  • Social need
  • Scientific curiosity
  • Extended impact statement

Have we included the right stakeholders?

Whether it is in the short or long term, and whether intended or unintended, research will impact society. To ensure desirability and acceptability of research impact, those affected need to be included into R&I processes as stakeholders. It therefore is important to investigate whether all relevant stakeholder-groups affected by the innovation are taken into consideration.  This question needs occasionally to be revisited throughout the research process to make certain stakeholder groups are included that came into scope while the research developed

Principles of stakeholder engagement (Sciencewise & BScienceAssoc)

What is Stakeholder Engagement?

Freeman (1984, p. 46) defines stakeholders as ‘any group or individual who can affect, or is affected by, the achievement of the organization’s objectives.’ Furthermore, Stakeholder engagement has been defined by the ISEA (1999, p. 91) as ‘the process of seeking stakeholder views on their relationship with an organisation in a way that may realistically be expected to elicit them’. Part of planning stakeholder engagement is identifying stakeholders and characterising relationships with each group of them (ISEA)

Stakeholder engagement then is the process used by an organisation to engage relevant stakeholders for a clear purpose to achieve accepted outcomes. It is now also recognised as a fundamental accountability mechanism, since it obliges an organisation to involve stakeholders in identifying, understanding and responding to sustainability issues and concerns, and to report, explain and be answerable to stakeholders for decisions, actions and performance (AA1000, 2011). In addition AA1000SES suggests different levels of engagement such as:

  • Consult: survey, focus groups, public meetings, workshops
  • Involve: multi-stakeholder forums, advisory panels, consensus building processes, participatory decision making processes, focus groups
  • Collaborate: Joint projects, partnerships
  • Empower: Integration of stakeholders into governance strategy and operations management

Links
Example of Stakeholder Engagement Principles
http://www.nda.gov.uk/documents/upload/Public-and-Stakeholder-Engagement…

AA1000 Stakeholder Engagement Standard (AA1000SES)
http://www.accountability.org/standards/aa1000ses/index.html

Reflect

Which mechanisms are used to reflect on process?

A key component of responsible innovation is the reflection by the various stakeholders on their own practice and its outcome. Reflexivity  is generally recognised as a positive trait of academic research. At the same time it is often difficult to achieve. The practical problem with reflexivity is that most of the time we are caught up in our daily practice and do not find the time or see the reason for reflecting on what we do, why we do it and how we do it. The literature on research governance therefore highlights some activities or project management strategies that can help strengthen reflexivity.

Advisory board

By integrating external experts or stakeholders into an advisory board, the researchers have a clear target audience who have a commitment to think about and provide feedback to the project. Advisory boards are widely used but they can only fulfil their role of providing useful feedback if they are managed well and have clear contact points and roles and if their input is taken seriously in the project.

Internal (reflective) workshop

There is often a lack of time to reflect on the project during normal meetings which tend to be caught up in functional requirements of the project. Reflective workshops that allow various project members (possibly including the advisory board) to get to know each other better and understand each others language and ways of thinking. This can include a better understanding of the rationale for joining the project as well as the various views on its content and collaboration. Further issues to be discussed can be topics directly linked to RRI such as gender, foresight, research ethics, open access or the relationship to the public and civil society.

‘Stage-gating’

This is a well-developed project management approach that defines certain goals which need to achieved at a particular stage of the project. These stage gates are decision points that require strategic decisions about the future of the project. The idea can be applied to RRI by defining objectives or stages that are expected. This can foster reflexivity within the project as the participants are forced to consider their progress and consequences arising from it.

Sociotechnical integration

The term refers to the explicit integration of broader societal concerns into research and innovation processes. It has significant overlaps with the idea of RRI in general. It furthermore contains a research agenda that aims to explore how the integration of scientific / technical work with social science / humanities can work.
Links:
http://cns.asu.edu/research/stir

‘Midstream modulation’

This is an example fo sociotechnical integration. The term refers to the metaphors of “upstream” and “downstream” engagement and seeks to find a way to influence techno-scientific research and development while it is being performed. The idea is to integrate scholars from “reflective disciplines” i.e. social sciences or humanities into the practical scientific / technical work. These embedded scholars can then raise questions from their perspective, which can lead everybody involved to reflect on their positions.

Backcasting / Hindsight

During the project researchers and other stakeholders build up experience of undertaking research responsibly as well as limitations and downsides of the various approaches. At present much of this experience remains unused and is not made explicit. Sharing good practice based on experience is important. The term backcasting furthermore refers to a planning methodology where, based on a desired future, current decisions are determined.
Links:
http://en.wikipedia.org/wiki/Backcasting

How could you do it differently?

The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to research process and methodology.

Questions to be considered with regards to research product are:

  • How will fundamental research influence application-oriented research?
  • How does the product deal with issues of ubiquity and pervasiveness?
  • Is there transparency regarding the features and functionality of the product?
  • Is it possible for users to control which aspects features and functionality to enable or disable?
  • Have target users feedback and suggestion been implemented in further iterations of the product?

How do you know what the consequences will be?

Investigate what might be the short and long-term the effect, result, or outcome of this product. For instance, how will it transform activities, communication, and collaboration? How will it be perceived, for instance, as trustworthy or misleading?

Systematic evaluation of technologies in situ

It is important to evaluate technologies with target users in the settings in which they will actually be utilized. This includes integrating user-centred design principles in the design of novel technologies. For instance, conducting interviews with target users to understand their requirements, observing the environment in which the technology might be embedded to develop an understanding of how it might transform activities, and conduct user studies with participants in an assessment of prototypes from the users’ point-of-view. Finally, iteratively incorporate findings from user studies into the further design of the product.

 

What might be the potential use?

Investigate how the product might be adapted or further developed into larger sociotechnical systems. How could the technology be combined with other infrastructures or devices? What might be the impact of possible or plausible adaptations?

Intended and unintended

Examine plausible positive, unexpected benefits as well as negative, unexpected harms. These might be related to effects upon the environment, animal welfare, human health and behaviour, and social interaction.

Misuse cases

Examine how the product might be used for harmful purposes, for example, surveillance or personal data mining. These will be specific to the product in development, but the key is to examine the possible consequences of misleading uses.

What don’t we know about?

Investigate factors that have been unexplored or where evidence of social impacts are inconclusive.

Blind spots

This is where one has to make an effort to check for potential misjudgments in strategic direction. The metaphor is taken from when drivers check when changing lanes in a car by turning their head and not relying on side mirrors alone. In this way, examine the ambiguities of the potential impacts of a product. What is unclear about its impacts? Articulating vague outcomes can assist in planning for their possible appearance in the future.

What don’t we know about?

Investigate factors that have been unexplored or where evidence of social impacts are inconclusive.

Blind spots

This is where one has to make an effort to check for potential misjudgments in strategic direction. The metaphor is taken from when drivers check when changing lanes in a car by turning their head and not relying on side mirrors alone. In this way, examine the ambiguities of the potential impacts of a product. What is unclear about its impacts? Articulating vague outcomes can assist in planning for their possible appearance in the future.

How can we ensure social desirability?

Incorporate techniques that take into account social norms such as respect for human values, quality of life, security, sustainability, privacy and social justice. These shared values are articulated in the Treaty of the European Union and should be reflected in research outcomes (Von Schomberg 2013).

Privacy by design

Where privacy and data protection compliance is designed into systems at the early stages of development rather than being added on later after a system has been designed. This is includes making privacy the ‘default setting’ including encryption of personal information, and secure retention of data.

How could you do it differently?

The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to research process and methodology.

Questions to be considered with regards to research product are:

  • How will fundamental research influence application-oriented research?
  • How does the product deal with issues of ubiquity and pervasiveness?
  • Is there transparency regarding the features and functionality of the product?
  • Is it possible for users to control which aspects features and functionality to enable or disable?
  • Have target users feedback and suggestion been implemented in further iterations of the product?

Who is affected?

In order to reflect upon who is affected by the research an explain needs to be drawn up on how the innovation impacts and/or concerns each stakeholder. The following questions should be considered:

  • Who might care?
  • Which groups or individuals have a stake in the research and its outcomes?
  • Who benefits?
  • Which groups or individuals will benefit from the research and its outputs?
  • Who is in control?
  • Which groups or individuals influence the research? In what ways do these stakeholders bring their influence to bear?
  • Who will decide?
  • Which groups or individuals make decisions concerning the research and its outcomes?
  • Who will take responsibility if things go wrong?
  • Which groups or individuals can be held responsible in any way for unwanted/harmful consequences of the research?
  • What is the gender balance in the project?

How could you do it differently?

It is part of all reflective activities to consider possible alternatives. The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to people.

Questions to be considered with regards to people involved are:

  • How were stakeholders identified?
  • Who was involved in deciding what stakeholders to involve?
  • How are the stakeholders relevant to meeting social challenges?
  • Does the research consider the legitimate interests of those who will be affected by it?

Have we included the right stakeholders?

Whether it is in the short or long term, and whether intended or unintended, research will impact society. To ensure desirability and acceptability of research impact, those affected need to be included into R&I processes as stakeholders. It therefore is important to investigate whether all relevant stakeholder-groups affected by the innovation are taken into consideration.  This question needs occasionally to be revisited throughout the research process to make certain stakeholder groups are included that came into scope while the research developed

Principles of stakeholder engagement (Sciencewise & BScienceAssoc)

What is Stakeholder Engagement?

Freeman (1984, p. 46) defines stakeholders as ‘any group or individual who can affect, or is affected by, the achievement of the organization’s objectives.’ Furthermore, Stakeholder engagement has been defined by the ISEA (1999, p. 91) as ‘the process of seeking stakeholder views on their relationship with an organisation in a way that may realistically be expected to elicit them’. Part of planning stakeholder engagement is identifying stakeholders and characterising relationships with each group of them (ISEA)

Stakeholder engagement then is the process used by an organisation to engage relevant stakeholders for a clear purpose to achieve accepted outcomes. It is now also recognised as a fundamental accountability mechanism, since it obliges an organisation to involve stakeholders in identifying, understanding and responding to sustainability issues and concerns, and to report, explain and be answerable to stakeholders for decisions, actions and performance (AA1000, 2011). In addition AA1000SES suggests different levels of engagement such as:

  • Consult: survey, focus groups, public meetings, workshops
  • Involve: multi-stakeholder forums, advisory panels, consensus building processes, participatory decision making processes, focus groups
  • Collaborate: Joint projects, partnerships
  • Empower: Integration of stakeholders into governance strategy and operations management

Links
Example of Stakeholder Engagement Principles
http://www.nda.gov.uk/documents/upload/Public-and-Stakeholder-Engagement…

AA1000 Stakeholder Engagement Standard (AA1000SES)
http://www.accountability.org/standards/aa1000ses/index.html

Engage

How to engage a wide group of stakeholders? 

Broader public engagement is a central aspect of RRI. Participation allows for feedback on the research itself, the process and the purpose. It can increase the legitimacy of findings, broaden the knowledge base and enrich the research. Public engagement should go beyond increasing the public understanding of science (even though this is a legitimate aspect of it) and allow for a two-way communication between researchers and the public.

Identify stakeholders

The first step in public engagement is to identify those part of the public who are interested in the research. It is important to notice that not all stakeholders are interested in joining research and there are successful examples of uninvited participation:

Links for the integration of Civil Society Organisations in research see: http://www.consider-project.eu/

Participatory processes

There are numerous methodologies for the integration of wider stakeholders into decision processes and research. Choosing the appropriate one depends on the stakeholders in question, expected outcomes and the way in which public engagement links to the research.

Links
http://toolbox.ippaproject.eu/toolboxes

Process evaluation

Public engagement activities, if they are to have a transformative effect on the research, need to be well prepared and evaluated. Evaluation mechanisms should be implemented to ensure that the engagement process is successful.

What are the viewpoints of a wide group of stakeholders? 

Actively involve all stakeholders (e.g. industry, citizens, end users and policymakers) in the product design process ensuring a shared stake in successful outcomes. This includes incorporation of user-centred design principles and participatory design where target users are directly involved in the design of products.

Public engagement mechanisms

This is where the general public (both target users and the wider public) are consulted not only in the product design but also provide input regarding the strategic direction of the product, agenda-setting, and decision making related to how the product might influence social interaction and human behaviour.

Prototype / demonstrator evaluation (public)

Allow for public evaluation of research prototypes that, in the future, may be developed into commercial products. This could include exposure at public events or using the Internet to provide large, geographically distributed groups access. Ensure there are processes in place for the public to submit their feedback.

Who prioritises research? 

Investigate which stakeholders make decisions within the innovation process.

Public engagement mechanisms

For whom is the research done?

Explore stakeholders who (will) use or purposely will be/are affected by the innovation.

Example: Child voice and user-involvement in the development of learning technologies for children with autism

Act

How can your research structure become flexible?

In order to be responsive to societal and stakeholder needs, research and innovation activities need to be able to react flexibly to changing circumstances. While flexibility is something that is often difficult to achieve in academic projects which are funded on the basis of a plan covering several years, there are ways of rendering projects flexible.

Agile project management

Agile development methodologies are well established in software engineering. Many of the principles of ideas such as frequent engagement with customers and stakeholders, early development of prototypes etc. may be in the spirit of responsible innovation and help research activities remain flexible.

Document emerging perspective, views and norms

To justify flexible development of the project and responsiveness to changing circumstances, the reasons for the change should be documented and related to the project.

Recalibrating the vision of the project

It should be possible to re-evaluate and, where necessary, to recalibrate the vision of the project and, consequently the research approach. This may mean a revision of the research question, the research methodology or intended outputs.

What training is required?

In order to ensure that the research and innovation process is acceptable and complies with current statutory and other requirements, training needs to be available in the following areas:

Research integrity

This refers to accepted standards of researcher behaviour to ensure truthfulness, replicability

Links:
UK Research Integrity Office
The Concordat to Support Research Integrity

Research management

Research and project management are skills that any researcher, PI or coordinator should possess. They are a necessary precondition for steering a research project in desirable directions.

Skills and methods in public engagement

Data management

is required to ensure privacy and confidentiality of personal data as well as availability and transparency of other research data.
Links
UK Data Archive

What infrastructure is required?

Departmental ethics committee capable of addressing ICT concerns

Funding for engagement activities
Tools to support the ICT community
Database of project ‘lessons-learned’

What needs to be done to ensure social desirability?

  • Develop ways to motivate action for the co-design of products.
  • Create incentives for thinking about research outputs
  • Encourage responsible design by recognising efforts with awards, funding and other activities.
  • Encourage appropriate development approaches
  • Create internal processes that facilitate iterative, participatory design.

What training is required?

In order to ensure that research and innovation products are acceptable and complies with current statutory and other requirements, training needs to be available in the following areas:

Understanding regulation and compliance

This refers to conforming to government policies, standards and the law.

Links
The Information Commissioner’s Office (ICO)

What infrastructure is required?

Accessible participatory tools and methods

How do we ensure that the implied future is desirable?

Consider implied future state at project/programme inception.

Horizon-scanning

Develop scenarios that provide systematic examinations of plausible futures.

What training is required?

Consider new skills that could be developed.

Understanding of ELSI

Become familiar with analysing research through the lens of  ethical, legal and social implications (ELSI). Examine the moral, regulatory, and cultural impacts or consequences.

Understanding current debates and controversies

Keep up-to-date with social issues in your area of research. This can be achieved by engaging in debates that appear in the media or public reactions to research. Consider taking a course in media engagement or similar.

What infrastructure is required?

Consider new or existing organizational structures that could better support research outputs.

Reflect on purpose of funding mechanisms (What does this mean/refer to?)

Reflect on purpose of project evaluation criteria (What does this mean/refer to?)

Who matters?

Actively involve relevant stakeholders into the product design process.

What capacities are required?

In order for the research and innovation process to be inclusive and participatory, capacity needs to be built in the areas: (AA1000, 2011)

Knowledge

e.g. understanding and awareness of the issues, the organisation, the local cultures and politics.

Skills

e.g. language and communication skills, ability to examine and interpret the outputs of stakeholder engagement in a way that captures the key facts and figures, as well as messages and insights, and Individual personality traits such as integrity, ability to focus on solutions, motivation, and creativity

Opportunity

e.g. necessary financial and physical resources, time to participate, access to information

What training is required?

Consider new skills that could be developed.

Contextualise projects as sociotechnical

What infrastructure is required?

Consider new or existing organizational structures that could better support research outputs.

  • Community building
  • Leadership council
  • Champions / Advocates
  • Science education to allow the public to engage intelligently