This Framework is a tool that helps those involved in research and innovation in ICT to do so responsibly. The Framework presented here consists of a set of scaffolding questions that allow researchers, funders and other stakeholders to consider a range of aspects of ICT research. This introduction to the Framework answers the following questions:
Responsible research and innovation (RRI) aims to help individuals and organisations to ensure the acceptability and desirability of research and innovation. It is an agenda that has been adopted by major research funders including the European Commission and the UK Engineering and Physical Science Research Council (EPSRC). The EPSRC has published a framework for RRI that is based on the following four principles:
Process
Process refers to the processes undertaken in research and innovation. These cover all activities in preparing research, undertaking data collection and analysis, storage and presentation of data and interaction with respondents.
Product
Responsible Research and Innovation (RRI) is specifically
concerned with the outcomes of research and innovation activities. This can refer to products or services. It includes the consequences of use as well as misuse of research products and the impact that research has on the natural and social environment.
Purpose
Critical scrutiny in RRI extends beyond the conduct (process) and outcome (product) of research and covers the question why research is undertaken at all. The purpose of research is a crucial factor influencing acceptability and desirability and thus open to scrutiny.
People
Research and innovation are undertaken by people and for people and have intended and unintended consequences for people. People are at the heart of RRI and need to be explicitly considered.
Anticipate
Describing and analysing the impacts, intended or otherwise, (e.g. economic, social, environmental) that might arise. This does not seek to predict but rather to support an exploration of possible impacts and implications that may otherwise remain uncovered and little discussed.
Reflect
Reflecting on the purposes of, motivations for and potential implications of the research, and the associated uncertainties, areas of ignorance, assumptions, framings, questions, dilemmas and social transformations these may bring.
Engage
Opening up such visions, impacts and questioning to broader deliberation, dialogue, engagement and debate in an inclusive way.
Act
Using these processes to influence the direction and trajectory of the research and innovation process itself.
RRI is a topic that is strongly debated in more contested areas of research and technology, such as nanotechnology, synthetic biology or geoengineering. There are, however, some characteristics or features of ICT that set them apart from other fields. Key among them are:
Speed of innovation and diffusion
Ubiquity and pervasiveness
Applied and fundamental research
Logical malleability
Problem of many hands
Multitude of Backgrounds
The Framework maps the four main components of the EPSRC AREA principles against four core aspects of RRI:
Process refers to the processes undertaken in research and innovation. These cover all activities in preparing research, undertaking data collection and analysis, storage and presentation of data and interaction with respondents.
Product, RRI is specifically interested in the outcomes of research and innovation activities. This can refer to products or services. It includes the consequences of use as well as misuse of research products and the impact that research has on the natural and social environment.
Purpose. Critical scrutiny in RRI extends beyond the conduct (process) and outcome (product) of research and covers the question why research is undertaken at all. The purpose of research is a crucial factor influencing acceptability and desirability and thus open to scrutiny.
People. Research and innovation are undertaken by people and for people and have intended and unintended consequences for people. People are at the heart of RRI and need to be explicitly considered.
By mapping the AREA principles against the PPPP components of RRI, this framework creates the space to think about details of how RRI can be made relevant to ICT.
The framework consists of a set of questions that allow a researcher, funder, policymaker or other interested party to structure the way they think about research. It can be used to gain an overview of all sorts of different aspects of RRI in ICT. It can furthermore be utilised to gain insight into specific issues, questions or applications.
This Framework aims to provide a comprehensive overview that allows different stakeholders to navigate their way through RRI in ICT. Many of the issues, questions and suggestions are not likely to be relevant to all stakeholders or problems. The Framework has therefore been used to create a set of stakeholder-specific guidelines and recommendations that are more accessible and user friendly and easier to implement. Specific guidelines include those for
- Researchers
- Universities
- Funders
- Industry
- Civil Society
This Framework for RRI in ICT builds on and incorporates the broader EPSRC framework. It draws on research undertaken in the Framework for Responsible Research and Innovation in ICT to provide more detailed insights into how principles of RRI can be implemented.
This Framework is informed by characteristics that are typical for ICT, notably the high speed of innovation and diffusion, ubiquity and pervasiveness of ICT, the difficult distinction between applied and fundamental research, the logical malleability of ICT many artefacts and the problem of many hands that renders it particularly difficult to hold individuals accountable for the consequences of ICT use.
Anticipate
Is the planned research methodology acceptable?
The choice of research method and approach can have consequences that are problematic for participants of the research and the broader social and natural environment. Methodological choices also influence the possible findings and thus the relevance and consequences of the project. The following methods should be considered:
Justification / appropriateness of the methodology
Justification of the methodology is part of normal research work. There are differences between disciplines with regards to the perceived importance of methodology. From an RRI perspective it is important to specifically consider the consequences that choices of methodology have for the conduct and side effects of the research.
Lab health & safety
Ensuring the health and safety of researchers and others involved in research is paramount to ensure acceptability of the research process.
Links:
Responsible Research – Managing Health and Safety in Research
Risk assessment
Risk assessment and risk management are part of project management. The degree to which they are required depends on type of project and funder. Risk management can be done explicitly or implicitly. From the perspective of responsiveness in RRI, explicit risk assessment and mitigation strategies are desirable to allow for broad feedback on research approaches.
Data management
Data management strategies are crucial. Appropriate data management is required to ensure transparency of findings and conclusion. On the other hand, data may include personal information and therefore raise data protection and privacy issues, as well as questions of intellectual property. In addition there is a political push in the UK for open data to maximise the impact of research.
Links:
UK Data Archive
Ethical approval/Informed consent
Most research requires ethical approval by an appropriate university or other committee. Ethics approval tends to based on biomedical ethics with the core principle of informed consent. There is a debate about the extent to which the biomedical paradigm can be applied to ICT research. Ethics review can comprise all of the issues discussed in this section, in addition to questions concerned to the way in which respondents are treated.
Links:
Human Research Ethics Approval
Ethics and informed consent in online social network research
Digital technologies for supporting the informed consent of children and young people in research: the potential for transforming current research ethics practice
Rethinking informed consent
Will the products be socially desirable?
The social desirability of an intended research outcome can have consequences for the future uptake and use of products that may be developed by the commercial sector. For this reason it is important to investigate whether or not the innova tion is consistent with existing cultural norms and customs. The following questions should be considered:
Foresight
Involves the application of critical thinking to long-term developments. Developing a clear statement of the vision for how the research might influence the future will assist in making explicit what may be otherwise implicit impacts. Additionally, the envisioning of alternative futures might provide alternative pathways for development. Finally, foresight can be used for short, mid and long-term planning and priority setting.
Vision assessment
Is the critical analysis of visions for the future. These act as guides in the identification of ‘grand challenges’ and research funding. These are ‘attainable futures’ that are considered desirable by a collection of stakeholders. Analysis of these shared visions can reveal underlying assumptions in the construction of the problem definition and solution assessments. (Roelofsen et al 2007).
Scenarios
These are stories that describe different but equally plausible futures. They are used to systematically gather perceptions about certainties and uncertainties. They can be used to formulate strategies as well as assist in the decision-making process (Selin 2005). Scenarios are tools that can be used in foresight exercises.
How sustainable are the outcomes?
The social desirability of an intended research outcome can have consequences for the future uptake and use of products that may be developed by the commercial sector. For this reason it is important to investigate whether or not the innova tion is consistent with existing cultural norms and customs. The following questions should be considered:
Foresight
Involves the application of critical thinking to long-term developments. Developing a clear statement of the vision for how the research might influence the future will assist in making explicit what may be otherwise implicit impacts. Additionally, the envisioning of alternative futures might provide alternative pathways for development. Finally, foresight can be used for short, mid and long-term planning and priority setting.
Vision assessment
Is the critical analysis of visions for the future. These act as guides in the identification of ‘grand challenges’ and research funding. These are ‘attainable futures’ that are considered desirable by a collection of stakeholders. Analysis of these shared visions can reveal underlying assumptions in the construction of the problem definition and solution assessments. (Roelofsen et al 2007).
Scenarios
These are stories that describe different but equally plausible futures. They are used to systematically gather perceptions about certainties and uncertainties. They can be used to formulate strategies as well as assist in the decision-making process (Selin 2005). Scenarios are tools that can be used in foresight exercises.
Why should this research be undertaken?
Explain and justify the research for a broad audience using non-technical language.
Addressing grand challenges
Define the how your research might maximise the social and economic impact in the future.
- Economic growth
- Social need
- Scientific curiosity
- Extended impact statement
Have we included the right stakeholders?
Whether it is in the short or long term, and whether intended or unintended, research will impact society. To ensure desirability and acceptability of research impact, those affected need to be included into R&I processes as stakeholders. It therefore is important to investigate whether all relevant stakeholder-groups affected by the innovation are taken into consideration. This question needs occasionally to be revisited throughout the research process to make certain stakeholder groups are included that came into scope while the research developed
Principles of stakeholder engagement (Sciencewise & BScienceAssoc)
What is Stakeholder Engagement?
Freeman (1984, p. 46) defines stakeholders as ‘any group or individual who can affect, or is affected by, the achievement of the organization’s objectives.’ Furthermore, Stakeholder engagement has been defined by the ISEA (1999, p. 91) as ‘the process of seeking stakeholder views on their relationship with an organisation in a way that may realistically be expected to elicit them’. Part of planning stakeholder engagement is identifying stakeholders and characterising relationships with each group of them (ISEA)
Stakeholder engagement then is the process used by an organisation to engage relevant stakeholders for a clear purpose to achieve accepted outcomes. It is now also recognised as a fundamental accountability mechanism, since it obliges an organisation to involve stakeholders in identifying, understanding and responding to sustainability issues and concerns, and to report, explain and be answerable to stakeholders for decisions, actions and performance (AA1000, 2011). In addition AA1000SES suggests different levels of engagement such as:
- Consult: survey, focus groups, public meetings, workshops
- Involve: multi-stakeholder forums, advisory panels, consensus building processes, participatory decision making processes, focus groups
- Collaborate: Joint projects, partnerships
- Empower: Integration of stakeholders into governance strategy and operations management
Links
Example of Stakeholder Engagement Principles
http://www.nda.gov.uk/documents/upload/Public-and-Stakeholder-Engagement…
AA1000 Stakeholder Engagement Standard (AA1000SES)
http://www.accountability.org/standards/aa1000ses/index.html
Reflect
Which mechanisms are used to reflect on process?
A key component of responsible innovation is the reflection by the various stakeholders on their own practice and its outcome. Reflexivity is generally recognised as a positive trait of academic research. At the same time it is often difficult to achieve. The practical problem with reflexivity is that most of the time we are caught up in our daily practice and do not find the time or see the reason for reflecting on what we do, why we do it and how we do it. The literature on research governance therefore highlights some activities or project management strategies that can help strengthen reflexivity.
Advisory board
By integrating external experts or stakeholders into an advisory board, the researchers have a clear target audience who have a commitment to think about and provide feedback to the project. Advisory boards are widely used but they can only fulfil their role of providing useful feedback if they are managed well and have clear contact points and roles and if their input is taken seriously in the project.
Internal (reflective) workshop
There is often a lack of time to reflect on the project during normal meetings which tend to be caught up in functional requirements of the project. Reflective workshops that allow various project members (possibly including the advisory board) to get to know each other better and understand each others language and ways of thinking. This can include a better understanding of the rationale for joining the project as well as the various views on its content and collaboration. Further issues to be discussed can be topics directly linked to RRI such as gender, foresight, research ethics, open access or the relationship to the public and civil society.
‘Stage-gating’
This is a well-developed project management approach that defines certain goals which need to achieved at a particular stage of the project. These stage gates are decision points that require strategic decisions about the future of the project. The idea can be applied to RRI by defining objectives or stages that are expected. This can foster reflexivity within the project as the participants are forced to consider their progress and consequences arising from it.
Sociotechnical integration
The term refers to the explicit integration of broader societal concerns into research and innovation processes. It has significant overlaps with the idea of RRI in general. It furthermore contains a research agenda that aims to explore how the integration of scientific / technical work with social science / humanities can work.
Links:
http://cns.asu.edu/research/stir
‘Midstream modulation’
This is an example fo sociotechnical integration. The term refers to the metaphors of “upstream” and “downstream” engagement and seeks to find a way to influence techno-scientific research and development while it is being performed. The idea is to integrate scholars from “reflective disciplines” i.e. social sciences or humanities into the practical scientific / technical work. These embedded scholars can then raise questions from their perspective, which can lead everybody involved to reflect on their positions.
Backcasting / Hindsight
During the project researchers and other stakeholders build up experience of undertaking research responsibly as well as limitations and downsides of the various approaches. At present much of this experience remains unused and is not made explicit. Sharing good practice based on experience is important. The term backcasting furthermore refers to a planning methodology where, based on a desired future, current decisions are determined.
Links:
http://en.wikipedia.org/wiki/Backcasting
How could you do it differently?
The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to research process and methodology.
Questions to be considered with regards to research product are:
- How will fundamental research influence application-oriented research?
- How does the product deal with issues of ubiquity and pervasiveness?
- Is there transparency regarding the features and functionality of the product?
- Is it possible for users to control which aspects features and functionality to enable or disable?
- Have target users feedback and suggestion been implemented in further iterations of the product?
How do you know what the consequences will be?
Investigate what might be the short and long-term the effect, result, or outcome of this product. For instance, how will it transform activities, communication, and collaboration? How will it be perceived, for instance, as trustworthy or misleading?
Systematic evaluation of technologies in situ
It is important to evaluate technologies with target users in the settings in which they will actually be utilized. This includes integrating user-centred design principles in the design of novel technologies. For instance, conducting interviews with target users to understand their requirements, observing the environment in which the technology might be embedded to develop an understanding of how it might transform activities, and conduct user studies with participants in an assessment of prototypes from the users’ point-of-view. Finally, iteratively incorporate findings from user studies into the further design of the product.
What might be the potential use?
Investigate how the product might be adapted or further developed into larger sociotechnical systems. How could the technology be combined with other infrastructures or devices? What might be the impact of possible or plausible adaptations?
Intended and unintended
Examine plausible positive, unexpected benefits as well as negative, unexpected harms. These might be related to effects upon the environment, animal welfare, human health and behaviour, and social interaction.
Misuse cases
Examine how the product might be used for harmful purposes, for example, surveillance or personal data mining. These will be specific to the product in development, but the key is to examine the possible consequences of misleading uses.
What don’t we know about?
Investigate factors that have been unexplored or where evidence of social impacts are inconclusive.
Blind spots
This is where one has to make an effort to check for potential misjudgments in strategic direction. The metaphor is taken from when drivers check when changing lanes in a car by turning their head and not relying on side mirrors alone. In this way, examine the ambiguities of the potential impacts of a product. What is unclear about its impacts? Articulating vague outcomes can assist in planning for their possible appearance in the future.
What don’t we know about?
Investigate factors that have been unexplored or where evidence of social impacts are inconclusive.
Blind spots
This is where one has to make an effort to check for potential misjudgments in strategic direction. The metaphor is taken from when drivers check when changing lanes in a car by turning their head and not relying on side mirrors alone. In this way, examine the ambiguities of the potential impacts of a product. What is unclear about its impacts? Articulating vague outcomes can assist in planning for their possible appearance in the future.
How can we ensure social desirability?
Incorporate techniques that take into account social norms such as respect for human values, quality of life, security, sustainability, privacy and social justice. These shared values are articulated in the Treaty of the European Union and should be reflected in research outcomes (Von Schomberg 2013).
Privacy by design
Where privacy and data protection compliance is designed into systems at the early stages of development rather than being added on later after a system has been designed. This is includes making privacy the ‘default setting’ including encryption of personal information, and secure retention of data.
How could you do it differently?
The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to research process and methodology.
Questions to be considered with regards to research product are:
- How will fundamental research influence application-oriented research?
- How does the product deal with issues of ubiquity and pervasiveness?
- Is there transparency regarding the features and functionality of the product?
- Is it possible for users to control which aspects features and functionality to enable or disable?
- Have target users feedback and suggestion been implemented in further iterations of the product?
Who is affected?
In order to reflect upon who is affected by the research an explain needs to be drawn up on how the innovation impacts and/or concerns each stakeholder. The following questions should be considered:
- Who might care?
- Which groups or individuals have a stake in the research and its outcomes?
- Who benefits?
- Which groups or individuals will benefit from the research and its outputs?
- Who is in control?
- Which groups or individuals influence the research? In what ways do these stakeholders bring their influence to bear?
- Who will decide?
- Which groups or individuals make decisions concerning the research and its outcomes?
- Who will take responsibility if things go wrong?
- Which groups or individuals can be held responsible in any way for unwanted/harmful consequences of the research?
- What is the gender balance in the project?
How could you do it differently?
It is part of all reflective activities to consider possible alternatives. The question “how could I have done it differently” refers to all aspects of the research and innovation. It ranges from agenda setting and identification of research question to dissemination and results. In this section the question refers to people.
Questions to be considered with regards to people involved are:
- How were stakeholders identified?
- Who was involved in deciding what stakeholders to involve?
- How are the stakeholders relevant to meeting social challenges?
- Does the research consider the legitimate interests of those who will be affected by it?
Have we included the right stakeholders?
Whether it is in the short or long term, and whether intended or unintended, research will impact society. To ensure desirability and acceptability of research impact, those affected need to be included into R&I processes as stakeholders. It therefore is important to investigate whether all relevant stakeholder-groups affected by the innovation are taken into consideration. This question needs occasionally to be revisited throughout the research process to make certain stakeholder groups are included that came into scope while the research developed
Principles of stakeholder engagement (Sciencewise & BScienceAssoc)
What is Stakeholder Engagement?
Freeman (1984, p. 46) defines stakeholders as ‘any group or individual who can affect, or is affected by, the achievement of the organization’s objectives.’ Furthermore, Stakeholder engagement has been defined by the ISEA (1999, p. 91) as ‘the process of seeking stakeholder views on their relationship with an organisation in a way that may realistically be expected to elicit them’. Part of planning stakeholder engagement is identifying stakeholders and characterising relationships with each group of them (ISEA)
Stakeholder engagement then is the process used by an organisation to engage relevant stakeholders for a clear purpose to achieve accepted outcomes. It is now also recognised as a fundamental accountability mechanism, since it obliges an organisation to involve stakeholders in identifying, understanding and responding to sustainability issues and concerns, and to report, explain and be answerable to stakeholders for decisions, actions and performance (AA1000, 2011). In addition AA1000SES suggests different levels of engagement such as:
- Consult: survey, focus groups, public meetings, workshops
- Involve: multi-stakeholder forums, advisory panels, consensus building processes, participatory decision making processes, focus groups
- Collaborate: Joint projects, partnerships
- Empower: Integration of stakeholders into governance strategy and operations management
Links
Example of Stakeholder Engagement Principles
http://www.nda.gov.uk/documents/upload/Public-and-Stakeholder-Engagement…
AA1000 Stakeholder Engagement Standard (AA1000SES)
http://www.accountability.org/standards/aa1000ses/index.html
Engage
How to engage a wide group of stakeholders?
Broader public engagement is a central aspect of RRI. Participation allows for feedback on the research itself, the process and the purpose. It can increase the legitimacy of findings, broaden the knowledge base and enrich the research. Public engagement should go beyond increasing the public understanding of science (even though this is a legitimate aspect of it) and allow for a two-way communication between researchers and the public.
Identify stakeholders
The first step in public engagement is to identify those part of the public who are interested in the research. It is important to notice that not all stakeholders are interested in joining research and there are successful examples of uninvited participation:
Links for the integration of Civil Society Organisations in research see: http://www.consider-project.eu/
Participatory processes
There are numerous methodologies for the integration of wider stakeholders into decision processes and research. Choosing the appropriate one depends on the stakeholders in question, expected outcomes and the way in which public engagement links to the research.
Links
http://toolbox.ippaproject.eu/toolboxes
Process evaluation
Public engagement activities, if they are to have a transformative effect on the research, need to be well prepared and evaluated. Evaluation mechanisms should be implemented to ensure that the engagement process is successful.
What are the viewpoints of a wide group of stakeholders?
Actively involve all stakeholders (e.g. industry, citizens, end users and policymakers) in the product design process ensuring a shared stake in successful outcomes. This includes incorporation of user-centred design principles and participatory design where target users are directly involved in the design of products.
Public engagement mechanisms
This is where the general public (both target users and the wider public) are consulted not only in the product design but also provide input regarding the strategic direction of the product, agenda-setting, and decision making related to how the product might influence social interaction and human behaviour.
Prototype / demonstrator evaluation (public)
Allow for public evaluation of research prototypes that, in the future, may be developed into commercial products. This could include exposure at public events or using the Internet to provide large, geographically distributed groups access. Ensure there are processes in place for the public to submit their feedback.
Who prioritises research?
Investigate which stakeholders make decisions within the innovation process.
Public engagement mechanisms
For whom is the research done?
Explore stakeholders who (will) use or purposely will be/are affected by the innovation.
Act
How can your research structure become flexible?
In order to be responsive to societal and stakeholder needs, research and innovation activities need to be able to react flexibly to changing circumstances. While flexibility is something that is often difficult to achieve in academic projects which are funded on the basis of a plan covering several years, there are ways of rendering projects flexible.
Agile project management
Agile development methodologies are well established in software engineering. Many of the principles of ideas such as frequent engagement with customers and stakeholders, early development of prototypes etc. may be in the spirit of responsible innovation and help research activities remain flexible.
Document emerging perspective, views and norms
To justify flexible development of the project and responsiveness to changing circumstances, the reasons for the change should be documented and related to the project.
Recalibrating the vision of the project
It should be possible to re-evaluate and, where necessary, to recalibrate the vision of the project and, consequently the research approach. This may mean a revision of the research question, the research methodology or intended outputs.
What training is required?
In order to ensure that the research and innovation process is acceptable and complies with current statutory and other requirements, training needs to be available in the following areas:
Research integrity
This refers to accepted standards of researcher behaviour to ensure truthfulness, replicability
Links:
UK Research Integrity Office
The Concordat to Support Research Integrity
Research management
Research and project management are skills that any researcher, PI or coordinator should possess. They are a necessary precondition for steering a research project in desirable directions.
Skills and methods in public engagement
Data management
is required to ensure privacy and confidentiality of personal data as well as availability and transparency of other research data.
Links
UK Data Archive
What infrastructure is required?
Departmental ethics committee capable of addressing ICT concerns
Funding for engagement activities
Tools to support the ICT community
Database of project ‘lessons-learned’
What needs to be done to ensure social desirability?
- Develop ways to motivate action for the co-design of products.
- Create incentives for thinking about research outputs
- Encourage responsible design by recognising efforts with awards, funding and other activities.
- Encourage appropriate development approaches
- Create internal processes that facilitate iterative, participatory design.
What training is required?
In order to ensure that research and innovation products are acceptable and complies with current statutory and other requirements, training needs to be available in the following areas:
Understanding regulation and compliance
This refers to conforming to government policies, standards and the law.
What infrastructure is required?
Accessible participatory tools and methods
How do we ensure that the implied future is desirable?
Consider implied future state at project/programme inception.
Horizon-scanning
Develop scenarios that provide systematic examinations of plausible futures.
What training is required?
Consider new skills that could be developed.
Understanding of ELSI
Become familiar with analysing research through the lens of ethical, legal and social implications (ELSI). Examine the moral, regulatory, and cultural impacts or consequences.
Understanding current debates and controversies
Keep up-to-date with social issues in your area of research. This can be achieved by engaging in debates that appear in the media or public reactions to research. Consider taking a course in media engagement or similar.
What infrastructure is required?
Consider new or existing organizational structures that could better support research outputs.
Reflect on purpose of funding mechanisms (What does this mean/refer to?)
Reflect on purpose of project evaluation criteria (What does this mean/refer to?)
Who matters?
Actively involve relevant stakeholders into the product design process.
What capacities are required?
In order for the research and innovation process to be inclusive and participatory, capacity needs to be built in the areas: (AA1000, 2011)
Knowledge
e.g. understanding and awareness of the issues, the organisation, the local cultures and politics.
Skills
e.g. language and communication skills, ability to examine and interpret the outputs of stakeholder engagement in a way that captures the key facts and figures, as well as messages and insights, and Individual personality traits such as integrity, ability to focus on solutions, motivation, and creativity
Opportunity
e.g. necessary financial and physical resources, time to participate, access to information
What training is required?
Consider new skills that could be developed.
Contextualise projects as sociotechnical
What infrastructure is required?
Consider new or existing organizational structures that could better support research outputs.
- Community building
- Leadership council
- Champions / Advocates
- Science education to allow the public to engage intelligently