Source: Joel Gibbs
Recent developments such as the global pandemic, gender violence, Black Lives Matter, and not least the climate crisis, have accelerated the search for participatory methods of engagement, participation, and impact across a number of research disciplines. Within these disciplines, citizen voice and lived experience are sought to deepen understanding behind behavioural change as a result of policy or project intervention strategies. Within this there are limitations, for example, the UK Civil Service in 2021 recognises that ‘most policymakers do not consistently have the skills, incentives, or infrastructure to find new evidence about citizens’ (Knight, 2021). It is also argued that policy solutions are too often based on stakeholder opinion, and rarely are the voices and experiences of citizens used to evidence policymaking (UK Civil Service 2020, p.63). Therefore, policy engagement methodologies as the basis of evidence gathering are themselves generating ‘marginalised groups’, ‘the seldom heard’, and creating barriers to access for citizens. The report goes on to argue that this is due to policymakers not being ‘confident or effective at talking to diverse groups of people, whose background and culture differs from their own’ (UK Civil Service 2020, p.65) because it is ‘easier for officials to turn to groups which represent these people than it is to contact them individually’ because this can ‘whip up emotions, making it harder to actually do their job’ (UK Civil Service, 2020, p.65). Further, evaluations of any given policy may not take place until up to five years after its introduction, whereby ‘policymakers rarely find out whether their ideas have delivered meaningful change for citizens’ (UK Civil Service 2020, p.63) until a significant amount of time has passed.
Given these concerns there has been a recent burgeoning of interest in new forms of data gathering for project evaluations. In response to these challenges, a practice research methodology has been developed by Dr Charlie Ingram during his PhD studies called ‘Evaluative Performance’.
Briefly, Evaluative Performance is a collaboration between headphone verbatim theatre (HVT) arts practice and social science evaluation frameworks. It is a method of data gathering that involves a performance created solely from the words spoken by interview participants, combined with a post-performance contextual discussion. Interview data is collected by the researcher and then edited into a performance based around a theme, subject matter, or event. In the context of policy evaluation, the research data collected can be framed around the policy aims and objectives of an intervention with questions being asked of the beneficiary and/or participant – such as a citizen or stakeholder.
The process of conducting this method is as follows:
- Participant selection and question schedule creation.
- Interview process.
- Edit of collected audio into short clips or soundbites through content analysis.
- Allow participant to have final ‘sign off’ on their personal edit.
- Arrange clips alongside other participants’ voices to build narrative and create an ‘audio-script’.
- Rehearse audio-script with actors.
- Perform to (and record for) project evaluators, stakeholders and/or the general public (where appropriate).
- Conduct a roundtable/Q&A following the performance.
This approach has already been tested, with successes in both cultural and urban policy interventions through an evaluation of civic pride during Coventry UK City of Culture 2021 (Ingram, 2023), and energy policy innovation consultations with young people (Ingram et al., 2023).
Explore some of the examples of this method in practice below:
Capturing Citizen Experience
https://youtu.be/14_PGrwd2_Y
Capturing Social Change Attribution by Citizens
https://youtu.be/P5DLO6PpCgo
Young People and Smart Local Energy Systems
https://youtu.be/GcZ5mkOdi8Y
Further testing of this method is currently taking place, within Ingram’s postdoctoral research project. Innovative Approaches to Evaluating Impact: Testing The ‘Evaluative Performance’ Method and Approach in Transdisciplinary Environments.
If you would like more information on the method, or would be interesting in collaborating on future research, please get in touch with Charlie Ingram.
References:
Ingram, C. (2023). Theatre arts in UK City of Culture evaluation practices: the case for headphone verbatim. Arts and the Market, 13(3), 159-173. https://doi.org/10.1108/AAM-08-2021-0033
Ingram, C., Halford, A., & Gaura, E. (2023). Cyber-physical Advances in SLES: Resource pack for school workshop participants. EnergyREV. https://tinyurl.com/bpaufvy6
Knight, A. (2021) 11 Things Policymakers Need to Improve Outcomes for Citizens – Public Policy Design [online] available from [Feb 27, 2022]
UK Civil Service (2020) Policymaker Perspectives on Reform. Online: UK Civil Service.