Background

In 2019-20, the Centre ran a series of listening events. These identified the cultural sector's demand for better training and peer learning in all aspects of evaluation. Practitioners were eager to capture and share the comprehensive impact of their work in a way that spoke to funders, policymakers and their audiences.

In response, we convened a working group of 46 experts and, over nine months, we charted a path through a labyrinth of debates and ideas. The result was a bespoke set of Evaluation Principles.

We purposefully designed the Principles to be broad enough to be used by a range of practitioners working across various contexts. The danger of this approach was that the Principles would become too conceptual and distanced from practice.

To help avoid this pitfall, we have produced a range of accompanying resources, including an online training course and a podcast series.

In this resource, we have called on three experts to share reports, articles and projects that speak to using the Principles and highlight some of the most urgent concerns we face when designing, implementing and sharing the evaluation of cultural activities.

The first three reads are recommended by Stephen Welsh. Stephen is a cultural practitioner who works with organisations to support them to diversify and democratise their decision-making. He also hosted our podcast about what empathy looks like in cultural sector evaluation.

The next two recommendations come from Dawn Cameron, an experienced freelance evaluator who specialises in the arts, cultural and heritage sectors. Dawn hosted our podcast episode about when and why we should be transparent when sharing our evaluation.

The Centre’s Emma McDowell has recommended the final two reads. Emma is part of our research team and has led the development of the Evaluation Learning Space. You can also hear from Emma on our podcast episode about proportionate evaluation.

Selection by Stephen Welsh, freelance cultural practitioner

How much do you care?
Marge Ainsley, 2023

In recent years, cultural organisations have begun prioritising a wider range of care, including emotional, physical and social. This is particularly true when it comes to how they establish and maintain equitable and respectful relationships with audiences and communities.

In this article, arts and cultural consultant Marge Ainsley discusses the importance of a care-centred approach to monitoring, evaluation and research. Marge considers how the pressure cultural organisations are placed under by funders to gather a certain type of data can result in carelessness in how audiences and communities and their contributions are treated during evaluation.

To avoid this, Marge explains how to create a care-centred evaluation, one that aligns closely with the Evaluation Principles of being empathetic, many-voiced, people-centred and socially engaged.

Considering Co-creation
Heart of Glass and Battersea Arts Centre, 2021

Co-creation is increasingly gaining popularity in the cultural sector. In my work, I support cultural organisations to explore the full scope of co-creation, looking beyond using it to generate content and instead incorporating it into their overall practices and procedures, including evaluation.

With funding from Arts Council England, Battersea Arts Centre and Heart of Glass examined various aspects of co-creation in this report. They emphasise that as a method of shared creation, it can be applied to "all stages of programme development, design and evaluation." Their findings highlight the effective characteristics of co-creation, which chime with many of the Evaluation Principles, such as being committed to learning and open-minded. They also warn that the barriers to impactful co-creation are often a lack of time and resources and unrealistic expectations.

As a follow-on read, Mark Robinson's How to co-create an evaluation gives clear, expert advice on what to think about when it comes to the purpose and processes involved in co-creating evaluation.

If Nothing Changes, Nothing Changes
Ahmed Iqbal Ullah Race Relations Resource Centre and Education Trust, 2022

In recent years, there has been a surge in cultural organisations and the sector as a whole reaffirming their commitment to equity, diversity and inclusion. This is owing, in part, to the fact that marginalised and disadvantaged groups continue to be underserved in terms of access to culture and underrepresented in audiences and the workforce.

Evaluation continues to play a vital role in addressing obstacles and improving services. Yet, it may also be less successful and even harmful if certain groups are excluded from the process, or if they do participate, they are misinterpreted or exploited.

In this report, the Ahmed Iqbal Ullah Race Relations Resource Centre and Education Trust examined how heritage organisations in England were engaging with anti-racism and equity, diversity and inclusion work, including the role of evaluation. The research pointed out "limited approaches to evaluating collaborations with Global Majority groups and communities." In addition, it cautioned that: "Meaningful long-term change is unlikely without effective evaluation that delivers real learning and a willingness to apply that learning."

In 2022, Victoria Hutchinson wrote a blog post for The Sensory Trust titled: Our top tips for creative evaluation. In it, she outlined five key steps for making evaluation more inclusive that align with the Evaluation Principles of being ethical, people-centred and socially engaged.

Selection by Dawn Cameron, independent evaluator

Hearing Stories of Change
NSPCC, 2022

Stories exist in all cultures and help us to understand the world. They tell us something about people, their relationships to others and about the events they experience. A key function of story is to make someone else's experiences knowable to others.

Story is used extensively in evaluation practice. Practitioners will probably be familiar with case studies and with the Most Significant Change approach (which Emma highlights in the final resource).

Stories can clarify complicated ideas. A few years ago, I was evaluating a project using stories told to me by senior professionals in the arts sector. During a lengthy conversation, one woman said to me: "I don't want to always be the brown girl who talks about the brown things." In so doing, she illuminated – in her own words – the burden of representation faced by many people of colour working in the creative sector. She also provided the most quotable extract of the report. And in my view, it could only have emerged through a storytelling approach.

Stories are not necessarily straightforward, though. How do we decide whose stories to tell? What do we mean when we talk about expertise through lived experience? How do we avoid universalising individual experience? How much might our own positionality affect the stories we elevate and the stories we fail to tell?

With these questions in mind, I've chosen to highlight this report as an example from outside the cultural sector that aligns with the Evaluation Principle of being people-centred. The NSPCC used a Transformative Evaluation storytelling approach in their evaluation of its Together for Childhood programme. One of the report's key findings is that sharing stories can help highlight where progress is being made and what works well.

FailSpace
Jancovich and Stevenson, 2022

It's rare for projects to be entirely successful or entirely unsuccessful. Unexpected things happen.

I think most of us know that it's impossible for everything to go right all of the time. But, with increased competition for funding, a seemingly constant demand for innovation and a perceived need to demonstrate the efficacy of arts and culture to tackle disparate areas of social and public policy, it can feel difficult and risky to admit that something has not worked – or has not worked entirely. The challenge – given the inevitability that failures will occur – is to find ways to learn from them.

FailSpace is a research project led by Professor Leila Jancovich and Professor David Stevenson. The team has produced a series of useful tools for anyone keen to talk more openly about failure and explore the Evaluation Principles of being transparent, many-voiced and committed to learning. For example, the Wheel of Failure provides a framework for discussing a project's relative success and failure from various viewpoints.

Following on from this, it’s also enlightening to listen to Harvard researcher Amy Edmondson unpick what separates constructive failures from those we should try to avoid in this Hidden Brain podcast episode.

" … failure is just information."
Toni Morrison, American writer and editor

Selection by Emma McDowell, researcher at the Centre for Cultural Value

The Little Book of Creative Evaluation
Christou, Owen and Ceyhan, 2023

This online resource from the team at Lancaster University is great for organisations, teams or individuals who want to learn more about how creative evaluation can support their needs and goals.

As the authors point out, there is not one clear way of “doing” creative evaluation. Instead, they describe it as “a constellation of approaches that employ creative methods, tools and thinking” that produce evaluations that aim to “foster collaboration, mutual learning, inclusivity and engagement”.

The resource highlights how learning from and responding to the specific context of the evaluation, along with being open and flexible in how participants can contribute to the evaluation design and practice, are key elements of creative evaluation. Following these approaches often connects with the Evaluation Principles of being many-voiced, empathetic and committed to learning.

As well as outlining these principles, the book provides a range of case studies, introducing methods such as fieldnote diaries, storytelling, journey mapping, illustrate postcards and poetry.

Most Significant Change methodology
By Future Arts Centres and Mark Robinson, 2024

This report draws on the findings from an action research group made up of several UK arts centres who all tested the Most Significant Change (MSC) evaluation approach from 2023–24.

As a participatory, qualitative evaluation approach, the MSC methodology enables participants to talk about their experiences in their own words “rather than asking them to squeeze their experiences into pre-determined boxes labelled with either your own or funders’ set of prescribed outcomes”. It uses “stories of change” as data, which are collected and then analysed through a process of shared discussion with key stakeholders.

Because it can be embedded across many levels within an arts, cultural or heritage organisation, from governance to delivery, the MSC methodology can support organisational and programme development, as well as help further understand and articulate the impact of the work.

Combining case studies with learning on using the methodology, this is a rich resource with actionable learning for anyone interested in using the tool in their own evaluation practice. This resource works as an example of a connected and people-centred evaluation method.