Introduction

Since 1990, the UK has hosted two European Capitals of Culture and three UK Cities of Culture. This resource uncovers critical insights from the evaluations about the impacts of the following three UK Cities and two European Capitals of Culture programmes on the local and visitor economy:

• Glasgow European Capital of Culture, 1990
• Liverpool European Capital of Culture, 2008
• Derry-Londonderry UK City of Culture, 2013
• Hull UK City of Culture, 2017
• Coventry UK City of Culture, 2021

For ease, we will call these city-wide events ‘UK-based Cities of Culture’ when we refer to them collectively. Yet, it is important to note that all these events were distinct in terms of resource, budget, socio-economic and historical context. For example, unlike the European Capital of Culture, the UK City of Culture competition has no financial award. Rather, the successful hosting city is expected to attract significant monetary investment due to winning the title.  

The differing contexts in which these programmes and their evaluations took place can make comparison between the Cities of Culture difficult. Evaluation practice itself has also changed a great deal since 1990, and the evaluation reports and activity reflect these differences.

Evaluating a large-scale cultural project such as a City of Culture is not straightforward. There is a wide range of activities to evaluate and multiple stakeholders to involve and consult. The evaluations had to communicate the value of Cities of Culture activity across a broad range of areas, from health impacts to environmental outcomes. 

Each evaluation captured the breadth of material differently. Some reports were split into distinct sections, while others ‘clustered’ themes. For instance, the post-project evaluation for Derry-Londonderry 2013 was structured very differently from the others. It focused much more on the project performance and mapping the local policy. Liverpool 2008 and Coventry 2021 produced different reports for different audiences, hosted on a publicly available website. Hull 2017 reported consistently across thematic clusters. 

One of the most prominent themes for all the UK-based City of Culture evaluations was measuring impacts on the local and visitor economy, as well as local business and policy. This resource reviews the findings related to these areas in more detail.

While this is a common focus for much evaluation activity, the aim of this resource is not to suggest that this is the most significant impact area for City and Capital of Culture programmes. As our recent research digest on placemaking found, these large-scale programmes “should not be seen as a quick fix for structural socio-economic problems within post-industrial cities”. 

Of course, some policymakers regard their potential to contribute to city regions’ regeneration and economic development as the primary justification for their continuation. Others, including many cultural practitioners and researchers, argue that the actual (long-term) economic impact of these programmes is secondary to the broader social and cultural impacts.

This resource sits within a wider context of work taking place within the UK on capturing economic impacts of events more generally. This work is exploring how economic impact assessments (EIAs) can be more fully integrated into other significant narratives of value. For those interested in learning more about this, we have suggested a few recent publications from researchers and public bodies in our essential reads: the economic impacts of events. There is also an explanation of key terms related to economic impact in our glossary.

You may also find our other related resources on UK Cities and Capitals of Culture programmes useful reading, including How are UK Cities and Capitals of Culture evaluated? and The people and processes behind UK Cities and Capitals of Culture.

Evaluation Principles

How can the evaluations of UK-based Cities of Culture be viewed through the lens of the Evaluation Principles? These principles were co-created to inform and guide evaluation practice. In this resource, we provide some prompts to consider how the Beneficial principle ‘committed to learning’ relates to the reports we looked at and how it might be used in practice.

Key insights

We have reviewed the main reports and outputs relating to the evaluation of UK-based Cities of Culture from 1990-2021, drawing out the following learning:

• UK-based Cities of Culture programmes can boost visitor economies over the short term. All but one evaluation reported a marked increase in the number of tourist trips to the region during the programme’s year of culture. The exception was Coventry 2021, which was impacted by the Covid-19 pandemic. However, there is no evidence to suggest these impacts are sustained in the mid- to long-term.
  

• In addition to the direct impact on the number of tourists to the cities and their regions, evaluation reports also detailed additional knock-on effects those visits had on the broader economy in terms of “Gross Value Added“. These (estimated) figures were in the tens of millions.

• The evaluations concluded that the UK-based Cities of Culture programmes played a role in improving perceptions of a place. This included boosting the host city’s identity as a visitor destination during and immediately after the year’s programme.

• The breadth of stakeholders for Cities of Culture can provide valuable opportunities for impactful cross-sector collaborations in a host city. As such, UK-based Cities of Culture attract, and depend on, significant private and public investment. This investment is often presented as illustrating economic impact to the wider city region.

• All the evaluations underlined the importance of strategic planning to ensure networks not only sustain but thrive after the year. Safeguarding this legacy can be difficult once investment has ended and the temporary City of Culture delivery teams have disbanded.

• The evidence about the immediate economic impacts is short-term, raising questions about the sustainable legacies of these cultural programmes for those living and working in the area, including those working in the arts and cultural sectors.

• Cultural practitioners reported broadly positive perceptions of the programmes’ impacts on the sector. However, there were significant concerns about the long-term sustainability of these impacts and the negative impacts they could have on future funding levels and infrastructure. Relatedly, the UK-based Cities of Culture did not significantly increase arts and cultural participation after the programmes finished.

• Economic impact assessment (EIA), standardised monetisation and valuation techniques are crucial in capturing the impact of large cultural programmes and events, and recognised methods have established traction in public policy.

• However, economic impact should not be used as a primary justification for holding these events – not least because the evidence is unreliable, especially over the longer term. We now need more robust longitudinal research to better understand the impact on local economies.

• Cultural events and festivals can have a variety of impacts on a place, both good and bad. It is important to look at these in the round. Any economic impact assessment needs to consider how it interacts with environmental, cultural, educational, social and health impacts within a host city. It also needs to consider how it shapes and is shaped by a place’s workforce and tourism economy.

• Similarly, the economic impact of cultural mega-events needs to be placed in a wider political and economic context. This is highlighted by the impact of Covid-19 on Coventry 2021 and the 2008 financial crash on Liverpool’s year as European Capital of Culture. With this in mind, it is important not to see Cities of Culture as an easy and quick way to ‘level-up’ post-industrial cities with deep-rooted structural problems and inequalities.

Economic impact of UK-based Cities of Culture

“[T]here are a number of relatively standardised approaches that seek to understand fully the ”additionality” of an intervention or investment – in other words, the scale and scope of the economic difference that the intervention actually makes to an economy.” (Innovations in Economic Impact Assessment report, p.5)

The economic impact of UK-based Cities of Culture is articulated and measured in these key areas:

• Impacts on the visitor economy: how the programme boosts tourism
• Impacts on the business economy: how the programme creates jobs and employment opportunities, and its impact on local business turnover
• Impacts on the cultural economy: how the programme attracts financial investment for cultural assets and development, creates partnerships, and further impacts broader policy areas such as education or health

Each of these areas is explored in more detail in this resource.

Visitor economy: tourism

The impact of UK-based Cities of Culture on tourism within the local region was a particularly prominent narrative within evaluation reports. It was often grouped together with broader economic impacts such as job creation and added investment (see e.g., Liverpool 2008 impact report; Derry-Londonderry 2013 final report).

Tourist visits

One of the most prominent indicators of a boost to tourism was an increase in tourist visits to the region. All evaluations reported a marked increase in the number of tourist trips to the region during the programme’s year of culture. The exception was Coventry 2021, which reported a sharp decline in tourism due to the Covid-19 pandemic and subsequent lockdowns (impact evaluation, p.9).

• Glasgow’s monitoring report detailed how Glasgow 1990 “expanded the region’s tourism base” with an increase in tourist trips, particularly in the “short-breaks market”. It also found a large proportion of domestic tourists indicated “an intention to return to Glasgow” (monitoring report, p.iii).
• Derry-Londonderry, too, reported a “significant increase in tourism to the city” in 2013. The report claimed that the number of visitor trips was “up by over one-third in 2014 compared with 2011” (final report, p.6).
• Liverpool reported a 50% rise in visitor figures to its seven largest attractions since 2004, “peaking at 5.5 million people in 2008”. An estimated 27.7 million total visits to Liverpool represent a 34% rise from previous years (see graph below). This was higher than levels elsewhere in the region. The impact was particularly pronounced for day visits to Merseyside, which rose by 20% between 2007 and 2008 (impact report, p.2).

“Liverpool and Merseyside Tourism“ (Liverpool 2008 impact report, p. 27)

Often, supplied figures on tourism were collated in collaboration with tourist agencies in the region. They used several metrics including:

• hotel room and bed-space occupancy rates;
• visitors to major attractions and events;
• enquiries at Tourist Information Centres;
• visits to websites detailing the cities’ tourist offer and City of Culture programme.

Additional impacts of tourism on the local economy

In addition to the direct impact on the number of tourists and tourist visits to the cities and their regions, evaluation reports also detailed additional knock-on effects those visits had on the broader economy. Reports included additional visitor spend or “Gross Value Added” (GVA) to the local economy. These (estimated) figures were in the tens of millions. For example, Hull projected an increased GVA of £11 million, drawing from financial data around employment, supply chain spend and grants given (preliminary outcomes report, p.201). Coventry estimated an £ 11.6 million net additional GVA arising from expenditure in Coventry and Warwickshire (impact evaluation, p.8). 

Perceptions of place as a tourist destination

Another dominant narrative of the evaluations of the UK-based Cities of Culture was the impact on perceptions of the city. The views of both visitors and non-residents more generally were captured through local, regional and national population surveys. These surveys were able to capture data from large sample sizes and compare metrics before, during and after programmes and in subsequent years. You can read more about the methodology used here. 

• Glasgow 1990 reported a 15 percentage point increase in the belief that Glasgow was “changing for the better” by those living in London and the South East (monitoring report, p.iv).
• Derry-Londonderry 2013 reported that “external perceptions of the city by potential visitors have experienced a significant positive shift” in relation to its image as a tourist destination (final report, p.6).
• Liverpool found that, from 2005 to 2008, the “overall positive impressions of Liverpool increased amongst the UK population (from 53% to 60%), while negative views dropped from 20% to 14%” (impact report, p.ii).
• Hull reported that 3 in 4 visitors to Hull in 2017 said that the City of Culture programme had “changed their perception of the city for the better”, with 61% of visitors saying they were “likely” or “very likely” to return (preliminary outcomes report, p.9). They also noted that 46% of those surveyed across the UK suggested that Hull 2017 had positively changed their perceptions of Hull (p.9).

Many of the evaluations incorporated media analysis into their reports. They analysed the nature of the content (i.e. if it is positive or negative, broad themes), and the amount of coverage or Equivalent Advertising Value (EAV) in newspapers, online and broadcast media given to the Cities of Culture.

• Liverpool 08’s analysis of media coverage focused in particular on a shift of stories to more content on “current cultural offer and economic change”, resulting in “a much more nuanced view of the city”. The report also noted the ”balanced proportion of positive, negative and neutral coverage” (impact report, p.ii). As well as a shift in tone and content, the report also highlighted an increase of more than double of national and local media coverage on Liverpool’s cultural attractions in 2008 (p.ii). (You can find a more detailed overview of the media impact analysis methodology used by Liverpool 08 here).  
• Hull’s evaluation of media activity surrounding Hull 2017, summarised in the infographic below, formed part of the report’s “placemaking” theme.

Infographic “4.2 Key Findings on Outcomes for Place Making” (Hull 2017 main evaluation report, p.36).

In addition to analysing the media coverage, the report also pulled together a more “thematic” analysis of typical media narratives emerging from national print media between 2013 and 2018 which included: “the wounded city; the city unfairly represented as a ‘crap town’; the city at the end of the M62; the Brexit paradox; Hull’s world class culture; the city’s rebirth; Hull and the Cities of Culture movement; the revenge of the North; the ‘numbers’ of Hull UK City of Culture 2017” (main evaluation report, p.36).

Local economy: employment

“Community, local and regional events have a great deal of ‘placemaking’ potential in that they help create communities where people want to live and work, particularly if they become regular events.” How Can Events Help Build Connected Happy and Thriving Communities? Report by Spirit of 2012, 2022, p.35.

Job creation was one of the economic impacts described in evaluation reports. This included new roles directly relating to cultural delivery (e.g. Culture Companies or evaluation teams) or jobs in local businesses to deal with increased footfall during the year of culture. For example, 225 additional one year full-time jobs were created “from off-site visitor and organisational expenditure in Coventry and Warwickshire” due to Coventry 2021 (impact evaluation, p.115).

Employment levels

Beyond jobs created to directly deliver activity for the City or Capital of Culture, most of the reports included details of levels of employment more broadly. This was presumably to position the Year of Culture as part of a general upward trend of productivity and growth. For example:

• By the end of 2008, the Liverpool 08 impact report stated that there were 1,683 creative industry enterprises in Liverpool employing 11,000 people, representing an 8% growth in the number of enterprises since 2004 (p.ii).
• The Derry-Londonderry monitoring report estimated that between 300 and 400 jobs were directly created during 2013 (estimated 200 in the Chamber of Commerce, 60 in local hotel development, 30 in the public sector and 12 in the arts and creative industries) (p.36). 
• Hull reported that one in four businesses took on new staff during the 2017 programme (main evaluation report, p.50). The preliminary outcomes report also highlighted that almost 800 new jobs had been created in the creative and visitor economy since 2013 (p.10).

Like GVA and other monetary valuations of impact, the impact on employment levels in a city region can be used by funders and policymakers across different areas to compare the cost to the public sector and the subsequent return on this (public) investment. For example:

• Glasgow’s monitoring report found that the gross public sector cost per job compared “favourably with estimates in other initiatives” (monitoring report, p.iii)
• More recently, the Coventry 2021 impact evaluation calculated a monetised Benefit-Cost Ratio of 1.0:1 at the national level and 1.1:1 at the local level, illustrating “acceptable” value for money, given the impacts of Covid-19 on visitor attendances (p.60).

Estimated impacts on local business

Economic impact is often measured quantitatively, i.e., through monetary values such as GVA. However, many of the UK-based Cities of Culture measured the perception of (positive) impact alongside estimated impacts on turnover, product development, skills development and capacity. These were collated through interviews or surveys with those working in the wider business sector, as well as the creative industries.

• Although a small sample size, Derry-Londonderry recorded that “three-quarters of businesses surveyed felt that Derry-Londonderry’s status as UK City of Culture in 2013 had a very positive effect on the City and region, while the remaining quarter felt it was positive” (main report, p.60).
• Hull found that “two in five businesses surveyed stated that UK City of Culture motivated them to develop the products and quality of the services they offered to customers”, with “over half” feeling that it had” contributed to increased turnover” (main evaluation report, p.50). In the Partnerships and Development theme in the same report, the key benefits for Hull’s cultural sector were described, including financial support, enhanced industry connections, and the opportunity to develop skills (p.8). Nine in 10 cultural organisations consulted said that Hull 2017 enabled them to try something new. 89% of cultural organisations felt they had gained new skills, and 87% felt that Hull impacted their capacity and capabilities (p.8).
• Liverpool 08’s evaluation was more interested in businesses’ anticipation of “long-term positive impacts for their businesses” of the European City of Culture (impact report, p.ii), and found that members of the city region’s creative industries sector felt that it had improved the profile of the city, the “local morale” of the sector, and its “credibility within the city region” (p.ii). 

Listen to audio: Considering context

In this audio clip, Centre for Cultural Value Associate Director Dr Beatriz Garcia draws our attention to the pressures on evaluators to consider the different narratives of impact for different stakeholders, and the inherently political nature of value.

Download transcript

Cultural economy: investment and infrastructure

“[U]sing local resources and promoting the art scenes and industries of the city is another way to better contribute to the local economic development. A policy which targets distant industries without looking at the already existing resources might not benefit the region in itself.” (The European Capital of Culture: A review of the academic evidence, Creative Industries Policy and Evidence Centre, 2021, p.10).

Financial and capital investment

Unlike the European Capital of Culture, the UK City of Culture competition has no financial award. Rather, the successful hosting city is expected to attract significant monetary investment due to winning the title. The evaluations reported what levels of investment or funding those delivering the Capital or City of Culture programmes attracted. Sources included additional match funding, sponsorship and capital investment (e.g., for new cultural assets or renovations):

• Glasgow 1990 attracted £32.7 million of revenue support for the City of Culture, a large proportion of which was local and regional council support, plus £6.1 million in private investment (monitoring report, p.1).
• While the Derry-Londonderry 2013 final report stated that investment fell short of projected levels due to the economic recession, it nevertheless concluded that the City of Culture Derry-Londonderry 2013 “has been associated with a significant physical regeneration of the city through £160m capital investment” (p.8).
• One of Liverpool 2008’s key messages claimed they attracted the highest amount of sponsorship (£22.3 million cash and in-kind) and earned more income (£4.1 million) than any other European Capital of Culture (impact report, p.ii). 
• Hull 2017 reported that the City of Culture programme enabled the city to attract new funding and accelerate existing plans for major capital projects. They reported the figures to be in the region of £48 million for public realm enhancement and major developments as part of £676m of “new public and private investment in Hull from 2013 to 2019 that can be at least partially attributed to the UK City of Culture” (main evaluation report, p.50).
• Coventry 2021 reported attracting £183.1 million of additional investment to refurbish or renovate key cultural and heritage “assets” such as theatres and arts centres in the region (impact evaluation, p.6).

Cross-sector policy development

A key theme from the evaluations was how the UK-based Cities of Culture provided an opportunity for stakeholders from different policy areas to collaborate more effectively. This included work on cultural strategies specifically, and other policy areas, such as health, housing and education, more broadly.

• On a project level, the final Glasgow 1990 report cites the key impact of the “successful integration of arts into a pattern of social care through social work programme” of Glasgow 1990 (monitoring report, p.iv). 
• Liverpool described how the “extensive partnership across public, private and third sectors”, contributing to “the repositioning of culture as more central to cross-sectoral agendas” “reflected in a new city-wide cultural strategy for 2008 to 2013” (impact report, p.iii)
• The cultural access and participation section of the Derry-Londonderry monitoring report considered the role of the City of Culture programme in community engagement and cohesion. Metrics included monitoring levels of anti-social behaviour, measured by a change in the number of crimes in the centre. The monitoring report explained that while overall crime in the city centre actually rose by 2% during 2013, the proportion of “anti-social behaviour incidents” fell by 3%, and by 24% during the months where City of Culture events took place, compared with previous years (pp. 19-20). The report authors went on to suggest that this could be the result of “the increased police presence during these events” (p.20).  As part of their assessment of the wider costs and benefits of the Derry-Londonderry 2013 programme, the final report also noted the valuable experience of multi-agency collaboration and community engagement (p.60). 
• The Hull 2017 main evaluation report makes some valuable conclusions about the model of UK City of Culture governance and delivery, which they describe as only “only one plank in an integrated City Plan that would boost the city’s profile and its visitor economy, raise its ambitions, and help to catalyse inward investment” (p.69). It questions whether Cities of Culture constitute “a sustainable and transferable model for increasing cultural participation” for the city region (p.69). The report advocates for a future Cultural Strategy that adopts a “partnership and interdisciplinary, cross-sectoral, horizontal ‘cultural planning’ approach, making links between cultural policy and many other areas of public policy. These will range from youth policies and economic development to city marketing, health, education and social policies.” (p.80).
• A critical learning insight that emerged from Coventry 2021’s report explored the development of new partnerships between arts organisations and non-arts organisations. It stated how non-arts organisations felt that arts activities allowed them “to see how cultural engagement could be a tool for achieving their desired outcomes and impacts” (impact evaluation, p.10).

Strategic and sustainable legacy activity

The breadth of stakeholders for UK-based Cities of Culture means that they can provide valuable opportunities for cross-sector collaboration and work on shared policy priorities. Yet, the reports emphasised the importance of sustainable investment and organisational and network development beyond other associated more short-term impacts. 

All evaluations underlined the importance of sustainable and strategic legacy activity at a local level. Still, the temporary delivery organisation model used by UK-based Cities of Culture hinders the potential impact of legacy.

Listen to audio: Strategic and sustainable legacy

In this audio clip, Jonothan Neelands and Mark Scott from Warwick Business School and the Coventry 2021 evaluation team consider the importance of long-term change in event legacy.

Download transcript

Challenges of assessing economic impact

“[T]he kinds of metrics used to establish economic impacts such as GVA (Gross Value Added), job creation including employment within the cultural and creative sectors, number of businesses started and amount of inward investment into a city are themselves constructs of research processes and modes of classification that are contested and contingent… Even when causality can be claimed, it is not clear whether increases or decreases in any of such metrics are unproblematically “good things” over the long term.“ The uses and misuses of the evaluation of cities and capitals of culture, Arts & the Market, by Oancă et al, 2024, p.10

The differing contexts in which these programmes and their evaluations took place can make comparison between the UK-based Cities of Culture difficult. For example, Liverpool’s evaluation discussed the recession, whereas Coventry’s evaluation had to place its key findings within the context of the Covid-19 pandemic. The evaluation reports also vary their focus, depending on which framework was used and which key performance indicators (KPIs) were targeted. 

Our resource on How Cities of Culture are evaluated explores in detail how establishing a baseline, or fixed starting point, for data and metrics is an essential methodological tool to demonstrate impact across all areas of evaluations. Indeed, recent research has identified manifold benefits and opportunities of carrying out this work for those cities who are not successful in winning the City or Capital of Culture title, such as supporting communities engaged with and connections created through the process, and developing regional cultural strategies (Ashton et al., 2023).

Despite the documented benefits of addressing gaps in data, the lack of longitudinal data related to outcomes from Cities and Capitals of Culture more generally, and economic impact more specifically, remains an issue. You can read more about the challenges of longitudinal research design in relation to UK-based Cities of Culture on p.22 of our research digest on lifelong engagement. 

Specific statistical research techniques can help demonstrate causality, showing how one thing caused another, or establish a counterfactual, which aims to compare outcomes with what the economy was doing anyway. While statistically robust, these techniques are challenging to apply to the real world. 

As the Liverpool evaluation report pointed out, the high levels of investment secured for Liverpool European Capital of Culture 2008 resulted from the existing strong networks within the city’s cultural sector, such as the Liverpool Arts Regeneration Consortium and the previously established Arts and Culture Network. In this instance, it would be impossible to confidently declare that the investment is caused by the Capital of Culture title. It is also difficult to say whether Liverpool would have been awarded the title had these strategic cultural networks not already been in place.

Like any type of value, economic value cannot be measured reliably without taking into account the wider context. This resource has shown, for example, how the recession impacted investment levels for Derry-Londonderry 2013 or how the COVID-19 pandemic impacted expected tourism for Coventry 2021. 

The Innovations in Economic Impact Assessment report points to the narrowness of the traditional economic assessment model commonly used to measure economic impact. The report calls for more incorporation of other related policy domains, notably health and environmental outcomes, as well as how cultural engagement interacts with economic and social variables (p.7). Highlighting emerging trends in measuring economic impact from the last decade, namely the incorporation of social value, wellbeing, natural capital and heritage capital frameworks, the report explores in more detail the “extended” economic impact assessment (EIA) that AMION Consulting carried out for Coventry City of Culture 2021, which included a Social Cost Benefit Analysis. This example is just one way of  the importance of a more holistic approach when conducting economic impact assessments. 

It is also crucial to consider that programme impacts might not always be positive. For example, Coventry 2021’s impact evaluation claimed a significant impact and expenditure on the cultural and creative industries. However by February 2023, the report authors go on to note that the Coventry Trust had gone into administration with debts in excess of £4.25 million (p.16). At the time of publication, Coventry City Council, and a number of local organisations, including arts companies, remained creditors (p.82). While this does not cancel out completely the positive impacts that increased investment has had in many areas, it puts the positive rhetoric of “growth“ and “investment“ into perspective of its long-term impact on the actual economy of the place.

Nevertheless, the significant pressure on the cultural sector to attract greater levels of investment sits alongside the increased use of standardised recommended EIA methods for monetisation and valuation of benefits to the wider public. Despite the challenges they present, these types of assessments play a crucial role in articulating and communicating the value of cultural programmes to society in public policy terms. They represent the reality of hosting a large cultural event or programme which needs large sums of public and private investment. 

Listen to audio: Challenges of economic impact assessments

In this clip, Centre for Cultural Value Associate Director Dr Beatriz Garcia considers the importance of producing balanced evaluations that do not just focus on the final results, such as ‘value for money’ metrics, but are also transparent about the limitations of the methods chosen and data produced. She discusses in particular the economic impact approach of Liverpool European Capital of Culture 2008.

Download transcript

Evaluation Principles in Practice: Committed to Learning

“A UK City of Culture is a massive, unprecedented investment in culture in place. It’s absolutely fundamental that the rest of the UK gains from that. It’s a large chunk of public money, so that comes with its own responsibilities of being able to share any of the learnings for future for the greater good; for better use of public money in the future.” Jonothan Neelands, Professor at Warwick Business School and Coventry 2021 Evaluation Team.

The Evaluation Principles are a set of co-created principles that can be used to inform and guide evaluation practice. 

While influenced by the requirements of our funders and stakeholders, our evaluation activity must also address our own needs. It is crucial to prioritise meaningful learning and reflection and centre the experiences, wants, values, and viewpoints of the people at the heart of the activity. By doing so, we can strive for positive and actionable change rather than mere justification and advocacy.

As you consider some of the impacts included in this resource, or those contained within the evaluation reports themselves, you might find it useful to think about the following questions:

• How did the UK-based Cities of Culture capture baseline data to make meaningful comparisons?
• How did their evaluations enable them to challenge the purpose and aims of the programmes?
• How did they incorporate previous learning, existing data and research into their evaluations?
• How did they use different methods to explore the broader context of the programmes’ economic impacts?
• How could they use their evaluations to make changes moving forwards, learning from the evaluation insights? 

You can read more about our Evaluation Principles, including being committed to learning, on our website.

References

Glasgow 1990

• Myerscough, J. Monitoring Glasgow 1990 Glasgow City Council, Strathclyde Regional Council and Scottish Enterprise, 1991.

Derry-Londonderry 2013

• Monitoring Report on City of Culture 2013, Ilex, 2014.
• Post Project Evaluation of City of Culture 2013: Final Report, Derry City and Strabane District Council, 2016

Hull 2017

• Cultural Transformations: The Impacts of Hull UK City of Culture 2017 Preliminary Outcomes Evaluation, Culture, Place and Policy Institute, University of Hull, 2018.
• Cultural Transformations: The Impacts of Hull UK City of Culture 2017 Main Evaluation Findings and Reflections, Culture, Place and Policy Institute, University of Hull, Nov 2019 and revisited April 2021.

Liverpool 2008

• Garcia, B., Melville, R. and T. Cox. ‘Liverpool’s Arts Sector – Sustainability and Experience: how artists and arts organisations engaged with the Liverpool European Capital of Culture’, Impacts 08, 2009.
• Garcia, B. Melville, R. Cox, T. McEwan, S. Methodology for Measuring the Economic Impact of Visits Influenced by the Liverpool European Capital of Culture, Impacts 08 and England’s Northwest Research Service, 2009
• Garcia, B. Melville, R. Cox, T. Creating an impact: Liverpool’s experience as European Capital of Culture, Impacts 08, 2010.
• Garcia, B. Melville, R. Cox, T. The Economic Impact of Visits Influenced by the Liverpool European Capital of Culture in 2008, Impacts 08, 2010.
• Garcia, B. Media Impact Assessment (part II): Evolving press and broadcast narratives on Liverpool from 1996 to 2009, Impacts 08, 2010.

Coventry 2021

• Coventry UK City of Culture 2021 Impact Evaluation, The University of Warwick, Coventry University and Coventry City Council, 2023.
• Coventry City of Culture 2021 Economic Impact Assessment, AMION Consulting, 2023.

Other reports cited in this resource

More details and a summary of the reports listed below are explored in more detail in our essential reads: economic impact of events.

• Finch, A. et al. How Can Events Help Build Connected, Happy and Thriving Communities? Final Report 4, A Spirit of 2012 Inquiry, 2022. 
• Garcia, B. & T. Cox. European Capitals of Culture: Success and Long-term Effects Study, 2013.
• Henry, N. et al. Innovations in Economic Impact Assessment and Future Directions, 2022. 
• Nanda, S. The Economic impact of Sporting, Cultural, Commemorative and Community Events: A report by the Institute of Public Policy Research (IPPR), 2022.
• Neelands, J. et al. 2024 Creating the golden thread: an ambition for major events in the UK, Warwick Business School and Spirit of 2012.
• Nimrod, O. et al. The European Capital of Culture: A review of the academic evidence. Creative Industries Policy and Evidence Centre, 2021.
• Oancă, A. et al. Evaluating Cities of Culture, Arts & the Market special issue Part 1 2023, Part 2 2024.

All links are correct at time of publishing. If you spot a broken link, please let us know.

Glossary

• Advertising Value Equivalency (AVE) – A calculation that estimates the value of media coverage by comparing it to the cost of an advertisement of similar size and significance. It is often used to provide a monetary figure to the value of communications, press and PR activity. 
• Audience development (see also developing audiences) – A process used to describe a range of activity undertaken to meet the needs of both existing and potential audiences, visitors and participations. For example, this might include broadening the diversity of who organisations might engage or increasing the number of audience members for a particular programme.
• Benefit-Cost Ratio – The Coventry 2021 Economic Impact Assessment defines Benefit-Cost Ratios as a representation of “how much benefit in £s could be supported by £1 of public investment“ (p.60). This technique is commonly used in national policy to appraise value for money across governmental departments, and is the recommended metric in the HM Treasury’s Green Book.
• Causality – The relationship between two variables, where it can be demonstrated whether one variable impacts another directly. This term is not to be confused with correlation, where the two variables might be related in some way, but it is not clear which impacts the other directly.
• Counterfactual – A term used to compare the observed result of a particular intervention, or event, against what would happen if that intervention, or event, had not occurred.
• Developing audiences –  (see audience development)
• Equivalent Advertising Value (EAV) – (see Advertising Value Equivalent (AVE)
• Economic Impact Assessment (EIA)– A type of study that focuses on the economic impacts of a particular programme or event. As described within the Innovations in Economic Impact Assessment report, it is considered to be a “long-run mainstream staple for assessing the benefits to society of a policy intervention“ (Innovations in Economic Impact Assessment, p.4), used by UK City of Cultures and other initiatives.
• Gross Value Added (GVA)– measure of the contribution made to the economy of a particular product, service, or (in this case) the UK City or Capital of Culture programme.
• Impact evaluation – These typically take place at the end of a project or programme to provide insight into how and whether the activity has met its main objectives. These evaluations typically focus on the impacts of activity on participants and outcomes for stakeholders, such as funders.
• Narratives of value – A term used in this resource to delineate the different ways in which an evaluation might communicate and articulate different types of value (e.g. social, or economic value).
• Percentage point increase – This is used to describe the difference between percentages. For example, a 20% increase to 30% represents a 10 percentage point increase, but a 50% total increase.
• Place branding – An activity that is focused on the development of a brand for a particular place. While it could be understood as a form of placemaking, it has a stronger focus on tourism.
• Placemaking – This term has been defined in many ways and from multiple perspectives by different stakeholder groups. It recognises that places are not neutral or fixed entities, but are made through relationships between people, environments, cultures, traditions and histories. Our research digest on placemaking outlines some of the emerging positions.
• Process evaluation – These typically occur before or during the timeline of a project with the aim of feeding back insight to delivery teams and/or into delivery itself.
• Return on Investment (ROI)– A way of measuring the “worth“ of an investment made. It is often calculated by dividing the total profit or loss of an event or programme by its cost to determine its profit. It can also be adapted to include calculations of non-monetary value, such as social return on investment.
• Qualitative research – A type of research that aims to provide a deeper understanding into the ‘how’ and ‘why’ of a particular phenomenon, within its original setting or context. This can often mean generating and analysing non-numerical data that’s rich in detail, typically using methods such as in-depth interviews, focus groups or observations.  
• Quantitative research – A type of research that aims to find correlations or test hypotheses about a particular phenomenon by describing, predicting or controlling for particular variables of interest, then giving it a numerical value. By doing so, the researcher can test the causal relationships between variables, make predictions and generalise results to wider populations through statistical analysis.
• Social Cost-Benefit Analysis – An example of this type of analysis was carried out by the Coventry 2021 evaluation team. It aimed to identify monetisable costs and benefits (such as those described in this resource such as visitor expenditure or financial investment) and quantify non-monetisable costs and benefits and detail unquantifiable costs and benefits (cultural and creative sector development, impacts on wellbeing etc.) In total, the report found that while £418,863 was invested into the programme, a total of £1,365,566 in social value was returned (impact evaluation, p.7).

Related resources

The resource ‘How are UK Capitals and Cities of Culture evaluated?’ provides more on specialist methodologies used to measure economic impact.

Our essential reads: economic impacts of events resource explores some of recent work that has been conducted on the impact of events more broadly, such as sporting or cultural events. 

Our placemaking research digest provides an overview more broadly of how Cities and Capital of Culture programme fits within broader practice working with communities and place networks.

Our lifelong cultural engagement digest has a section on the challenges with measuring longitudinal impact of Cities of Culture as identified in the academic and sector literature.

You may also like

How are UK Cities and Capitals of Culture evaluated?

In this resource we look at how UK Cities and Capitals of Culture have approached their evaluations. Which methods and frameworks did they use, what was successful and where were the challenges? While the focus is on large-scale events, this resource will be helpful for anyone who has an interest in measuring the impact of cultural projects, big and small.

 

People and processes: who’s behind evaluations of UK Cities and Capitals of Culture?

At the start of any evaluation project it’s important to recognise the skills and teams you will need. In this rapid research review, we take a close look at who is involved in the complex, large-scale evaluations of UK Cities and Capitals of Culture.

How were teams structured, who did they collaborate with and how did they approach the challenge of objectivity whilst balancing the needs of multiple stakeholders?

 

Except where noted and excluding company and organisation logos this work is shared under a Creative Commons Attribution 4.0 (CC BY 4.0) Licence

and