Evaluating low/no-cost communications

All communication strategies should utilise the Evaluation Cycle to evaluate the impact of their communications.

This guidance serves as a companion resource to the Evaluation Cycle, helping you apply and adapt the core principles to assess low/no-cost communications.

A downloadable version of the guide will be available soon.

Read the Evaluation Cycle to learn the agile framework to meaningfully evaluate all types of communication activities, and promote continuous learning and innovation while integrating best practices for better impact and future planning.

Evaluation Cycle

Use and adjust the evaluation template as needed to plan and track low/no-cost communication evaluations in a digestible, easily sharable format.

Template for low/no-cost communication evaluation

PPTX, 2MB

Introduction
Key principles
Evaluation tiers – for low/no-cost comms
Six stages of low/no-cost evaluation
Further resources

Introduction

In contrast to paid-for campaigns, low/no-cost communications reach audiences by utilising existing resources within the department’s or organisation’s standard operating budget. These strategies employ communication channels that are often low-cost or free to achieve social media communications, content marketing, webinars, collaborations and partnerships, and earn media or public relations coverage.

They generally aim to raise awareness or change attitudes (rather than directly changing behaviours) but can still significantly impact wider communications, policy and organisational objectives – as standalone strategies and as part of wider comms which include some paid activity.

Key principles

Evaluating low/no-cost comms is just as important as evaluating paid-for campaigns. Using the Evaluation Cycle helps you measure and understand the impact of what you are delivering.

In a nutshell

Tell the story. Don’t just create a function-by-function list of what you did. Evaluation needs to bring together all the activities you have undertaken and provide a narrative of what your impact has been.
Data doesn’t have to be numbers. Low/no-cost comms don’t always come with neat numerical metrics that demonstrate impact. Qualitative performance metrics are just as useful.
Provide context. Data doesn’t have meaning without some form of benchmark to put it into context. Demonstrate your impact on your objectives/outcomes through comparisons with relevant activity.
Remember proportionality. Low/no-cost comms come in different shapes and sizes. Adapt your evaluation approach accordingly, using the evaluation tiers (see below).
Embrace continuous learning. Focus on what you are learning throughout the process about what is working well/less well and feed that into the later activity.

Evaluation tiers – for low/no-cost comms

Low/no-cost communications can vary significantly in complexity and scope and therefore require different Evaluation tiers.

Tier	Focus	Example
BASIC monitoring	Tracking simple metrics to gauge real-time performance and flag potential issues/risks. Comparing metrics to benchmarks or targets. Predominantly output-focussed.	Reporting on the performance of specific posts on social media, and volume of press releases (e.g., likes, shares, comments, media coverage).
ENHANCED monitoring and reporting	The elements of basic monitoring plus presenting key metrics in a dashboard, slide pack or other report to guide ongoing comms optimisation. Monitors outputs and outtakes to help understand audience reaction. Present your analysis using concise reports and visual representations to keep actionable recommendations digestible.	Extracting actionable insights from tracking dashboards of web traffic, social media engagement and media mentions.
COMPREHENSIVE low/no-cost evaluation	The elements of enhanced monitoring plus utilising qualitative and quantitative metrics to tell the story of how outputs and outtakes contributed to outcomes and/or Impact. Utilise the low/no-cost evaluation template for more comprehensive reporting of impact. Consider coordinating with the wider communications teams to examine the combined impact and effectiveness.	Evaluating waves of social media activity, monitoring sentiment/tone of media coverage, and stakeholder engagement to support paid activity.

Evaluation decision tree

This decision tree may help you determine the most appropriate tier – Basic, Enhanced, or Comprehensive – to evaluate your low/no-cost communications.

The key is to evaluate within your means; a targeted, well-executed evaluation is more valuable than an overly ambitious one that stretches resources too thin.

Decision tree illustrating a series of questions to help communicators select the appropriate evaluation tier (Basic, Enhanced and Comprehensive).
It first asks if the communication is part of a wider paid-for campaign. If yes, the path leads to “paid-for campaign evaluation”, which means you should refer to the Evaluation Cycle and collaborate with your campaign team.
If not, the next question asks if it has paid-for elements, such as spending on social media boosting. If yes, you should conduct a comprehensive low/no-cost evaluation.
If not, the next question asks if it is responding to a crisis. If yes, you should conduct a comprehensive low/no-cost evaluation.
If not, the next question asks if it is one-off, such as a single press release. If yes, you should conduct basic monitoring.
If not, the next question asks if you can only track simple metrics. If yes, you should conduct basic monitoring.
If not, the next question asks if you expect any changes in Outtakes, which is about audience awareness, beliefs and feelings. If not, you should conduct basic monitoring.
If yes, the next question asks if you expect any long-term policy or organisational impact. If not, you should conduct enhanced monitoring and reporting.
If yes, the final question asks if you aim to contribute to Outcomes, which means behaviour change. If yes, you should conduct a comprehensive low/no-cost evaluation. If not, you should conduct enhanced monitoring and reporting.

Diagram of the decision tree illustrating a series of questions to help communicators select the appropriate evaluation tier (Basic, Enhanced of Comprehensive).

Your choice between the three tiers will also be determined by the time, resources and technical capability available to you. For example, if you lack enough human or financial resources to gather qualitative metrics, like your audience’s or stakeholders’ sentiments and reactions, you may need to focus on Enhanced monitoring and reporting instead of a Comprehensive evaluation.

Six stages of low/no-cost evaluation

Diagram of the Evaluation Cycle which illustrates evaluation as a recurring process and includes 6 stages, namely Inputs, which is about evidence-based planning; Outputs, which is about audience experience; Outtakes, which concerns audience beliefs or feelings; Outcomes, which is about audience behaviour; Impact which links inputs, outputs, outcomes and ROI; and finally Learning and Innovation, which is about strategic insights.

Diagram of the Evaluation Cycle, which illustrates evaluation as a recurring process and includes six stages.

While campaigns with a large budget gather many metrics to be able to measure impact, low/no-cost evaluations should focus on the most relevant elements of the Evaluation Cycle. This does not mean cutting corners; it means taking a proportional approach to maintaining the most rigorous evaluation standards despite resource and budget constraints.

1. Inputs – evidence-based planning

Must have: OASIS – scoring and evaluation for low/no-cost comms

Evaluation planning begins at the start of the comms planning process. Using OASIS guidelines with a proportional level of detail prepares you to better evaluate your low/no-cost comms throughout the delivery process.

Plan what you will measure and how from the beginning.
Ensure what you plan to measure will enable you to demonstrate whether objectives are met.
Set targets and/or benchmarks (details on how to do this are below).

Must have: Demonstrate impact with context

Collecting benchmark data and setting appropriate targets is crucial for conducting pre-and post-implementation comparisons that help with your evaluation. When setting benchmarks and targets, consider:

Industry performance: What is the expected performance for your particular industry (education, health, trade, transport, etc.), by your chosen metric, on the channel in question? Source published benchmarks from internal or external studies or reports.
Historical performance: What are your own historical performance levels? Compare current metrics against past data to identify trends and improvements over time.
Ambition and progression: How ambitious do you want to be, and what progress do you envisage in future? Set ambitious yet realistic targets to encourage continuous improvements. Be sure to also review your data regularly to adjust what goals you want to reach and devise what strategies can help you achieve them.

When comparing benchmark data with post-implementation results, note any assumptions you have to make if you can’t rule out other external factors, like concurrent events that may have helped or hindered the effectiveness of the communication.

If no benchmark data is available, compare metrics with the targets set out in your OASIS plan.

Should have: Make best use of available tools and collaboration

Utilise no/low-cost tools to gather qualitative, social listening data and understand audience sentiments.
Explore adopting a test-and-learn approach and implement pre-and-post comparison to measure the effectiveness of specific activities and inform improvements.
Foster collaboration with other teams or departments via Connect or other collaboration spaces to tap into skills and insight that apply to your comms strategy and target audience.
Capitalise on existing touchpoints (meetings, email, etc.) to implement regular feedback loops like quick pulse surveys, and check-ins.

Could have: Further considerations

Use A/B testing if you want to understand the impact of different messaging, wording, format and other elements for content creation.
Collaborate with project teams of other concurrent low/no-cost communications and explore whether a joined-up research approach (surveys, focus groups, etc.) can help evaluate multiple low/no-cost initiatives simultaneously.
Consider collaborating with the target audience or stakeholder groups where appropriate, e.g., experimenting and testing messages for stakeholders’ reactions and feedback, or co-designing communication assets through ideation sessions with users/citizens or audiences.

2. Outputs – what was delivered and audience reach

Evaluating your digital comms in real-time

Digital channels offer a cost-effective way to reach audiences while enabling real-time evaluation and quick adaptation. Rapid, real-time analysis is beneficial for informing immediate strategy improvements.

Focus on measurable outputs that directly contribute to your objectives:

Media impact measurement – Where possible, automate the collection of press cuttings so you can count mentions of press releases, media appearances, etc., to estimate overall audience reach. If not, keep a light-touch activity and coverage log to help with manual sentiment analysis.
Leveraging social analytics – Where applicable, track engagement metrics such as likes, shares, comments, hashtag usage and click-throughs on your social media posts, and gauge the spread of your messaging by monitoring the growth of your followership and audience engagement levels.
Utilising web/GOV.UK statistics – Where appropriate, track the user journey from engaging with your post to visiting the related web page or information, for example, through using UTMs (Urchin Tracking Modules).
Include a benchmark for comparison – Remember to always include targets, benchmarks or other data as a comparison to help interpret your data. Your post received 13,000 likes; is that good, average or could it be improved? Consider:
- Did you reach your target or exceed your benchmark?
- How did your comms compare to other similar scenarios?
Make the best use of digital metrics – You can find further guidance on our digital communications page.

Measuring audience reach via non-digital comms

Non-digital communications are also a means for understanding how well you have reached your audience. Methods include:

QR codes and trackable links – Free tools online can generate QR codes for posters or trackable URLs for physical publications that allow you to monitor offline engagement.
Event attendance tracking – Use sign-in sheets or registration forms to gauge event participation and popularity with minimal costs. These can be used in parallel with online event registration tools to gain a full picture of your audience when combined with collecting basic audience demographic information.
Other manual tracking logs – Other than for event attendance, tracking logs can also track offline activities where automatic logging is not feasible, e.g., the distribution of flyers, brochures and other materials, readership figures of physical media like newspapers and magazines, participation in contests and giveaways, etc.

Integrating digital and non-digital monitoring methods will help you comprehensively understand your reach via different media. Refer to the Evaluation Cycle for more suggested measurement metrics.

3. Outtakes – audience beliefs and feelings

You can understand how well your communications are received by examining what your target audience thinks, feels or intends to do as a result of your comms intervention.

Capturing audience sentiments and intentions

To build a picture of your audience’s understanding and retention of key messages, and intention to take action, consider:

Investigating the user journey – Use web/GOV.UK statistics like dwell time, bounce rate, behaviour flow and more. Whether post/webpage views convert to downloads of a related guide/toolkit/or other form of information can build a picture of your target audience’s intentions to take action. Dwell time on a page can also indicate whether your audience has taken the time to understand the message.
Pre-and-post analysis – Conduct short online surveys with social media audiences before and after low/no-cost communications to measure shifts in awareness, attitudes and behavioural intentions. This approach also detects how your messaging is landing, so that any adjustments can be made to the communication strategy.
Leverage qualitative data – Qualitative data is very useful to help understand your audience’s reaction to your comms and whether their feelings or intentions have changed as a result. You can gather qualitative data through:
- Using feedback forms after virtual events.
- Analysing comments, interactions and sentiments on social media posts.
- Gathering feedback from stakeholders and collaborative partners.

These approaches provide insights into the audience’s level of engagement in addition to their understanding. Within the richness of qualitative data, you can also look for indications of perception changes among your audience.

How qualitative data can help tell the story of your comms evaluation

Qualitative approaches (such as feedback, case studies and comments) often reveal a deeper understanding of audience attitudes and intentions, beyond what can be captured through quantitative (numeric data) methods alone.

This is why qualitative data collection can be cost-effective in understanding how your comms have influenced what your audiences think, feel and do.

Make the most of stakeholder feedback:

Segment your stakeholders based on their outlook (e.g., positive, neutral or negative) or other relevant criteria. (See guidance for segmenting your audience for specifically internal communications.)
Tailor your measurement metrics accordingly.
- For supportive stakeholders, track actions like information sharing, quoting of key messages or slogans, or other forms of content redistribution/amplification.
- For potentially critical stakeholders, monitor media outlet notifications, public comments, and other information that is useful to inform any negative coverage mitigation.
Set realistic targets for each audience segment based on your past experiences at similar events and engagements.

Demonstrate impact over time so that you can:

Show the gradual evolution of audience attitudes and beliefs.

Highlight any adjustments to your communication strategy in response to real-time insights and any unforeseen changes in the political or societal landscape.
Indicate how these adjustments may have resulted in better audience sentiments and feedback over time.

Evaluating impact among key audiences

Different members of your audience are likely to hold different opinions and may interpret or respond to your communications in distinct ways. Similarly, you may have several target audiences who will respond to your comms differently. To examine how your messaging lands with different audience segments, consider the following steps:

Track metrics throughout the communication activity to understand how different audience segments are engaging.
Compare responses across different audiences, platforms or stakeholder groups.
Map how different audiences move from outputs to outcomes based on their specific contexts, beliefs and/or needs.

4. Outcomes – audience behaviour

Measuring meaningful actions

All communications, including low/no-cost efforts, should contribute to the overall policy objectives. While it may be challenging to illustrate the direct impact of low/no-cost comms on outcomes (i.e., behaviour change), it is important to highlight any incremental contribution to these overarching objectives.

This involves:

Aligning your low/no-cost communication objectives with overarching policy key performance indicators (KPIs).
Measure specific actions that can indicate progress towards these objectives and, by extension, targets or KPIs. For example, tracking the number of downloads for an advice toolkit or subscribers to a channel can be a cost-effective way to indicate progress towards your objective, when measuring behaviour change is not possible due to budget constraints or infeasibility.
Highlight intermediate behavioural outcomes that signify advancements towards the ultimate behaviour change desired on the policy level.

Focus on COM-B

In cases where you cannot directly capture actual behavioural changes (i.e., outcomes), consider how your communications have helped your audience to overcome behavioural barriers to the behaviour you are trying to enable instead.

You can use the step-by-step Principles of Behaviour Change Communications guide to identify audience barriers. This follows the COM-B model of behaviour, highlighting that for a behaviour to occur, a person needs the right capability, opportunity and motivation to do it.

Once you have identified your audience’s barriers, you can then measure these before and after your communications (e.g. using surveys and qualitative research) to understand whether your activity has reduced these barriers.

For example, in the context of household recycling, it is likely to be impractical to measure changes in household waste recycling behaviour within a limited budget. However, you may be able to demonstrate how comms has helped overcome a key barrier to the behaviour of recycling, for example:

Capability: Have your communications increased your audience’s knowledge of what can be recycled or how to apply for household recycling bins if they do not already have one?
Opportunity: Have you implemented your communications in a way which has reached more of your target audience than previously, for example using new channels, partners or messengers, and thus enabled you to increase awareness of the importance of recycling (a component of capability)?
Motivation: Have you made your audience feel confident about what they need to do? Has sharing the environmental benefits of recycling boosted the audience’s motivation to recycle or changed perception amongst an audience who did not previously consider recycling?

This helps provide evidence of your contribution to creating a more conducive environment for behaviour change. To learn more about how COM-B can be integrated into your evaluation, refer to the Evaluation Cycle.

5. Impact – long-term effects on policy and organisation

Emphasise long-term value

Many communication efforts, especially those aimed at upholding organisational reputation, initiating behaviour change or changing perceptions, may take time to show impact. It is therefore essential to be upfront about the realistic timelines within which your metrics should start showing impact and communicate these clearly with your stakeholders.

Set milestones at which you may highlight incremental progress and celebrate small wins.

Measuring broader, organisational impact

On top of linking to overarching policy KPIs (as mentioned under ‘Outcomes – audience behaviour’)), you should consider:

Cost savings and efficiency: Highlight how using low/no-cost comms reduces expenditure compared to paid-for campaigns.
Sustain/amplify audience engagement: Assess how ongoing low-cost efforts (e.g. newsletters and social media) create a receptive foundation for concurrent or future campaigns.
Media relations: Track media sentiment and feedback to demonstrate how low/no-cost efforts may foster collaborative relationships with the media sector and amplify the organisation’s long-term influence.
Strategic value: Substantiate how low/no-cost comms can act beyond standalone initiatives but reinforce an organisation’s long-term ambitions and reputation.

Measuring broader impact may be challenging for certain types of communications, such as one-off announcements, emergency statements or crisis communications. In these cases, pay attention to the immediate policy or behavioural effect, so that they can be reflected on when managing scenarios in the future.

6. Learning and innovation – strategic insights

With a limited budget and possibly time pressure as well, reporting evaluations of low/no-cost comms requires capturing actionable insights concisely.

Reporting your low/no-cost comms evaluations

We provide a template specifically designed for reporting low/no-cost comms evaluations, which should help you achieve the following:

Keeping it brief: Unlike large paid-for campaigns, be concise and focus on the most critical insights and outcomes.
Highlight cost-effectiveness: If applicable, emphasise how impact was achieved using minimal resources.
Focus on key metrics: Concentrate on only the metrics directly related to your primary communication objectives.
Incorporate qualitative insights: Given the often limited quantitative data available, utilise relevant qualitative feedback and observations to gain a comprehensive understanding of your communication’s impact and effectiveness

Further resources

This guidance outlines strategies that help evaluate low/no-cost communications while reflecting the core values and principles of the Evaluation Cycle.

Read the comprehensive, six-stage Evaluation Cycle to learn more about continuous learning, metrics measurement, audience inclusivity and linkages with policy and organisational objectives.

We also recommend reading our OASIS guide for campaign planning and The Principles of Behaviour Change Communications for more information on COM-B.