Evaluating internal communications

Evaluation is an integral part of a strategic approach to internal communications. This guide aims to provide a best-practice approach to the evaluation of communication.

On this page:

Why you must evaluate

Evaluation allows you to measure the impact of your work and provides the evidence and insight to prove what works and what doesn’t, so enabling us to improve the strategy and delivery of future activity.

It enables you to clearly demonstrate the role of internal communications in staff engagement and your contribution to departmental or organisational objectives and strategic priorities.

Guiding principles for evaluation

You can usee the tool: PROOF:

  • Pragmatic – best available within budget, not best ever
  • Realistic – prove what you can, acknowledge what you can’t
  • Open – record and share as much as possible
  • Objective – be honest and constructive about results so we can learn for the future
  • Fully integrated – it’s an integral part of planning and delivery, not an add-on at the end

Scoping your evaluation

Agree the scope of the evaluation with the key decision-makers at the same time as you agree the communications plan. You need to include:

  • What communication activities you are going to evaluate? Is it a one-off piece of activity such as an all-staff event, the launch of a new intranet or is it multiple activities over a longer time frame, such as a change programme or Civil Service Reform?
  • If you are evaluating the communications for a long-term piece of activity (such as change management), include all elements of the plan (like briefings to senior staff, all-staff emails, events and meetings).
  • What are the key questions the evaluation will need to answer?
  • How often and in what format will reporting be required?
  • Who will be the evaluation project lead and who will need to contribute data to the evaluation?
  • The resources required to deliver the evaluation, including any budget.
  • Start small and build from there – some evaluation is better than none.

Developing an evaluation plan

Your plan should consider the departmental objectives your activity is supporting and the role for communications in support of this objective.

Measure communication with:

  • Communication objective: to ensure staff have access to and understand the current changes in order for them to understand what it means for them and to achieve their acceptance, support and engagement
  • Sub-objective 1: to ensure the change pages on the intranet are kept up to date and staff are signposted to them through a range of channels. For example, measure it with weekly number of hits on the change pages on the intranet
  • Sub-objective 2: to ensure staff understand the information contained with the change pages on the intranet. For example, measure it with telephone survey on a sample 

How to evaluate high-level objectives

As an example, let’s use the Engage for Success enablers, or priorities.

Strategic narrative: visible empowering leadership

Providing a strong strategic narrative about the organisation, where it’s come from and where it’s going.

Examples of metrics that could feed into the objective:

  • hits on intranet news stories (number of downloads)
  • forums (participation rates/effectiveness)
  • team briefings (percentage of staff who understood the briefings)
  • clarity of communications (percentage of staff understanding internal messages)
  • leader visibility (number of staff having face-to-face briefings with senior managers)
  • support from staff for the values and vision of the organisation

Engaging managers

Focus their people and give them scope, treat their people as individuals and coach and stretch their people.

Examples of metrics that could feed into the objective:

  • face to face briefings (percentage of staff who accessed the online briefing pack)
  • team briefings (number of staff having face-to-face briefings with line-managers)

Employee voice

There is employee voice throughout the organisation, for reinforcing and challenging views, between functions and externally. Employees are seen as central to the solution.

Examples of metrics that could feed into the objective:

  • staff ezine / bulletin hits (percentage of staff who accessed the online briefing pack + feedback on the channel)
  • feedback (percentage of staff feeding back + quality of feedback) e.g. communications focus groups

Organisational integrity

There is organisational integrity – the values on the wall are reflected in day-to-day behaviours. There is no ‘say–do’ gap.

Examples of metrics that could feed into the objective:

  • Acting on feedback (percentage of responses to staff feedback + quality of these responses)

You cannot evaluate if clear/SMART communication objectives are not in place. SMART stands for: Specific
Measurable, Attainable, Relevant, Time-bound.

You need to think about any other interventions or external factors that may affect the success of your communications activity, and consider factors that could impact on both the overall departmental objective and more specifically on the communications. For example, the target audiences, including any intermediaries you may be using to pass on your message (like line managers)

Map out how you expect the activity to work. What will staff think, feel or do as a result of the communications and if intermediaries are being used, what role will they play? Mapping will help you set the key performance measures required for the evaluation

To evaluate, use our Evaluation Framework or look at Engage for Success website that provides lots of evidence to show why measurement is important – the proof that engagement delivers improved performance.

Performance measures for internal communications

There are 5 standard performance metrics required for the evaluation of communications. Examples of performance measures for internal communications:

Inputs

Number of manager briefings; the number of staff events; new intranet pages; the number of all staff emails sent out; the number of articles in staff magazine. Include costs (both internal and external costs) of producing and delivering the activity). Recording costs are required to show ROI (return on investment).

Outputs

Number of managers attending briefings; the number of staff attending events; the number of emails opened; readership of staff magazine; the number of unique visits to the intranet pages.

Out-takes

How many staff recognised the messages; what did they think of the event; did it result in a change in attitude? The content of the out-take measures are dependent on the desired response as detailed in your map of how the audience will respond if your activity works.

Intermediate outcomes

How many line managers held team meetings in response to the communications activity; how many staff attended training sessions if required; how many staff responded within deadlines; how many staff commented on blogs, forums, Yammer; how many viewed content on the intranet or downloaded content?

Final outcome

This is related to the departmental objective e.g. change in organisational structure, process or reform. 

If you are using intermediaries such as line managers to reach staff you should consider evaluation in two stages: a) how effectively you engaged the intermediary and b) the impact on the end audience of what the intermediary did. Use the same standard model of performance measures at each stage.

Once you have an agreed set of performance measures for your activity then identify KPIs (key performance indicators) and set targets for these. Ensure your evaluation contains a range of measures from each of the categories.

Develop your evaluation plan as soon as your strategy is in place.

Sourcing the data

Identify how you will source your data before the activity begins, this will ensure all opportunities to collect data are optimised. Adopting a systematic approach to sourcing data will minimise your costs.

A two-step systematic approach to sourcing data:

  1. You may be surprised at the amount of existing data you can use so identify data and evidence that is already available. It may be:
    • gathered from the communications activity
    • gathered by intermediaries such as line managers
    • existing data sources within the organisation or other government departments
  2. Consider the need for any monitoring, research or informal feedback outside of any that already exists.
    • Monitoring of media/social media activity.
    • Use research or feedback to gather data on what staff think, feel or do as a result of your activity.

Pull results from the People Survey but if you need different measures or more frequent measures to help you make recommendations more frequently than once a year, design your own survey using tools such as Survey Monkey. If you run your own, follow best practice in survey design, question wording and sampling.

Consider discussion groups or in-depth interviews for a more qualitative view of what staff think or feel, and include any informal/anecdotal feedback in your evaluation (e.g. comments following a staff event, feedback in team meetings).

Consider whether you can create proxies or assumptions if you can’t get the exact data you need to measure success. Look at sourcing a similar piece of evidence as an alternative (i.e. a proxy measure). If you can’t measure something look for similar activity from the past where this measurement has been included. Using the findings from this will not give you current evidence but could reflect performance.

Be clear on whom, when and in what format data is to be collected before activity begins. Where possible ensure data is collected in a consistent and comparable way so you can later show if your results are statistically significant.

If others are collecting data on your behalf create specific templates for data requirements. Collecting data before activity starts will ensure you have benchmarks in place.

Acknowledge any performance metrics in the evaluation plan that you will not be able to measure.

Analysing and reporting

Return to your evaluation plan, your objectives and expected behaviour from the basis of your analysis. Did the activity work in the way we expected it to work?

  • analyse on a bottom-up basis starting with specific activities or sub-objectives
  • look for trends and links between data sources, can you draw common conclusions?
  • use statistical techniques to understand the relationship between quantitative data sets (ask experts for help if need be)
  • can you calculate return on marketing investment? There is a need to show ROI (Return on Investment) where possible
  • look at what worked, what didn’t and why.
  • don’t cherry-pick the positive results
  • consider validation of initial conclusions with others in your team

Check that the reporting format you agreed at step 1 is still expected

  • provide clear conclusions and recommendations for the future, whatever your reporting format.
  • if using a dashboard, ensure that it contains some conclusions. RAG (red, amber, green) status helps to clearly signpost what is working and what is not.
  • separate fact from opinion and conclusions.
  • acknowledge gaps in the data and the implications.
  • tailor the report for different audiences such as the communications team, staff or senior managers.

Summary

The best practices to get engaging internal communications are:

  • as with all communications, you must be able to measure outcomes. If you can’t measure the activity, don’t do it.
  • your measurement must directly link to your objectives, so know what you are trying to do and then measure whether you have achieved it
  • before commissioning new evaluation measures, find out what you already have, talk to colleagues and look at your annual People Survey results
  • evaluate as regularly as is practical
  • look for simple, brief insights so that it’s easy to understand what the results mean
  • your evaluation must be honest and unbiased; picking only the best results will damage your reputation
  • ensure you embed the learning from your evaluation when developing the strategy for future activity

In this section

Related topics