Evaluating your communication campaigns

By: the Government of Canada Communications Community Office based on Canada School of Public Service courses

Evaluation is an essential part of strategic communications, but it is often overlooked. However, it’s an important step in understanding an activity’s successes and the shortcomings.

Evaluation is an ongoing process over a project’s life cycle:

• The criteria used in the evaluation is established at the beginning of a project and is based on communications objectives and key performance indicators (KPIs).
• Evaluation is used during a communications campaign to course correct tactics and messages that are not reaching their intended audiences.
• At the end of a communications campaign, evaluation will help you determine the project’s success and to glean lessons learned and best practices.

The evaluation criteria, or the KPIs, should always be linked to your tactics and your communications objectives.

KPIs describe what you will be evaluating and should be linked directly to your SMART (specific, measurable, achievable, relevant, and timely) objectives. In essence, ask yourself: “What do the results need to tell you?” and “What outcomes do you want to see?” Try to have indicators related to the following:

• Reach (did you reach your audience?): This could include indicators related to the products you put out for your audiences, such as newsletter readership, social media impressions, and event attendance. These indicators show that you were able to reach your audience using the tactics you selected.
• Reaction (were they receptive?): This could include KPIs related to engagement, sentiment, feedback, comments received, and positive or negative reaction.
• Results (did you achieve the desired outcome?): This could include KPIs related to changes in knowledge, opinions, behaviours, or desired actions, such as applications, subscriptions, changing habits, and changing opinions. The desired result must be directly tied to program objectives.

Methods

Tools or methods used to evaluate communications objectives can vary, depending on the scope of the plan and the communications tactics. For each objective, you will need to develop KPIs to determine if the identified results are being achieved. Indicators may be qualitative or quantitative and should indicate that change is taking place. Potential indicators and associated methods include the following:

• Reach:
  • Readership/views
  • Web views through the Page analytics tool
  • Video views
  • Social media impressions
  • UTM codes
  • Attendance
• Reaction:
  • Media sentiment
  • Social media sentiment
  • Social media engagement
  • Audience participation or questions
  • Media requests, public inquiries, and client service feedback
  • Click-throughs
• Results:
  • Public opinion research (attitude/behaviour changes)
  • Application rates
  • Web visits (for an ad campaign driving people to a vanity URL)

Effectiveness

Although communicators are responsible for evaluating the effectiveness of their work, comprehensive evaluations can be time consuming and resource intensive. It is possible to do a simple tactical evaluation summary for lower-priority initiatives (or no evaluation for non-priority initiatives) in order to balance the need for performance information against available resources. Start with the information that is readily available and build from there.  

Key planning tips

• Recommending the right products and activities should not be based on hunches or best guesses.
• Consult with your team or specialized communicators and gather evidence of best practices, what has worked and what has not (lessons learned) and why.
• Products and activities will in large part determine the success of your plan and require a methodical and reflective approach.
• Factor evaluation into your communications plans.

Key evaluation tips

• Remember that you are evaluating communications objectives based on various levels of measurement (reach, reaction, and results).
• The criteria used in evaluations should specifically reflect these levels:
  • Were the target audiences reached?
  • Did the audience receive and understand the messages and information?
  • Did it have the intended results?

References

1. T720 Evaluating Communications Programs and Services: participant’s manual, version 1.01, Canada School of Public Service, 2004 (Revised December 2011).
2. T712 Understanding and applying strategic communications: participant’s manual, version 1.03, Canada School of Public Service, 2004 (Revised December 2014).
3. I712 Orientation to government communications: participant’s manual, version 1.02, Canada School of Public Service, 2013 (Revised 2013).