Developing your communications strategy with evaluation in mind

I first introduced the Barcelona Principles to some colleagues a little while back in a virtual open chat forum I hold regularly to discuss analytics related to communications. My invite said I’d be presenting on the Barcelona Principles. A lot more colleagues attended my session than normal. I was excited, thinking I had gained some more analytics enthusiasts until I found out afterwards that some thought that because of the ‘exotic’ name it must be something completely new and exciting. Spoiler Alert: many of the things we do to measure and evaluate communications in the GoC already fall within the Principles. So let’s dive into them a little more to see what exactly they are, why they matter, and how we can apply them in our daily business.

Where do the Barcelona Principles come from?

I learned about the Barcelona Principles when I was researching how to evaluate communications. There has been some great work done in the GoC towards this front, but I always caught myself asking the question – “Is this the right way to measure what we do?” or “Are we measuring everything we could be?”.

The Barcelona Principles were developed by the International Association for the Measurement and Evaluation of Communication (AMEC) and are a global, industry-wide consensus on measurement and evaluation of communications.

There have been three iterations since 2010 and the latest broadens its relevance to a wider and more diverse range of organizations and roles to reflect that measurement and evaluation best practice is equally essential in government communications, charities, NGOs, and other non-commercial entities.

What are the Barcelona Principles?

There are seven principles in total that can be applied to measurement and evaluation of all communications activities.

1. Setting goals is an absolute prerequisite to communications planning, measurement, and evaluation.

• The founding principle of SMART (specific, measurable, actionable, relevant, and time-bound) goals as a foundation for communications planning has been promoted to an essential prerequisite. It pushes measurement and evaluation as a core component of the planning process, articulating target outcomes and how progress towards these will be assessed.

2. Measurement and evaluation should identify outputs, outcomes, and potential impact.

• Counting outputs is no longer enough. The Principles extend this to consider the longer-term impact of communications strategies by measuring outcomes. According to Ben Levine, Director & Partner, TRUE Global Intelligence, this means thinking about “the channels we are impacting, and change we would like to see through campaigns, events, and activations.”

3. Outcomes and impact should be identified for stakeholders, society, and the organization.

• Business metrics, such as number of media query responses and recruitment numbers are good outputs and outtakes but a more holistic view of performance is required. This is where the outcomes and impacts are important, they allow a more inclusive, broader range of organizations and communications roles that are not necessarily profit-driven.

4. Communication measurement and evaluation should include both qualitative and quantitative analysis.

• A full suite of methods to measure outcomes through qualitative and quantitative analysis will help communicators and clients understand the full impact of their work. Quantifying is no longer enough – we must understand how messages are being received, believed, and interpreted.

5. Advertising Value Equivalency (AVEs) do not represent the value of communication.

• AVEs refer to the cost of buying the space taken up by a particular article. AVEs do not demonstrate the value of our work. It is important that communications measurement and evaluation employs a richer, more nuanced, and multi-faceted approach to understand the impact of communications.

6. Holistic communication measurement and evaluation includes all relevant online and offline channels.

• Social media measurement has been important for several years; however, it has changed over time and the Principles reflect this shift in social communications’ capabilities, opportunities, and influence, such that all relevant online and offline channels should be measured and evaluated equally. The AMEC measurement framework promotes clarity across earned, owned, shared, and paid channels to ensure consistency in approach towards a common goal.

7. Communication measurement and evaluation are rooted in integrity and transparency to drive learning and insights.

• Sound, consistent, and sustained measurement calls for integrity and transparency in recognition of today’s attention to data privacy and stewardship as organizations comply with new regulations. Measurement isn’t simply about data collection and tracking, but about learning from evaluation and applying insight back into communications planning. It recognizes the need to be transparent about the context in which programs are run and being aware of any bias that may exist in the tools, methodologies, and interpretations applied.

Why does it matter to us (GC Communicators)?

I hope you were able to find correlation between the above principles and the work you and your colleagues are already employing in your measurement and evaluation activities. But why does it matter to know and employ these Principles? There are multiple ways to communicate and any combination of methods that could be employed together in one communication strategy or plan. By following the Barcelona Principles when measuring and evaluating our communications we can ensure we are giving a holistic view and providing the most relevant information for all communications activities. Concise and focused information will help your clients understand how communication supports their programs and activities without it being overshadowed by volumes of unnecessary data, and also improve overall communications within your organizations and the GoC by providing relevant, standardized, and measureable results. 

Okay, I gave the what, the why, but what about the how?

The simplest way to employ the Barcelona Principles is to keep them handy and refer to them often – print and post them on a bulletin board, make the infographic your desktop background, or save the link to your favourites. Then every time you start planning communications, whether it is providing advice or creating a plan or strategy, consider how to measure and evaluate your communications, review and incorporate the Barcelona Principles to increase the effectiveness of how you report on communications.

I have seen the Barcelona Principles being used within Public Affairs at the Department of National Defense (DND) on several recent communications plans for priority files not only on outputs and outcomes, but also on potential impacts separated by relevant audiences.

Various teams at DND also employ the use of quantitative and qualitative measurements of their events and products. The content of the social media comments associated with social media posts are just as, if not more important, than the number of engagements, so both are included for analysis and evaluation.

If we all start to employ these seven simple Principles, we’ll see our own communications improve and the ‘exotic’ will become the norm!

For more information on communications evaluations, join the discussion with the Evaluations Community of Practice on GCconnex (Available only on the Government of Canada network).