Defining public opinion research
Public opinion research (POR) is a communications activity that falls under the responsibility of departmental heads of communications. This guidance supports the requirements in the Policy on Communications and Federal Identity, the Directive on the Management of Communications, and the Mandatory Procedures for Public Opinion Research.
On this page
What is public opinion research?
Public opinion research (POR) is the planned, one-way systematic collection of opinion-based information from the public, private individuals, and representatives of businesses or other entities by or for the Government of Canada.
POR uses quantitative or qualitative methods and techniques to provide insight and support decision making. Some collection techniques are as follows:
- online research panels
- virtual or in-person focus groups or mini-groups
- in-depth or intercept interviews
- telephone, mail, online or electronic surveys, and surveys that use other means
POR activities include designing and testing methods to collect data; data collection itself, including sampling, data entry, coding; primary data analysis; and convening and managing online panels. Activities that must be considered POR include:
Contracted and non-contracted projects
Contracted POR is conducted through a service contract by an external organization for the Government of Canada. Logistics such as renting space are not considered components of contracted POR.
Non-contracted POR is carried out using internal resources only, or by another department through a letter of understanding.
Significant non-contracted projects
Departments are responsible for determining whether a non-contracted research project is significant. Significant non-contracted POR projects:
- support legislation, regulations or litigation
- support government or departmental priorities
- address the development of new government policies, programs, services or initiatives
- touch on issues that are of high public interest or sensitivity, or
- relate to any other important or high-risk issue
Significant non-contracted projects are included in a department's annual public opinion research plan, which is submitted to the deputy head for approval.
Behavioural and factual research
Behavioural and factual research activities are normally not considered POR. However, if opinion-based questions are asked, then the POR approval process applies, as would the policy requirements.
Behavioural research is generally defined as how people react or behave in a given situation. Factual research often answers who, what, when or where.
The Public Opinion Research Directorate of Public Services and Procurement Canada can provide advice and help determine whether a project is considered POR.
Experimentation research
Experimentation can include experimental or quasi-experimental research.
Experimental research refers to research that is designed to rigorously measure the impact of a specific intervention. It involves making comparisons across groups: typically, one group is exposed to the intervention and another is not. Participants are usually randomly allocated to the groups.
Quasi-experimental research also measures the impact of a specific intervention on different groups; however, in quasi-experimental research, participants are assigned to groups based on non‑random criteria.
Both experimental and quasi-experimental research measure changes in behaviours, performance and processes. Both types of research may involve observing participant behaviour or collecting factual information. They may also involve web-based testing (for example, A/B testing) that measures task completion to assess the impact of design and content changes on usability. For both experimental and quasi-experimental research, user research such as ethnographic research may be conducted as part of the early phases of the experimentation process.
For a more formal definition of experimentation, see the entry for “experimentation” in the glossary to the MAF (Management Accountability Framework) 2018 to 2019 results management methodology.
If experimentation uses behavioural and factual questions, then public opinion research requirements do not typically apply. If opinion-based questions are asked, then the public opinion research requirements and approval process may apply.
Usability testing
Usability testing is not considered POR. It evaluates the speed, accuracy and confidence with which users can complete tasks using an existing or proposed website, application, system, form or product. Such testing can reveal problems with elements of design or functionality such as the following:
- navigation, layout and structure issues
- users' ability to complete critical tasks
- impact of the content
- overall user experience
Tests can be moderated or unmoderated, and generally include a set of predetermined tasks that participants complete, often while narrating their thought process. Data on completion and error rates, the amount of time spent on a task, and confidence ratings are collected by an observer or remotely through a web-based tool.
User research
User research is a key part of designing, building and continually improving services for users. It is conducted in order to better understand:
- a person who uses a product or service
- their needs, wants and behaviours when they use the product or service
User research can be conducted by:
- observing users in person or online in the service environment
- asking users fact-based questions about their experience with the service
- asking users to complete tasks and observing their experience
This research can occur at various stages of designing a service or product. User research is different from usability testing because it is about discovering user needs during the design stage rather than testing the service or finished product.
User research that involves only using observation techniques and collecting fact-based information is normally not considered public opinion research. However, some methods of conducting user research could be considered public opinion research as defined in the Directive on the Management of Communications. If the user research being conducted includes elements of public opinion research, those elements would need to comply with the directive.
Consultations and public engagement activities
Consultations and public engagement activities are not considered POR. They occur through active dialogue, whether in person or online, and take into account the public's views, concerns, ideas and proposals for the development or assessment of government policies, programs, services and initiatives.
In consultation or public engagement activities, ministers and public service employees (or their representatives):
- actively participate in the exchange or debate
- share their views, concerns, ideas and proposals
- provide additional information
Participants in consultations can include the general public, clients, experts, public service employees, stakeholders and representatives of organizations. Information is generally provided to participants beforehand to frame or guide discussion.
Program evaluation
Program evaluation is not considered POR. It is the systematic collection and analysis of evidence on the outcomes of programs to make judgments about their relevance and performance, and about alternative ways to deliver them or to achieve the same results.
POR components of program evaluations are not subject to the Directive on the Management of Communications when they are part of the Departmental Evaluation Plan.
POR components of program evaluations outside departmental evaluation plans must meet POR policy requirements.
Page details
- Date modified: