Defining public opinion research

Contracted and non-contracted projects

Contracted public opinion research is conducted through a service contract by an external organization for the Government of Canada.

Non-contracted public opinion research is carried out using internal resources only, or by another department through a letter of understanding.

Significant non-contracted projects

Departments are responsible for determining whether a non-contracted research project is significant. Significant non-contracted public opinion research projects:

• support legislation, regulations or litigation
• address new government policies, programs, services or initiatives, and changes to those currently in place
• relate to any high-risk issue

Significant non-contracted public opinion research projects are included in a department's annual public opinion research plan, which is submitted to the deputy head for approval.

Behavioural and factual research

Behavioural and factual research activities are normally not considered public opinion research. However, if opinion-based questions are asked, then the public opinion research approval process applies, as would the policy requirements.

Behavioural research is generally defined as how people react or behave in a given situation. Factual research often answers who, what, when or where.

The Public Opinion Research Directorate of Public Services and Procurement Canada can provide advice and help determine whether a project is considered public opinion research.

Experimentation research

Experimental research refers to research that is designed to rigorously measure the impact of a specific intervention. It involves making comparisons across groups: typically, one group is exposed to the intervention, and another is not. Participants are usually randomly allocated to the groups.

Quasi-experimental research also measures the impact of a specific intervention on different groups; however, in quasi-experimental research, participants are assigned to groups based on non-random criteria.

Both experimental and quasi-experimental research measure changes in behaviours, performance and processes. Both types of research may involve observing participant behaviour or collecting factual information. They may also involve web-based testing (for example, A/B testing) that measures task completion to assess the impact of design and content changes on usability. For both experimental and quasi-experimental research, user experience research such as ethnographic research may be conducted as part of the early phases of the experimentation process.

If experimentation uses behavioural and factual questions, then public opinion research requirements do not typically apply. If opinion-based questions are asked, then the public opinion research requirements and approval process may apply.

Usability testing

Usability evaluates the speed, accuracy and confidence with which users can complete tasks using an existing or proposed website, application, system, form or product. Such testing can reveal problems with elements of design or functionality such as the following:

• navigation, layout and structure issues
• users’ ability to complete critical tasks
• impact of the content
• overall user experience

Tests can be moderated or unmoderated and generally include a set of predetermined tasks that participants complete, often while narrating their thought process. Data on completion and error rates, the amount of time spent on a task, and confidence ratings are collected by an observer or remotely through a web-based tool. If the usability testing includes elements of public opinion research, those elements need to comply with public opinion research policy requirements.  

User experience research

User experience research is a key part of designing, building and continually improving services for users. It is conducted to better understand a subset of the population that uses a specific service or program; to examine how they use the service or program, and to confirm that the service or program meets their needs.

User experience research can be conducted by:

• observing users in person or online in the service environment
• asking users fact-based questions about their experience with the service
• asking users to complete tasks and observing their experience

User experience research is different from usability testing because it is about discovering user needs during the design stage rather than testing the service or finished product.

User experience research that involves only using observation techniques and collecting fact-based information is normally not considered public opinion research. However, some methods of conducting user experience research could be considered public opinion research as defined in the Policy on Communications and Federal Identity. If the user experience research includes elements of public opinion research, those elements need to comply with public opinion research policy requirements.

Consultations

Consultations activities occur through active dialogue, whether in person or online, and seek input from informed, interested or impacted groups or individuals on a range of topics such as policies, guidance, programs, services, and legislative and regulatory initiatives.

In consultation activities, ministers and public service employees (or their representatives) may:

• actively participate in the exchange or debate
• share their views, concerns, ideas and proposals
• provide additional information

Participants in consultations can include, for example, the public, clients, experts, public service employees, stakeholders or representatives of organizations. Reports of findings from consultations can include comments attributed to participants. 

See definition of consultations in the Policy on Communications and Federal Identity for more information.

Program evaluation

Program evaluation is the systematic collection and analysis of evidence on the outcomes of programs to make judgments about their relevance and performance, and about alternative ways to deliver them or to achieve the same results.

Public opinion research components of program evaluations are not subject to the Directive on the Management of Communications and Federal Identity when they are part of the Departmental Evaluation Plan.

Public opinion research components of program evaluations outside departmental evaluation plans must comply with public opinion research policy requirements.

Internal audit

Internal audit is an independent and objective assurance and activity designed to add value and improve an organization’s operations. It helps the Government of Canada accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes.

Public opinion research components of internal audit are not subject to the Directive on the Management of Communications and Federal Identity when they are part of the departmental risk-based audit plan.

Public opinion research components of internal audits outside the departmental risk-based audit plans must comply with public opinion research policy requirements.