Results at a glance: Evaluation of the National COVID-19 Exposure Notification App (COVID Alert)
Office of Audit and Evaluation – June 2022
On this page
Program context
On July 31, 2020, the federal government launched the National COVID-19 Exposure Notification app (COVID Alert) as part of Canada’s effort to limit the spread of COVID-19. The app complements manual contact tracing by notifying users when they have been in close contact with another user who tested positive for COVID-19. The app was built for Health Canada by the Canadian Digital Service using the COVID Shield application source code developed by Shopify volunteers, and included a security review by Blackberry.
Along with other tools, the app is intended to limit the spread of COVID-19 through widespread use across Canada. Use of the app is voluntary, and was promoted through focused branding, marketing and communications campaigns. An external advisory council was established to provide guidance and ensure the app operated in the public interest.
Evaluation approach
The evaluation covered Health Canada activities related to the app from April 2020 to July 2021.
The following elements made up the evaluation approach:
- Document review
- Key informant interviews
- Performance data review
- International comparison
What the evaluation found
There is evidence that the design and implementation of the app, as well as any subsequent changes to it, adhered to the FPT privacy principles. Health Canada ensured that privacy principles were upheld by conducting a privacy assessment of the app and updating the privacy assessment or conducting stand-alone assessments for changes or new features, as well as through early and consistent engagement with the Office of the Privacy Commissioner (OPC).
Extensive governance mechanisms were established to support the design and implementation of the app. Health Canada engaged with a wide variety of internal and external stakeholders to share information and seek advice. While these governance mechanisms generally worked well, some challenges remain. In particular, there was a need for greater clarity with respect to roles and responsibilities between Health Canada and the Canadian Digital Service.
The COVID Alert app is one tool that can be used along with other public health measures to help limit the spread of the virus. The evaluation found that the app has had an impact; however, given the novelty of this tool, it was difficult to quantify its effectiveness due to a lack of data and the absence of pre-determined indicators of effectiveness such as benchmarks, targets, and public health impacts. Furthermore, a number of challenges including privacy and trust concerns, technical issues, perceived lack of effectiveness and, most importantly, issues with the distribution of one-time keys have limited the app’s effectiveness and, as a result, it may not have reached its full potential.
The findings from this evaluation resulted in the five lessons learned listed below. These lessons should be considered to ensure the successful development and launch of a similar app by the Government of Canada in the future.
Lessons learned
- Streamline the process so there is less of a burden on app users to achieve the desired outcome.
- When possible, consult with PTs earlier in the process to determine their needs and gain their input into the app’s design to ensure greater buy-in and engagement with all PTs.
- While keeping privacy at the core of app design, explore options to increase the ability to collect data, including personal information, to aid public health measures. Any discussion about the collection of personal information would benefit from early engagement of internal privacy experts, as well as the OPC.
- Establish clear leadership among federal partners, and clearly articulate, communicate and update the roles and responsibilities and agreements of all parties.
- To the extent possible, determine what effectiveness might look like in advance and set targets and goals accordingly. Also, take more steps to demonstrate effectiveness earlier in the process as this may help address the public’s hesitancy to use apps in the future.
Page details
- Date modified: