At-a-glance – Development and formative evaluation of the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS): applications for COVID-19 and beyond

Health Promotion and Chronic Disease Prevention in Canada Journal

| Table of Contents |

Christine Dubiniecki, MScAuthor reference footnote 1; Shannon Gottschall, PhDAuthor reference footnote 2; LCdr Jeff Praught, MDAuthor reference footnote 1

https://doi.org/10.24095/hpcdp.42.3.02
© Her Majesty the Queen in Right of Canada, represented by the Minister of Health and Department of National Defence, 2022

This article has been peer reviewed.

Author references
Correspondence

Christine Dubiniecki, Department of National Defence, 60 Moodie Drive, Ottawa, ON  K1A OK2; Email: Christine.Dubiniecki@forces.gc.ca

Suggested citation

Dubiniecki C, Gottschall S, Praught J. Development and formative evaluation of the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS): applications for COVID-19 and beyond. Health Promot Chronic Dis Prev Can. 2022;42(3):96-9. https://doi.org/10.24095/hpcdp.42.3.02

Abstract

The COVID-19 pandemic highlighted limitations in the current public health data infrastructure, and the need for a comprehensive, real-time, centralized, user-friendly data management system suitable for both disease surveillance and outbreak management. To address these issues, the Canadian Forces Health Services Group developed the web-based Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS). This paper details the development of the CAF SOMS, provides formative evaluation results and includes a discussion of the lessons learned and intent to use the CAF SOMS in future to enhance the CAF’s disease surveillance and outbreak management capability beyond COVID-19.

Keywords: surveillance, COVID-19, SARS-CoV-2, information management, data systems, contact tracing, data infrastructure, public health, case and contact management, outbreak management, pandemic, epidemic, evaluation

Highlights

  • The Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS) was developed to address gaps in information management identified during the COVID-19 pandemic.
  • Integrating a formative evaluation in the phased development and implementation helped to address issues with the system prior to its national roll-out.
  • Lessons learned from its development, implementation and evaluation can inform further refinement and future applications of the CAF SOMS, and potentially of other public health information systems.
  • Increased uptake of the system, integration with electronic records and alignment with the pan-Canadian Health Data Strategy may enhance responses to disease threats and improve CAF health outcomes.

The COVID-19 pandemic and the Canadian Armed Forces

During the early stages of the COVID-19 pandemic, outbreak management relied on traditional public health measures, such as case isolation and contact tracing.Footnote 1Footnote 2 The Canadian Forces Health Services Group (CF H Svcs Gp) operates a separate medical and public health system for Canadian Armed Forces (CAF) personnel. As part of its public health program, the CAF has developed its own contact tracing capability, which is run in coordination with local public health authorities across Canada and abroad.Footnote 3Footnote 4Footnote 5 To meet the growing information management needs associated with case and contact management, the CF H Svcs Gp’s National Contact Tracing Team developed the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS), a centralized, secure, relational database designed to facilitate contact tracing efforts while also providing real-time disease surveillance.

COVID-19 data management: systems and information gaps

Case and contact management for COVID-19 requires multipurpose health data collection tools that can be used for clinical case management, contact tracing, surveillance and epidemiological research. A comprehensive data management system can enable timely and effective follow-up, describe disease transmission dynamics, identify disease clusters and evaluate outbreak management effectiveness.

The COVID-19 pandemic highlighted existing limitations in the Canadian public health data infrastructure that impacted the efficiency and effectiveness of measures taken to control the COVID-19 outbreak.Footnote 6Footnote 7Footnote 8 These limitations include poorly designed data collection tools, non-interoperability of systems across jurisdictions, inadequate data sharing policies and regulations, nonintegration with electronic medical records and laboratory data, and nonstandardized data. To overcome these difficulties, several provinces, municipalities and the CAF have separately developed COVID-19 case and contact management information technology solutions.Footnote 6 

During the early phases of the pandemic, an urgent need for a comprehensive data management tool specific to CAF contact tracing was identified. The main goal was to replace spreadsheets that were cumbersome to complete, file, track and collate. These issues resulted in poor data quality and limited real-time CAF COVID-19 situational awareness beyond case monitoring. Also, it was recognized that this system should collect additional demographic, clinical and risk factor information to improve understanding of disease trends among population subgroups. To overcome these limitations, the National Contact Tracing Team worked with the Health Informatics team to design a centralized, easy-to-use, secure, relational database that linked case and contact data, allowed for multiple follow-up entries, improved data quality and could streamline data collection for current and future CAF disease surveillance, beyond COVID-19.

Development and formative evaluation of the CAF SOMS

The CAF SOMS was developed, evaluated and implemented in three stages, the last of which is ongoing (Figure 1).

Figure 1. Development and formative evaluation of the CAF SOMS
Figure 1. Text version below.
Figure 1 - Text description
  • System development
    • Definition of business requirements
    • Definition of variables
    • Validations
    • Definition of role permissions
    • Design of forms
    • Refinement of functionality
  • Formative evaluation
    • 3 phases of testing and piloting integrated with the CAF SOMS development process
    • Feedback used to iteratively refine the system throughout the process
  • Implementation, refinement, and expansion
    • CAF-wide roll-out
    • Further evaluation and refinement
    • Added functionality (case follow-up form)
    • Expanded use for general CAF disease surveillance and outbreak management (additional pathogens)
    • Streamlining and integration with other CAF information management systems

Abbreviation: CAF SOMS, Canadian Armed Forces Surveillance and Outbreak Management System.

The design of the CAF SOMS included four separate data entry forms: (1) case details (e.g. demographic and contact information, symptoms, comorbidities, risk factors or exposures, isolation period); (2) contact elicitation (e.g. demographic and contact information, diagnostic status, date of last exposure, risk category); (3) contact notification (e.g. symptoms if applicable, self-isolation period, supports required if applicable, contact status); and (4) contact follow-up (e.g. symptoms if applicable, contact status, supports required if applicable, self-isolation period, isolation day). Features were embedded to allow navigation between forms for related cases and contacts using a unique identification code, as well as line listing functionality to subset case-contact relationships.

A formative evaluation was employed to inform further development and implementation of the system through two waves of testing during December 2020 and January 2021, and through the piloting of the system in February and March 2021. The first testing was completed by five members of the National Contact Tracing Team using a prespecified test plan developed by the Health Informatics team to assess the basic functioning of the database. The second wave of testing was completed by four CAF-trained contact tracers at a single CAF base using scenario data to simulate real-world contact tracing. The CAF SOMS was then piloted using real data at two bases to assess the real-world performance of the system. Contact tracers involved in the second wave of testing and the pilot of the system were emailed 22 open-ended survey questions and asked to provide feedback on the CAF SOMS functionality and data flow and the utility of an embedded user guide (Table 1). The results of the testing and piloting were used to refine the system for each subsequent stage of the formative evaluation and implementation.

Table 1. Open-ended survey questions given to contact tracers testing and piloting the CAF SOMS

Does the CAF SOMS contain the correct field and variable options to adequately document case and contact information?

  • Were there any data entry fields you felt were missing?
  • Were there any data entry fields you felt were not necessary? If so, please explain.
  • Were there any missing/incorrect field choices (i.e. missing/incorrect information in drop-down menus)

Are there any variables or fields that are unclear in terms of what information should be inputted?

  • If yes, please identify these variables/fields and explain.

Is it easy to navigate between forms in the CAF SOMS?

Is it easy to enter information directly into the CAF SOMS while conducting case/contact interviews?

  • If no, please explain.

Do you find it easy to search for specific case and contact files?

  • Can you easily find contacts that need to be followed up on using the filtering functions?
  • Are there any filtering functions at the top of case/contact lists you feel would be helpful to improve search capability and to manage daily contact tracing work?

Approximately how long did it take you to enter the data in the database for each scenario, on average?

  • Do you have any suggestions that might make the process of entering the information in the database faster? If yes, please describe.

Did the process of entering data in the database flow well based on how the different screens and data entry fields were linked?

  • If no, please describe the issues that you noticed with the flow of the process.

Do you feel the Excel spreadsheet reports are (1) easy-to-interpret and (2) helpful for contact tracing work?

Did you find the user guide was easy to follow and contained the information you needed to understand how to use the system?

  • Is there any other information that you think should be added to improve the user guide?

Are there additional features that you think would help to make the CAF SOMS more user-friendly?

  • If yes, please describe the features that you would recommend adding.

Abbreviation: CAF SOMS, Canadian Armed Forces Surveillance and Outbreak Management System.

The final phase of the development and evaluation process was a national roll-out of the CAF SOMS in April 2021. A short user feedback survey was also sent to the broader CAF contact tracing community, and analyses are underway to validate the accuracy and completeness of COVID-19 cases and contact data to identify strengths and weaknesses.

Lessons learned

The preliminary results of the formative evaluation are promising and suggest that the CAF SOMS is an effective tool for contact tracing. User perceptions of the system were positive, and some suggested improvements could be acted on immediately during the phased development and implementation of the system (e.g. providing additional response options in drop-down menus, correcting the auto-calculations for self-isolation periods, using additional variables for filtering and searching for records and searching for contacts directly rather than through case files, simplifying the process to close contact records separately from cases). Although some feedback could not be addressed prior to the national roll-out (e.g. include checkboxes to indicate “no change” for contact follow-up, add case follow-up forms), these items may be addressed in future versions of the database, and the changes that were prioritized and carried out were well received.

Overall, a key lesson learned is that newly developed outbreak management information systems must be continually evaluated to proactively ensure they function adequately as new information and shifting priorities emerge. In this regard, our preliminary formative evaluation results suggest that the built-in flexibility of the CAF SOMS design supports the ability to make critical system modifications as required to meet changing outbreak management needs.

COVID-19 and beyond: recommendations and future directions

The COVID-19 pandemic has underscored the need for better disease surveillance and outbreak management systems that allow for timely, informed and agile public health management of disease threats.Footnote 7Footnote 9 As the emergence of COVID-19 has demonstrated, the ability to respond quickly by continually incorporating updated knowledge into systems and processes is crucial for managing a novel and emerging disease that may not be fully understood. Evidence has shown that contact tracing, when performed quickly, can be one of the most effective strategies for disease containment.Footnote 8 This is especially true for COVID-19, since it is estimated that approximately 40%, and possibly up to 70%, of disease transmission occurs when cases are presymptomatic or asymptomatic.Footnote 8Footnote 10

While the CAF SOMS was initially designed to respond to the current public health crisis, the intent is to further scale and develop the system for future pandemic preparedness. Thus, the system was designed to be flexible and adaptable, and to allow for the easy incorporation of updated public health guidance and epidemiology regarding COVID-19, as well as for other diseases. Expansion of the system to capture other diseases is planned for the next phase of system development, as well as the inclusion of other functionality based on feedback from the formative evaluation.

Additionally, there are several challenges that need to be addressed to maximize the uptake and utility of the system moving forward. These include streamlining CAF data collection mechanisms, integrating the system with electronic medical records and improving internal and external data sharing capabilities. To streamline CAF data collection and reduce redundancies, it will be necessary to collaborate with key stakeholders and system end users. Increased promotion of the CAF SOMS, along with training and strong leadership support can accelerate this type of organizational change. Dedicated resources and continuing multilevel health informatics support are key to ensuring sustainability of the system, as are efforts to find solutions to create a seamless and connected data management system that services the entire continuum between clinical care and public health. These efforts include exploring ways to transfer critical public health data, such as case and laboratory reports, between the CAF SOMS and electronic medical records, since the latter system was not designed for public health and epidemiological purposes.

Lastly, since diseases are not constrained by jurisdictional boundaries, options should be explored to develop a consistent approach with external Canadian public health agencies. Enhanced data sharing would be beneficial for minimizing gaps in public health coverage and ensuring continuity in case and contact management. To this end, alignment with the vision of the pan-Canadian Health Data Strategy to create an effective, interoperable, integrated health data ecosystem that is continually optimized could provide additional direction for correcting some of the CAF’s current health data management limitations, and would ensure a consistent and collaborative approach with Canadian public health partners.Footnote 8

Conclusion

The results of the CAF SOMS formative evaluation demonstrated the information system’s potential as a tool to improve the CAF’s disease surveillance and outbreak management capability. The system’s success will depend on effective organizational change, continued leadership support, and adequate resourcing. Further alignment with the vision of the pan-Canadian Health Data Strategy, along with improved data management and collaboration with external public health agencies, can help to increase pandemic preparedness and improve outcomes.

Acknowledgements

The authors would like to acknowledge the contributions of Dr. Maureen Carew, Kristen Simkus, Master Warrant Officer Natalie Deziel and Captain Kristian Cunningham to early discussions in the development of the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS). We would also like to recognize the work of Lucie Bergeron, Cheryl Ward, Paul Jacobs, Guy Lefebvre and Loan Nguyen in the development of the web application, and the contributions of Second-Lieutenant Rachel Dubé and contact tracers in the field to the testing and piloting of the system.

Conflicts of interest

The authors have no conflicts of interest to disclose.

Authors’ contributions and statement

CD and SG: conceptualization, data collection and analysis, methodol­ogy and writing—review and editing; JP: conceptualization, data collection, methodology and writing—review and editing.

The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.

| Table of Contents |

Page details

Date modified: