Government of Canada Standards on APIs
Application Programming Interfaces (APIs) are foundational to a modern digital ecosystem. These standards govern how APIs are to be developed across the Government of Canada (GC) to better support integrated digital processes across departments and agencies.
- Why APIs? – APIs provide an efficient and controlled way to make data accessible to other systems. This promotes reuse and sharing of data within the Government of Canada and with the Canadian public. Making Government of Canada services and data available through APIs, also promotes a digital economy where private industry can better integrate their services with and alongside government.
- How do I build APIs? – This document provides the baseline standards and guidance on how APIs should be developed. We leveraged established expertise from the UK Government, 18F within the Government of the United States, Government of New Zealand, and the Australian Government to build these standards. This is a rapidly developing market and we strongly encourage leveraging well-established industry expertise and resources to ensure the latest best practices are being followed. We reference several throughout this document.
- Should I expose everything as APIs? – APIs are not the silver-bullet of integration. We recognize the need for many integration styles ranging from messaging to bulk file. Use APIs appropriately to support real-time system interactions and data access.
1. Follow the Government of Canada Digital Standards
APIs must be developed following the Government of Canada’s Digital Standards. Specific attention should be paid to:
- Design with users – Work with the developers who are expected to consume your API to ensure the interface specification meets their needs and addresses any constraints or limitations they may have. APIs should be built against the business requirements consuming systems are intended to support and not the backend data structures they’re accessing.
- Empower staff to deliver better services – Ensure the necessary tooling, training, and processes are in place to support a robust and agile API development and lifecycle management process. Teams should aspire to adopt Continuous Integration and Delivery (CI/CD) and Test Driven Development (TDD) practices supported by automation tools and integrated security testing. This provides the basis for DevOps adoption as maturity improves.
- Work in the open by default – When working with non-sensitive data, build APIs which can be publically consumed and allow reuse.
- Use open standards and solutions – APIs must be exposed using industry accepted open standards, while vendor proprietary protocols and data schemas must be avoided. Leveraging open source tools and frameworks to implement your API is encouraged.
- Iterate and improve frequently – Design APIs with reuse in mind, but don’t anticipate or guess at future requirements. Ensure APIs are designed in such a way as to enable iterations as new requirements and use cases emerge, while providing a reasonable level of backwards compatibility.
2. Build robust RESTful APIs
APIs must follow the RESTful model by default. Representational State Transfer (REST) is effectively the standard for integration with cloud services and is also the standard set by the majority of other Governments with mature API programs. Follow industry best practices when designing and developing your RESTful API. In particular:
- Represent resources as URLs – Uniform Resource Locators (URLs) should represent entities and business objects, not operations on those entities and objects (i.e., avoid verbs inside the URL string).
- Form responses as a JSON object and not an array. Arrays can limit the ability to include metadata about results and limit the APIs ability to add additional top-level keys in the future.
- Avoid unpredictable (i.e., dynamic) object keys such as those derived from data as this adds friction for clients.
- Use consistent grammar case for object keys. Choose underscore or CamelCase and be consistent.
- Don’t overload verbs – Each verb should represent a single operation on a given resource. Avoid using request parameters to pass additional operations. The following are appropriate uses of HyperText Transfer Protocol (HTTP) verbs in the context of a RESTful API:
- GET – retrieve or query a resource
- POST – create a new resource or initiate an action
- PUT – update or replaced an existing resource
- Delete – remove a resource
- Use URIs to identify response data – If data is returned as a part of a response, use Uniform Resource Identifiers (URIs) to uniquely identify the data as a resource so future operations can be performed on it, and iterations can leverage existing resources with minimal rework.
- Use HTTP headers – Content negotiation should be done using the agent-driven approach via HTTP headers:
- ACCEPT and CONTENT-TYPE request headers are mandatory
- AUTHORIZATION header is mandatory for secured APIs
- The API Key must be passed in the header rather than the URI
- API keys/tokens must be securely setup and used appropriately
- The response must contain the CONTENT-TYPE header.
- Use SOAP only if absolutely necessary – It is acceptable to publish Simple Object Access Protocol (SOAP)/Extensible Markup Language (XML) APIs if there are technical constraints on either the provider or consumer sides. All SOAP endpoints should follow SOAP 1.2 specifications and be WS-I Basic Profile 2.0 compliant.
3. Build well-defined and easy to consume message schemas
APIs should respond with message schemas that are easy to understand and consume. The following practices should be applied:
- Use common information models – Leverage industry recognized common information models (e.g., NIEM, HR-JSON, HL7) where possible. If you have to define your own information model, create a model which is technology and platform agnostic rather than simply reusing a vendor proprietary schema. The appropriate use of common information models must follow Government of Canada Data Standards.
- Avoid raw data structure – Exposing raw data structures (e.g., rowsets, table arrays, LDAP DN) from backend systems must be limited to open data, reporting, and statistical APIs only, and strictly prohibited on master data, transactional, or business APIs. APIs should abstract backend physical data representation from the consumer.
- Avoid relational data structures – JSON and XML are hierarchical data structures and are not well suited to representing relational data. Relational data schemas should be flattened based on the perspective of the API consumer.
- Avoid generic/open schemas – The constraints of a data schema must be apparent when reading the schema definition. Generic data structures such as key-value pairs and generic fields are prohibited due to the inability to test API compatibility at the contract level.
- Standardized error codes – Avoid building custom error codes and schemes which require heavy parsing by the consuming system. Conform to HTTP status codes when building RESTful APIs and conform to SOAP 1.2 faults when building SOAP APIs.
- Abstract internal technical details – Responses, including error messages, should abstract technical details which the API consumer has no visibility into. Internal technical errors, thread dumps, and process identifiers etc. should all be kept out of response data.
- Implement stateless interactions – The interaction between API consumer and provider must be stateless. APIs must not expect any concept of session or management of state on the part of the consumer (e.g., passing session IDs). Any interactions where multiple APIs are called in a repeatable sequence to create a singular business interaction should be implemented as a composite API to avoid the burden of consumer-side orchestration.
4. Consume what you build
The best way to validate your API design is to consume it with a production application within your organization.
- Build once for multiple channels – APIs should be designed in such a way that they can be consumed by internal Government of Canada systems, trusted partners, and external parties (i.e., public). Design should allow for different data access profiles to be applied, either to the API or at a proxy layer, without the need to build additional APIs.
- Pilot internally first – Build APIs in parallel with an internal use case which would integrate with the API. Use this internal pilot to validate the API implementation before publishing it for external use.
5. Secure the API
Security must be at the forefront when designing and implementing any API which provides access to protected or privileged data. The following practices must be followed for any API other than those exposing public data (e.g., open data). It is important to note that these practices are to provide a baseline set of security controls. Additional controls (e.g., message-level encryption, mutual authentication, and digital signatures) may be required based on the sensitivity level of the data.
- Enforce secure communications – Never send sensitive data over an insecure or unencrypted connection, and where possible non-sensitive data should also be sent over a secure connection. Enable TLS 1.2 or subsequent versions, in accordance with CSE guidance.
- Design APIs to be resistant to attacks – All APIs should be designed and implemented to be resistant to common API attacks such as buffer overflows and SQL injection. Treat all submitted data as untrusted and validate before processing. Leverage schema and data models for ensuring correct data validation.
- Avoid putting sensitive data in request URLs – Request URL strings can be tracked and compromised even with transport encryption. If a query involves sensitive data elements (e.g., SIN), pass the query parameters as a JSON message payload rather than in the URL request string.
- Protect access to APIs - Implement an access control scheme that protects APIs from being improperly invoked, including unauthorized function and data references. Always authenticate and authorize before any operation to ensure access to APIs are restricted to permitted individuals and/or systems. Use open standards such as OpenID Connect and Open Authorization 2.0 (OAuth 2.0) for RESTful APIs, and Security Assertion Markup Language 2.0 (SAML 2.0) for SOAP APIs. Ensure that the API key/secret is adequately protected. Open data APIs must be secured with an API key to allow for usage tracking and provide the ability to identify and prevent potential malicious use.
- Apply secure token management practices – Token-based authentication is strongly recommended and is mandatory for any APIs published to be consumed across the Government of Canada and/or externally. Use industry standard tokens; do not create custom tokens; and avoid using vendor proprietary token schemes. JSON Web Token (JWT) is required for RESTful API interactions. WS-Security SAML Token Profile is required for SOAP APIs. All access tokens must expire within a reasonable amount of time (i.e., less than 24 hours). In the case of SAML, the assertion expiry must be set to control the validity period of the entire authentication and authorization session.
- Use gateways and proxies instead of whitelists – When exposing APIs to the internet, use a secure gateway layer to provide a security control point instead of simply whitelisting inbound Internet Protocol addresses (IPs). The API Store’s gateway functionality may be used. When consuming external APIs, route flows through a forward (egress) proxy instead of using IP address whitelisting on the outbound firewall.
- Integrate security testing - Automate security testing to validate any new changes to API source code and to ensure robustness of requested changes. Assess the change impact and conduct testing accordingly.
- Audit access to sensitive data – Access to APIs dealing with sensitive and/or personal data must be logged for future audit and reviewed on a regular basis. Access logs must include as a minimum both the system and individual identifiers attempting to access the API along with the timestamp. Periodic audits of API access may be required depending on the nature of the data and its usage.
- Log and monitor for performance and activity – Track usage and monitor for suspicious activity including abnormal access patterns such as after-hours requests, large data requests, etc. Use logging standards (e.g. common event format) and integrate logs centrally. Identify dependencies and monitor for vulnerabilities, especially those for uploaded run-times that work as part of the API. Suspicious events must be sent to the appropriate security operations capability or authority in compliance with Government of Canada Cyber Security policies and Government of Canada Cyber Security Event Management Plan.
6. Use consistent encoding and meta-data
Consistent meta-data and encoding ensures that APIs are interoperable across organizations and helps to maintain data consistency. The following practices should be followed when defining your API:
- Use Unicode for encoding – Unicode Transformation Format-8 (UTF-8) is the standard encoding type for all text and textual representations of data through APIs. It must be adhered to for all APIs published across the GC and externally. Other encodings may be used for single purpose and/or intra-organizational APIs if and only if there are technical limitations to using UTF-8.
- Use consistent datetime format – International Organization for Standardization 8601 (ISO 8601), in Coordinated Universal Time (UTC), is the standard datetime format for data and timestamp fields in APIs published across the GC and externally. The date format is <yyyy-mm-dd> while timestamp format is <yyyy-mm-dd>T<hh:mm:ss>Z. Any other representation of time in the source system must be converted to this format by the API.
- Support official languages – All English or French content returned as data are to be nested with BCP-47 language codes used as keys, specifically “en” and “fr”. External facing APIs must reply with content in the requested language if the backend data support it. APIs must interpret the ACCEPT-LANGUAGE HTTP header and return the appropriate content. If the header is not set, then content in both languages should be returned. In the case of unilingual data and systems, every effort should be made to ensure that the language is appropriately indicated in the response message.
7. Evolve and support the API throughout its lifecycle
APIs will change over time as system and user needs evolve. That change must be properly supported and managed through the following practices:
- Build iteratively – Build and release new versions of the API in an iterative fashion as requirements change and/or new requirements are introduced. Actively solicit feedback from the API consumers to understand whether the API is providing appropriate value and make adjustments in future iterations.
- Version your APIs – Every API must be versioned. Every change to an API, no matter how small, should be indicated by a new version. Follow the v<Major>.<Minor>.<Patch> versioning structure whereby:
- Major = Significant release which is likely to break backwards compatibility
- Minor = Addition of optional attributes or new functionality that is backwards compatible, but should be tested
- Patch = Internal fix which should not impact the schema and/or contract of the API
For example, going from v1.1.0 to v1.1.1 would allow a simple deploy-in-place upgrade as it is a patch, while going from v1.1.0 to v2.0.0 would be a major release and would require the legacy version to be kept while consumers test and migrate to the new version.
The URL must reflect only the major version (e.g., v3). Versions must not passed as a parameter or in the request header to force the API consumer to explicitly identify the version and to avoid defaulting to an incompatible version. Minor and patch versions do not need to be in the URL as they should not break backwards compatibility, but they should be clearly identified in the contract, interface documentation, and response message.
- Respect existing consumer dependencies – Support at least one previous major version (i.e., N-1) to ensure consuming systems have time to migrate to the latest version of the API. Communicate your development roadmap with consuming teams and work with them to understand the impact of any major changes. Set clear deprecation policies and timelines up front so consumers understand how long they have to migrate to each new release before the legacy one is placed offline. Coordinate any necessary testing on all minor and major releases.
- Provide a point of contact – Publish a designated point of contact to any teams consuming your API to the API Store. If the API is published for GC-wide or external use, publish a support email account. A phone number should also be provided for high-criticality APIs.
- Define an SLA up front – Each API should be accompanied with a clearly defined Service Level Agreement (SLA). At a minimum, the SLA should define the following:
- Support hours (e.g., 24/7, 9/5, coast to coast business hours)
- Service availability (e.g., 99%)
- Support response time (e.g., within the hour, 24 hours, best effort)
- Scheduled outages (e.g., nightly, weekly, every 2nd Sunday evening)
- Throughput limit (e.g., 100 requests per second per consumer)
- Message size limit (e.g., <1Mb per request)
8. Measure and publish API benchmarks
API performance should be benchmarked periodically to ensure the performance and capacity continually meets existing and projected business needs. The following actions should be taken:
- Load test the API – Run performance tests against the API to determine the response time and throughput during reasonable load as well as identify performance thresholds beyond which the API becomes unstable. Performance tests should be integrated into the development cycle, preferably through an automated CI/CD pipeline to ensure they’re done on each new release.
- Measure and publish performance benchmarks – Performance summaries (e.g., average response time and associated throughput, max stable throughput) should be published alongside the API contract and SLA. This should be updated on every release.
- Monitor performance – Runtime performance should be monitored and reported on to identify trends and to ensure the API has appropriate amount of capacity to meet usage demand.
- Throttle against the SLA – Throttling mechanisms should be implemented to control throughput against the stated SLA to guard against unexpected spikes in activity. It is better to reject requests that exceed the pre-defined throughput limits than to let the API crash.
9. Use and design APIs sensibly
APIs are not the silver bullet for all integration scenarios. Care must be taken when deciding to implement an API as well as designing what types of queries are allowed to ensure the integration architecture is appropriate and sustainable. The following considerations must be made:
- Preference for query APIs over data sinks – Query-based APIs (pull pattern) are preferred over data sink APIs (push pattern). Having consuming systems query APIs based on specific parameters ensures that only data that’s required in the context of a business process or transaction is being passed. This approach also ensures that data access can be more finely controlled from a security perspective. Data sink APIs promote mass data synchronization and proliferation, which is antithesis to what an API-centric architecture is intended for. Bulk data integration techniques and tools should be used for data synchronization patterns and only when absolutely necessary.
- Restrict wildcard queries – Wildcard queries in APIs can be dangerous from a data performance perspective. If wildcard characters are allowed, ensure there are restrictions on which and how many parameters can have wildcard input to prevent large data query sizes. It is much safer to reject a query with too many wildcards than to timeout on a database query on the backend.
- Segment response data for large queries – APIs exposing large datasets must support some form of data segmentation. The following are some common patterns for pagination along with appropriate use cases:
- page and per_page – Best used to navigate large static datasets (e.g., reference data) where the same set of data is likely to be returned given the same page reference over time.
- offset and limit – Best used for APIs fronting Structured Query Language (SQL)-based backends where the offset represents the data cursor on a given indexed column.
- since and limit – Best used for queries where the consumer is interested in the delta since the last query and the backend data structure is indexed based on time.
- Restrict dynamic or open queries – The ability to inject consumer defined query strings or objects into an API must be limited to open data, reporting, and statistical APIs only, and strictly prohibited on master data, transactional or business APIs. Dynamic and open queries create dangerous attack surfaces for APIs. It’s better to invest more effort in identifying all the valid query use cases and design the API to specifically meet them. GraphQL can be used for statistical, analytics, and reporting purposes, but should not be used to support business transactions. OData should only be used if there are technical limitations by the backend system and only for an organization’s internal APIs.
- Apply special considerations for bulk datasets – There will be scenarios where APIs will need to be involved in making bulk datasets available between systems or to the public. In those scenarios, apply the following considerations:
- small datasets – Smaller datasets should be returned in low overhead formats (e.g., Comma-separated values (CSV) or JSON) rather than XML. The use of compressed file attachments should be avoided, especially when consuming external APIs as they may bypass malicious content scanning mechanisms.
- trigger API – An API may be implemented as a trigger to initiate an out-of-band interface (e.g., managed file transfer) more appropriate for moving large data volumes.
- search and link API – If the dataset is published on file servers already available to the consumer, an API could be implemented to return a link to a specific file based on specific request parameters.
10. Publish and document the API
APIs must be published to be discoverable. How each API is to be consumed must be clearly documented. The documentation must be concise and up to date. The following practices help ensure APIs are documented appropriately without incurring the excess burden of managing companion documentation:
- Publish APIs to the API Store – All APIs should be published to the API Store for the purposes of discovery and lifecycle management. APIs must be appropriately tagged to indicate whether they are for intra-departmental, Government of Canada internal, or public consumption.
- Use OpenAPI for RESTful – OpenAPI is a machine-readable interface specification for RESTful APIs. There are open source tools (e.g., Swagger) which can then generate human-readable documentation from this specification which avoids the need to create and maintain separate documentation.
- Publish well-constructed WSDLs for SOAP – Each SOAP API must be accompanied with a Web Services Description Language (WSDL) contract. The WSDL is a machine-readable specification which allows the API consumer developer to generate the consumer code.
- Publish unit tests and test data – The most effective way to document what an API is supposed to do is to publish the test cases and data used to validate it alongside the API contract. This becomes easier if Test-Driven-Development (TDD) methodology was followed in the development of the API.
- Avoid heavy companion documents – The need for a large separate document explaining each method and attribute is usually an indication that the API is too large, generic, or complex. Consider breaking the API down into smaller components and making the message structure more constrained.
- Date modified: