Canadian Substance Use Survey (CSUS): technical notes for 2023

Background

The Canadian Substance Use Survey (CSUS) (previously the Canadian Alcohol and Drugs Survey) is conducted every other year by Health Canada (HC). It aims to collect data from people living in Canada to identify the prevalence of and trends in alcohol and drug use. Sociodemographic information are also captured in order to assess use and risks amongst population groups in the Canadian provinces.

CSUS methodology from 2008 to 2019

CSUS has evolved over time. Initially, the Canadian Tobacco Use Monitoring Survey (CTUMS, 1999 - 2012) measured the use of tobacco and the Canadian Alcohol and Drug Use Monitoring Survey (CADUMS, 2008 - 2012) measured the use of alcohol and drugs. CTUMS and CADUMS were merged into the Canadian Tobacco, Alcohol and Drug Survey (CTADS) in 2013. CTADS was conducted every other year from 2013 to 2017. In 2019, HC separated CTADS into two surveys: the Canadian Alcohol and Drugs Survey (CADS), focused on alcohol and drug surveillance, and the Canadian Tobacco and Nicotine Survey (CTNS), focused on tobacco use and vaping. In 2023, CADS was renamed as CSUS following large methodological changes.

2008 – 2012

Following a tender process, the firm Jolicoeur & Associés was chosen to carry out the sampling and data collection for CSUS (then CADUMS). During this time frame, the methodology for contacting respondents remained stable. A two-stage sampling method was used: first households, then the individual respondent. Households were contacted through random sampling from an inventory of all active telephone area codes and exchanges in Canada. The individual respondent was selected based on the birthdates of each household member aged 15+. The selected individual was the person who would celebrate their birthday next. If the selected individual agreed to participate, they completed the survey on the phone via a computer-assisted telephone interview. The sample sizes obtained during these iterations ranged from 10-15,000.

2013 – 2017

In 2013, CSUS (then CTADS) was conducted in collaboration with Statistics Canada, which oversaw the sample selection, data collection, and data processing. A two-stage sampling method was used, first sampling households by random digit dialing (RDD). In the second phase of sampling, one or two individuals were selected, based upon household composition. The random selection of respondents was arranged so that at least one person aged 15 to 19 or 20 to 24 would be selected within a household, if they existed. Two people were selected if more than one of the age groups 15 to 19, 20 to 24, and 25 and over were represented in the household. When two people in the same household were selected, they were always from different age groups. If the selected individual agreed to participate, they completed the survey on the phone via a computer-assisted telephone interview.

In 2015, cell phone numbers were added to the sampling frame for households. In these years, the number of respondents was in the range of 15-16,000.

2019

In 2019, Statistics Canada continued to oversee the sample selection, data collection, and data processing of CSUS (then CADS). In this iteration, two large methodological changes occurred. First, the household sampling strategy was changed from RDD to contact via mail. The second change was a move from computer-assisted telephone interviews to online questionnaires.

22,000 households were randomly sampled from the Dwelling Universe File (DUF), the same sampling frame used for the Labour Force Survey. These households were contacted by mail, which included instructions on how to identify who should complete the survey, using age-order selection. Selected individuals completed the questionnaire online by following a link provided in the letter.

Non-responding households were sent up to 4 reminder letters. If no response was obtained after 4 letters, these households were contacted via telephone. In these instances, the questionnaire was administered by phone. The final number of respondents in 2019 was 10,293 with 591 aged 15 - 24.

2023

Following a tender process, the firm Advanis was chosen to carry out the sampling and data collection. The target number of respondents was 36,000. Age targets were set by province, with the aim of having a 50/50 split between men and women.

Respondents were recruited by telephone using lists of random in-service telephone numbers. When an individual was reached, they were asked a short set of screener questions including their willingness to participate. Those who agreed to participate, were at least 15 years old and currently a resident of a Canadian province were sent a link to the online survey by email or text message depending on their preference.

Most telephone numbers called were cell phone numbers as youth are very hard to reach by conventional landline. This is because younger people are less likely to have a landline, or do not answer landlines that are present in their homes. Additionally, the overall coverage of landlines is declining rapidly in Canada, with many households choosing to use only cell phones.

To further increase youth representation, telephone number sampling was supplemented with a general population random sampling panel (GPRS). This is a group of people who have previously agreed to participate in future studies. They were recruited to the panel through random sampling processes. As such, they were recruited in the same manner as the remainder of the CSUS participants. Within the panel, only those aged 15-24 were invited to respond to the questionnaire for CSUS 2023.

Reminders to complete the survey were sent on days 3 and 6 days after the initial invitation. For those 15-24, a third reminder was sent on day 9. Additionally, all of those aged 15-19 and recruited via telephone sampling were offered a $20 incentive for completing the survey (those aged 15-19 recruited through the GPRS were not offered this incentive).

The final number of respondents in 2023 was 36,180 with age, province, and sex breakdowns as follows:

In order to recruit these individuals, Advanis called more than 1.2 million phone numbers. Approximately half of calls reached a potential respondent. Of the potential respondents, 108,708 agreed to receive the link. This includes individuals who would have participated but were ineligible (younger than 15 or living outside of the Canadian provinces) and complete responses.

A very thorough information for participants form was used in 2023 before consent to participate was obtained. The information provided in 2023 was much longer and included stronger wording for possible risks of participating than previous cycles. It is possible that this dissuaded participants in 2023. The direction of any resulting bias in the outcome measures is unknown however, as both those who use substances and those who do not may have been dissuaded.

Understanding CSUS 2023

The changes to the sampling methodology in 2023 may have impacted estimates of substance use. CADS 2019 may have underestimated substance use because of the household sampling frame. In contrast, cell phone recruitment and oversampling of youth and young adults with the knowledge that the survey would be asking about substance use, may have resulted in an overestimation of substance use relative to 2019. While weighting was performed to allow for estimates at the population level in both instances, any biases in the samples that are associated with substance use would impact the final estimates. As such, we recommend using CSUS 2023 data without comparison to previous iterations. You can explore historical CSUS data on the Canadian Substance Use Survey page.

Going forward, comparison can be made between CSUS 2023 and future cycles of CSUS.

Survey weighting

For 2023, the data were weighted to add up to the population projections for 2022 from the 2021 census, based on province, age, and sex at birth. Age groups for weighing were: 15 to 17, 18 to 19, 20 to 24, 25 to 34, 35 to 44, 45 to 54, 55 to 64, and 65+. For variance estimation, the bootstrap method was used to generate 1,000 sets of mean bootstrap weights. Many statistical analysis packages are able to use bootstrap weights to generate variances and confidence intervals.

Statistical notes

All analyses were performed in SAS Enterprise Guide v. 7.1. For the data tool, Wilson confidence intervals are reported. While SAS survey analysis procedures can use bootstrap weights, they do not presently accommodate mean bootstrap weights. An alternative way to generate proper variances within SAS was therefore used. Mean bootstrap weights were still used, but the variance estimation method was set to balanced repeated replication (BRR) with Fay's coefficient set to 0.7113248654. Other statistical packages such as Stata can accommodate mean bootstrap weights, and would not require this workaround solution.

Suppression rules

For the data tool, results were suppressed where any denominator was <30. In some instances, this suppression results in data being shown for some population groups but not others.

Recommended citation

Canadian Substance Use Survey 2023 – Technical Notes. (2024) Government of Canada. Health Canada. Controlled Substances and Overdose Response Directorate, Office of Drug Research and Surveillance.

Glossary

Age-order selection

A random sampling method to identify the individual in a household who should respond to a survey. The question "how many individuals aged 15+ live in this household" is asked. The individuals are listed in ascending order of age, and a random number from 1 to the total in the household is chosen. As the number of individuals in the household was not known before the letter was sent for CADS 2019, a table was included to identify the random number for individual selection.

Balanced repeated replication

A resampling method that uses balanced half-samples drawn from the original sample. The BRR method's variance formula is similar to the bootstrap variance estimation method.

Bootstrap variance estimation

A resampling method that relies on successive samples from the original sample to generate replicate weights (called bootstrap weights). Bootstrap variance estimates are obtained from the bootstrap estimates.

Mean bootstrap weights

Under the standard bootstrap resampling method, units not selected in a particular bootstrap sample will have zero bootstrap weights. An abundance of zero weights can be problematic when they lead to estimates of zero in a denominator. The mean bootstrap method serves to reduce the occurrence of zero bootstrap weights. Instead of producing B sets of bootstrap weights, B sets of mean bootstrap weights are generated. This is done by generating B*R sets of bootstrap weights, and taking the average of R bootstrap weights to generate each mean bootstrap weight. For CSUS 2023, 12,000 sets of bootstrap weights were averaged, 12 at a time, to generate 1,000 sets of mean bootstrap weights. When using mean bootstrap weights, standard bootstrap variances have to be adjusted by a factor of R, i.e., multiplied by 12 in the case of CSUS 2023.

Fay's coefficient

A correction factor for variance estimates under balanced repeated replication (BRR). CSUS 2023 uses mean bootstrap weights, which requires multiplying standard bootstrap variance estimates by R=12, the number of bootstraps weights per mean bootstrap weight. Since SAS bootstrap variance procedures do not handle mean bootstrap weights, it is necessary to resort to a trick to generate proper mean bootstrap variances in SAS. SAS users must request BRR variance estimates, with the Fay coefficient set to 1 – R–½ which, in the case of CSUS 2023, equals 1 – 1/√12 = 0.7113248654.

Random digit dialling (RDD)

A method for selecting respondents for telephone surveys by generating telephone numbers at random from a frame, such as the list of all area code+prefix, or all telephone banks – the first 6 and 8 digits of a 10-digit number, respectively – known to have numbers in service from the target geographic area.

Resampling methods

Statistical techniques used to facilitate variance estimation in sample surveys. Sub-samples drawn from the original sample are used to generate replicate weights which, when used instead of original weights in survey estimates, produce replicate estimates. Variances and confidence intervals for survey estimates are obtained from the replicate estimates.

Wilson confidence intervals

An asymmetric confidence interval for estimates of ratios and percentages which has good performance for small samples and where the estimate is close to 0 or 100%. The Wilson confidence intervals for a percentage will not go below 0 or exceed 100%.