Chronic Diseases and Injuries in Canada

Volume 32, no. 1, December 2011

Research methods of the Youth Smoking Survey (YSS)

T. Elton-Marshall, PhD (1,2); S. T. Leatherdale, PhD (2); S. R. Manske, EdD (1); K. Wong, MSc (1); R. Ahmed, PhD (1,3); R. Burkhalter, MMath (1)

https://doi.org/10.24095/hpcdp.32.1.07

This article has been peer reviewed.

Author references:

1. Propel Centre for Population Health Impact, Canadian Cancer Society / University of Waterloo, Waterloo, Ontario, Canada
2. School of Public Health and Health Systems, University of Waterloo, Waterloo, Ontario, Canada
3. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada

Correspondence: Scott T. Leatherdale, School of Public Health and Health Systems, Faculty of Applied Health Sciences, University of Waterloo, 200 University Ave. West, Waterloo ON  N2L 3G1; Tel.: (519) 888-4567 ext 37812; Fax: (519) 886-6424; Email: sleather@uwaterloo.ca

Abstract

This paper describes the survey development, design and data collection protocol for the 2008/2009 Youth Smoking Survey (YSS) and the changes to the YSS survey and protocols across the 5 survey cycles (1994, 2002, 2004/2005, 2006/2007, 2008/2009). Canada’s Youth Smoking Survey is a nationally representative school-based survey of students (grades 6 to 12 in 2008/2009) from randomly sampled public and private schools in the ten provinces. The main objective of the YSS is to provide benchmark data on national smoking prevalence rates for youth. Key features of the 2008/2009 YSS include consistent measures across survey cycles, a survey team of researchers and non-governmental organizations, a link to school and student level measures, provision of tailored feedback reports to schools and publicly available datasets.

Keywords: youth, smoking behaviour, Canadian Youth Smoking Survey, survey cycles, questionnaires

Introduction

Nationally representative surveys of youth smoking behaviour are necessary to understand the social, regulatory, educational and commercial factors that influence smoking; to provide evidence for tobacco control policies and programs; and to monitor tobacco consumption in Canada.^{Footnote *} 

Methods

2008/2009 YSS development

A pan-Canadian consortium of university and non-governmental organizations implemented the 2008/2009 YSS. Members of the Youth Health Team at the Propel Centre for Population Health Impact at the University of Waterloo (Ontario) provided central leadership, while members from the other nine provinces provided leadership in their respective provinces. Members developed survey content during teleconferences. Those who could not participate in the scheduled meetings were asked to provide input prior to the teleconference. This approach allowed provincial stakeholders and the federal government to ensure the survey content included measures relevant to each jurisdiction. Content meetings ensured that core items (those required to compute smoking prevalence rates and derive other key, comparable variables) were retained. Questions added to the existing survey were those deemed higher in priority particularly if they were relevant to active policy agendas. The consortium made consensus decisions about which questions to include in the survey after discussing the merit of all survey questions during team teleconferences.

Each iteration of the YSS allows for a few new items; however, for every addition, about the same number is removed to keep the questionnaire the same length. Those items that tend to appear every other cycle are considered “periodic.” Some items, known as “deleted items,” are phased out completely if the issue/question is no longer relevant. While consistent content permits monitoring of trends over time, introducing new items permits identifying new trends that need to be monitored (see the 2008 YSS user guide for a list of survey items by cycle).

Survey measures

Core measures. To be consistent, “core” survey measures remain the same across all survey years. These include the measures used to define the smoking status of each respondent according to Health Canada definitions, measures of key prevention indicators such as susceptibility to future smoking, age of initiation and amount smoked, and key demographic variables. The core outcomes measured in the YSS are susceptibility to future smoking among never smokers and smoking status. The validated algorithm of Pierce et al. was used to measure susceptibility to future smoking among never smokers (those who have not smoked even a few puffs of a cigarette)^{Footnote +}. The de-identified dataset available to researchers. If a question could be skipped within the structure of the questionnaire, it was coded as 96 or 996 or 9996 within the PUMF dataset. For example, a smoker would still be asked questions about susceptibility to smoking but the responses for those questions would be coded as a “valid skip” and would be excluded from the analyses associated with smoking susceptibility.

Provision of school feedback reports to schools^{Footnote 8}

Starting with the 2004/2005 cycle, the YSS used the School Health Action, Planning and Evaluation System (SHAPES) for school-based data collection. Thus each participating school received a school-specific feedback report and executive summary within 10 weeks of data collection. This report provides customized information including smoking rates and other behavioural (e.g. time spent reading) and environmental information (e.g. smoking on school property) specific to the school. As a supplement to the YSS, information about the school environment (programs, policies and the built environment) was also collected^{Footnote +}

Sampling design

The target population for the YSS consisted of all young Canadian residents attending private and publicly funded schools in the 10 Canadian provinces. Those residing in the Yukon, Nunavut and Northwest Territories and those living in institutions or on First Nations reserves were not included in the sampling frame. Young persons who were attending special schools (e.g. schools for visually and hearing-impaired) or schools located on military bases were also excluded from the sampling frame.

The YSS team at the Propel Centre obtained a comprehensive list of all schools in each province via provincial Department of Education websites. The sampling for the YSS was based on a stratified multistage design. Sampling was stratified according to health region smoking rate and type of school (elementary or secondary). In Stage 1, the Canadian Community Health Survey (CCHS) was used to calculate the smoking rate among 15 to 19 year olds for each health region. The school lists obtained from the provincial Departments of Education for each of the 10 provinces included enrolment data by grade for each school. Using this list, the total eligible grade enrolment in a health region was used as a weight to compute the median smoking rate for each province. Each school’s six-digit postal code was used to identify the health region in which it was located. Schools were then categorized as “low” or “high” smoking rate stratum based on the smoking rate in their health region compared to the median (where greater than or equal to the median was categorized as “high”).

In Stage 2, schools were stratified into elementary or secondary school strata (calculated based on whether there was a higher enrolment of students in grades 6 to 8 or 9 to 12). Elementary and secondary schools were sampled on a 2:1 ratio due to the smaller enrolment sizes of the elementary schools. Schools were also over-sampled in each province based on the provincial school recruitment rate from the 2004/2005 and 2006/2007 YSS cycles.

In Ontario, the design of the 2008/2009 cycle included a third health region stratum, Greater Toronto Area (GTA). The GTA health region stratum acknowledged the size of the GTA and the importance of being able to capture schools from the GTA even if there were refusals from the larger school boards in the city of Toronto.

Lastly, sampling of private schools was based on a simple random sample of private schools in each province. The number of schools originally selected was roughly proportional to the number of students enrolled in private schools in that province as compared to the total in public schools. The sampling design is constructed to provide a representative sample of youth in all provinces in Canada.

In the 2008/2009 cycle, the school board response rate was 84% (the number of school boards that agreed to participate/the number of school boards that were approached); the school level response rate was 59% (the number of schools that agreed to participate/the number of schools that were approached); and the student level response rate was 73.2% (based on the number of completed surveys/the number of eligible students; students who were absent during the data collection were counted as a non-response).

Survey protocol

In all provinces, YSS site coordinators contacted school boards prior to approaching schools. Private schools were approached directly because there is no governing board to review research requests for these schools. School boards were typically contacted via a formal board-specific application or a standard board recruitment package that included a school invitation letter, a project brochure, a sample student survey, sample parent information and permission materials, and a template school feedback report.^{Footnote #}  Provincial site coordinators made follow-up calls to the school board to answer any questions and, ideally, obtain board permission to recruit schools. Once a school board was successfully recruited, the schools within that school board were approached via a school recruitment package and follow-up phone calls. The contents of the school recruitment packages were the same for both boards and schools. Only when the school had agreed to participate in the YSS was the survey implemented with eligible students in that school.

Within each participating school, all students in the eligible survey grades (6 to 12) were requested to complete the survey. Active parental permission was required by the school or board for 62% of grade 6 to 8 classes (n = 913) and 19% of grade 9 to 12 classes (n = 372). Students in eligible classrooms took home information letters describing survey details. Active permission protocols required signed parental and child permission forms for the child to receive and complete a survey. In 81% of secondary school classes (n = 1631), passive permission protocols were used to reduce the burden on schools and improve response rates. In this procedure, the school mailed an information letter home to parents that detailed survey procedures, and asked parents to call a toll-free number or inform the school if they did not want their child to participate. Students whose parents objected were put on a “no permission” list and did not receive a survey on the day of data collection. All other students received a survey to complete. Regardless of whether parents provided permission, students were able to decline participation on the day of data collection.

Provincial site coordinators worked with a school contact to arrange data collection at each school. On the day of data collection, teachers administered the survey using standardized protocols during a designated class period. To ensure confidentiality and therefore encourage honest responses, teachers were asked to avoid circulating among the students. Students were also required to place their completed survey in an envelope and seal this envelope before it was collected by a student in the classroom. When parents as well as students were surveyed, active consent was required, and a tear-off sheet with the student’s name was attached to the front of the survey. Students removed the tear-off sheet. A serial code on both the tear-off sheet and the student survey enabled linkage for survey cycles that included a parent interview to be linked to the student responses. The information containing the student’s identification and responses were removed from all public datasets and only those directly related to the research had access to any identifying information. On average, the survey took 30 to 40 minutes to complete. A data collector was on site at the school throughout the data collection period and available to answer respondent questions and collect the completed student surveys.

Data management

Surveys were machine scanned using Optical Mark Reading (OMR) technology. Quality control measures (e.g. visual scanning, OMR scanning twice to find discrepancies) were used to ensure accuracy of the scanned data. An online survey implementation system (OSIS) permitted central management of recruitment, implementation, analysis and feedback processes.

Survey weights

Survey weights were created to “weight” the data to be representative of the general population of Canadian youth in school. The survey weights were developed in two stages. In the first stage, a weight (W1) was created to account for the school selection within health region and school strata. A second weight (W2) was then calculated to adjust for student non-response. The weights were then calibrated to the provincial gender and grade distribution so that the total of the survey weights by gender, grade and province would equal the actual enrolments in those groups. Finally, bootstrap weights for each province (to estimate sampling error) were generated.

Evolution of the YSS

The protocols described were used to implement the 2008/2009 YSS cycle. One of the strengths of the YSS has been its consistent protocols, which allow comparisons over cycles. However, there have been slight modifications to the sampling and protocols in each cycle based on experience in previous cycles; these modifications were made to improve student recruitment and survey completion rates, and to reduce the burden on participating schools. The following section describes some of the significant differences in the YSS over the various cycles.

Changes to survey administration. One of the most significant changes to the YSS occurred in 2004/2005 when the survey administration shifted from Statistics Canada to the University of Waterloo. In 1994 and 2002, the YSS content was developed by Health Canada’s Office of Tobacco Control and data were collected by Statistics Canada. As previously noted, the University of Waterloo’s Propel Centre for Population Impact (formerly the Population Health Research Group and the Centre for Behavioural Research and Program Evaluation) has provided central leadership since 2004/2005.

Changes to the survey. Table 1 summarizes the differences in the survey over time. Until 2006/2007, the sample included grades 5 to 9 only. In 1994, all students in grades 5 to 9 responded to the same survey. In 2002 and 2004/2005, students in grades 7 to 9 answered additional questions about alcohol and drug use. In 2006/2007, students in grades 7 to 12 were randomly assigned to receive one of two versions of the survey. While the majority of the questions were the same in both versions, including those that related to alcohol and drug use, some different questions were added to each (e.g. in one version there were questions about smoking on school property whereas another version had questions about beliefs about the harmful effects of smoking). All other students (grades 5 and 6) received a survey with no questions on alcohol and drug use. Because there were two different versions of the survey in this cycle, there were two survey weights calculated for this dataset and two User Guides to facilitate use of the dataset.

In 2008/2009, grade 5 students were no longer included in the survey, primarily because of the low prevalence of smoking among students in this grade and the challenges of having students in this grade to complete the survey in the time allotted. Grade 6 students completed the survey without the alcohol and drug use questions whereas those in grades 7 to 12 completed a survey that included alcohol and drug use questions.

Collaboration. Whenever possible, YSS data collection was coordinated with other data collections taking place at the same time. In 2006/2007, YSS collaborated with the University of New Brunswick’s Health & Education Research Group (HERG) and with the Comprehensive School Health Research Group in Prince Edward Island to implement their provincial surveys in 2008/2009 (NB Wellness Survey and SHAPES-PEI, respectively). Both initiatives collected data on smoking (YSS), healthy eating (HE), physical activity (PA), and mental fitness (MF) from students in grades 5 to 12 (grades 6 to 12 for NB Wellness with the exception of YSS-sampled schools, which were grades 5 to 12). The data included a census of eligible schools in the respective provinces. The YSS dataset does not include any data collected from the NB Wellness or SHAPES-PEI additional modules, but the dataset does include the additional students who responded to the YSS. The data collection procedures therefore varied slightly for NB and PEI. Table 1 summarizes these differences in data collection.

Changes to sampling design

In 1994, the sample design consisted of a two-stage stratified clustered design in which schools were the primary sampling units and classes were the secondary units. There were two levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. The school sample was selected systematically with probability proportional to school size (the total number of students for each grade). Classes within schools were randomly selected and all students in a selected class were included in the final sample.

In 2002, the sample design featured three levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. Schools were also stratified by census metropolitan area (CMA) versus non-CMA, with additional strata in Quebec (Montréal) and Ontario (Toronto). The sample was then selected in each stratum independently, meaning that some schools could be selected more than once for different grades. Classes were randomly selected from the schools that were recruited.

In 2004/2005, the sampling was conducted in two stages. In stage 1, school boards were sampled within each province. The Canadian Community Health Survey (CCHS) was used to estimate the current smoking rate at the level of health region. Estimated adult smoking rates were calculated for each school board and the school boards were ranked and categorized as “upper stratum” or “lower stratum.” In stage 2, schools were sampled from the list of selected school boards. School boards were selected based on their adult smoking rate. Within each selected school board, schools were stratified into two strata: senior strata (students in senior elementary or high school grades) or junior strata (students in a school with grades 5, 6, 5–6, 5–7, and 6–7). Where possible, there was an over-selection of junior stratum schools. All eligible grades within a school were selected to participate, rather than just a random selection of classes within a school.

The sample design in 2006/2007 was the same as the design described for the 2008/2009 survey cycle with a few small exceptions. The smoking rate calculated for the province and health region was based on adult smoking rates, and there was no separate stratum for the Greater Toronto Area. Again, all classes in eligible grades in selected schools were surveyed.

Discussion

The Youth Smoking Survey is a nationally representative school-based survey of youth in Canada. The YSS was designed to provide both national (excluding Yukon, Northwest Territories and Nunavut) and provincial estimates of smoking prevalence, as well as surveillance of tobacco-related knowledge, attitudes and behaviours of young people in Canada. However, the YSS is more than a surveillance tool. It was designed to assess and help develop public education programs and policies for tobacco control. With the integration of the YSS with the SHAPES, the YSS is even more capable of integrating tobacco control policy and practice and monitoring the effectiveness of tobacco control strategies through the school-specific feedback reports.

There are several unique features of the 2008/2009 YSS when compared to other Canadian surveys:

• The core measures used in the YSS are maintained over survey cycles. This allows monitoring tobacco use over time and evaluation of tobacco control policies/programs (using a quasi-experimental design, comparing survey measures before and after a tobacco control policy or program is implemented). These core measures are also consistent with other existing surveys to allow comparisons between groups.^{Footnote §} which also has publicly available raw data, and Statistics Canada Data Liberation Initiative (DLI).^{Footnote **} Both the Propel Centre and Health Canada also have summary tables from each survey year available on their websites.

The YSS has been used to guide tobacco control policies and programs nationally. For instance, the 2008/2009 YSS data were instrumental in prompting the federal government to amend the Tobacco Act in 2009 as part of Bill C-32 to prohibit the use of flavour additives in cigars and cigarillos.

Acknowledgements

The authors would like to thank the Propel Centre for Population Health Impact for providing support for this project. Dr. Leatherdale is a Cancer Care Ontario Research Chair in Population Studies. The 2008/2009 Youth Smoking Survey is a product of a pan-Canadian capacity-building project that includes Canadian researchers from all provinces and provides training opportunities for university students at all levels. Production of this paper has been made possible through a financial contribution from Health Canada. The views expressed herein do not necessarily represent the views of Health Canada. The authors recognize the key role that Murray Kaiserman, now retired Director in Health Canada's Tobacco Control Programme and the Controlled Substances and Tobacco Directorate, has played in the initiation and continued funding of the YSS.

References

1. Stephens T, Morin M, editors. Youth Smoking Survey, 1994: Technical Report. Ottawa (ON): Minister of Supply and Services Canada; 1996.
2. Youth Smoking Survey (YSS): 2008-09 YSS Microdata User Guide [Internet]. Waterloo (ON): Propel Centre for Population Health Impact; 2009 [cited 2011 April 11]. 53 p. Available from: http://www.yss.uwaterloo.ca/yss_papers/Documents/yss08_user_guide_EN.pdf
3. Global Youth Tobacco Survey Collaborative Group. Tobacco use among youth: a cross-country comparison. Tobacco Control. 2002;11:252-70.
4. Paglia-Boak A, Mann RE, Adlaf EM, Rehm J. Drug use among Ontario students, 1977-2009: Detailed OSDUHS findings. (CAMH Research Document Series no. 27). Toronto (ON): Centre for Addiction and Mental Health; 2009.
5. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG. Reliability of the 1999 Youth Risk Behavior Survey Questionnaire. J Adolesc Health. 2002;31:336-42.
6. Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey Questionnaire. Am J Epidemiol. 1995;141(6):575-80.
7. Ford KH, Diamond PM, Kelder SH, Sterling KL, McAlister AL. Validation of scales measuring attitudes, self-efficacy, and intention related to smoking among middle school students. Psychol Addict Behav. 2009;23(2):271-8.
8. Pierce JP, Choi WS, Gilpin EA, Farkas AJ, Merritt RK. Validation of susceptibility as a predictor of which adolescents take up smoking in the United States. Health Psychol. 1996;15:355-61.
9. Summary of Results of the 2008-09 Youth Smoking Survey [Internet]. Ottawa (ON): Health Canada; 2010 [cited 2010 Jul 25]. Available from: http://www.hc-sc.gc.ca/hc-ps/tobac-tabac/research-recherche/stat/_survey-sondage_2008-2009/result-eng.php
10. Wang S, Moss JR, Hiller JE. Applicability and transferability of interventions in evidence-based public health. Health Promot Int. 2006;21:76-83.
11. Bill C-32: an act to amend the Tobacco Act [Internet]. 2009 Jun 4; revised 2010 Feb 4 [cited 2010 Oct 22], c. 32. Available from: http://www2.parl.gc.ca/Sites/LOP/LegislativeSummaries/Bills_ls.asp?lang=E&ls=c32&source=
    library_prb&Parl=40&Ses=2
12. Smoke-Free Ontario Scientific Advisory Committee. Evidence to guide action: comprehensive tobacco control in Ontario. Toronto (ON): Ontario Agency for Health Protection and Promotion; 2010. Chapter 5, Prevention of tobacco use among youth and young adults; p. 75-92. Available from: http://www.oahpp.ca/services/documents/evidence-to-guide-action/Chapter%205%20-%20Evidence%20to%20Guide%20Action%20-%20CTC%20in%20Ontario%20SFO-SAC%202010E.pdf
13. Elton-Marshall T, Leatherdale ST, Burkhalter R. Tobacco, alcohol and illicit drug use among Aboriginal youth living off-reserve: results from the Youth Smoking Survey. CMAJ. 2011;183(8):E480-6.
14. Callaghan RC, Veldhuizen S, Leatherdale S, Murnaghan R, Manske S. Use of contraband cigarettes among adolescent daily smokers in Canada. CMAJ.2009;181(6-7):384-6.
15. Chan WC, Leatherdale ST, Burkhalter R, Ahmed R. Bidi and hookah use among Canadian youth: an examination of data from the 2006 Canadian Youth Smoking Survey. J Adolesc Health. 2011;49(1):102-4.
16. Hammond D, Ahmed R, Yang WS, Burkhalter R, Leatherdale S. Illicit substance use among Canadian youth: trends between 2002 and 2008. Can J Public Health. 2011;102:7-12.
17. Leatherdale ST, Ahmed R. Alcohol, marijuana, and tobacco use among Canadian youth: do we need more multi-substance prevention programming? J Prim Prev. 2010;31(3):99-108.
18. Leatherdale ST, Hammond D, Ahmed R. Alcohol, marijuana, and tobacco use patterns among youth in Canada. Cancer Causes and Control. 2008;19:361-9.
19. Leatherdale ST, Hammond D, Kaiserman M, Ahmed R. Marijuana and tobacco use among young adult smokers in Canada: are they smoking what we think they are smoking? Cancer Causes Control. 2007;18(4):391-7.
20. Leatherdale ST, Ahmed R, Barisic A, Murnaghan D, Manske S. Cigarette brand preference as a function of price among smoking youth in Canada: are they smoking premium, discount or native brands? Tob Control. 2009;18(6):466-73.
21. Sen A, Wirjanto T. Estimating the impacts of cigarette taxes on youth smoking participation, initiation, and persistence: empirical evidence from Canada. Health Econ. 2009;1264-80.
22. Leatherdale ST, Ahmed R. Second-hand smoke exposure in homes and in cars among Canadian youth: current prevalence, beliefs about exposure, and changes between 2004 and 2006. Cancer Causes Control. 2009;20(6):855-65.
23. Leatherdale ST, Smith P, Ahmed R. Youth exposure to smoking in the home and in cars: how often does it happen and what do youth think about it? Tob Control. 2008;17(2):86-92.
24. Leatherdale ST, Ahmed R, Vu M. Factors associated with different cigarette access behaviours among underage smoking youth who usually smoke contraband (native) cigarettes. Can J Public Health. 2011;102(2):103-7.
25. Lovato CY, Pullman AW, Halpin P, Zeisser C, Nykiforuk CI, Best F, et al. The influence of school policies on smoking prevalence among students in grades 5-9, Canada, 2004-2005. Prev Chronic Dis. 2010;7(6):1-10.
26. Nowatzki J, Schultz AS, Griffith EJ. Discrepancies between youth and parent perceptions of their household environment relevant to smoking: a secondary analysis of the 2004/05 Canadian Youth Smoking Survey. Chronic Dis Can. 2010;30(3):78-83.
27. Schultz AS, Nowatzki J, Dunn DA, Griffith EJ. Effects of socialization in the household on youth susceptibility to smoking: a secondary analysis of the 2004/05 Canadian Youth Smoking Survey. Chronic Diseases in Canada. 2010;30(3):71-77.
28. Seguire M, Chalmers KL. Late adolescent female smoking. J Adv Nurs.2000;31(6):1422-9.