A review of recent analyses of the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) - HPCDP: Volume 35-8/9, October/November 2015

Volume 35 · Number 8/9 · October/November 2015

A review of recent analyses of the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS)

D. Potter, MA; T. Nasserie, MPH; L. Tonmyr, PhD

https://doi.org/10.24095/hpcdp.35.8/9.02

This article has been peer reviewed.

Author reference: Surveillance and Epidemiology Division, Public Health Agency of Canada, Ottawa, Ontario, Canada

Correspondence: Lil Tonmyr, Surveillance and Epidemiology Division, Public Health Agency of Canada, 785 Carling Ave, Ottawa, ON K1A 0K9; Tel: 613-240-6334; Fax: 613-941-6439; Email: Lil.Tonmyr@phac-aspc.gc.ca

Abstract

Introduction: The objective of this analysis is to identify, assess the quality and summarize the findings of peer-reviewed articles that used data from the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) published since November 2011 and data from provincial oversamples of the CIS as well as to illustrate evolving uses of these datasets.

Methods: Articles were identified from the Public Health Agency of Canada's data request records tracking access to CIS data and publications produced from that data. At least two raters independently reviewed and appraised the quality of each article.

Results: A total of 32 articles were included. Common strengths of articles included clearly stated research aims, appropriate control variables and analyses, sufficient sample sizes, appropriate conclusions and relevance to practice or policy. Common problem areas of articles included unclear definitions for variables and inclusion criteria of cases. Articles frequently measured the associations between maltreatment, child, caregiver, household and agency/referral characteristics and investigative outcomes such as opening cases for ongoing services and placement.

Conclusion: Articles using CIS data were rated positively on most quality indicators. Researchers have recently focussed on inadequately studied categories of maltreatment (exposure to intimate partner violence [IPV]), neglect and emotional maltreatment) and examined factors specific to First Nations children. Data from the CIS oversamples have been underutilized. The use of multivariate analysis techniques has increased.

Keywords: child maltreatment, child abuse, public health surveillance

Key findings

  • In this review of 32 peer-reviewed published articles, the majority were of high quality with clearly stated research aims, appropriate control variables, appropriate analyses, sufficient sample sizes, appropriate conclusions and relevance to practice or policy.
  • Researchers using CIS data have recently focussed on inadequately studied categories of maltreatment, including exposure to intimate partner violence, neglect and emotional maltreatment, and have examined factors specific to First Nations children.
  • The use of complex multivariate analysis methods has recently increased.

Introduction

The Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) has been completed for three cycles-1998,Footnote 1 2003,Footnote 2 and 2008Footnote 3-and has provided a wealth of information on investigations of reported child abuse and neglect nationwide.Footnote 4 The CIS collects data about children who have been reported to child welfare agencies because of allegations of maltreatment.Footnote 1-Footnote 3 CIS data can be used as a source of information about children who have experienced alleged and substantiated neglect, exposure to intimate partner violence (IPV), emotional (or psychological) maltreatment, physical abuse and/or sexual abuse. It includes information on the characteristics of the child, caregiver, household and investigating agency as well as short-term service outcomes such as placement. CIS data is used by senior child welfare decision makers to help determine resource allocation, identify at-risk populations, understand reported maltreatment trends, validate findings at individual agencies and direct changes in practice.Footnote 4 It is also accessed by a broad range of experts in clinical medicine, public health, social work, law and justice, education, sports, recreation, and faith-based groups.Footnote 5

The CIS-2008: Major Findings reportFootnote 3 details the breadth and scope of the CIS. Each cycle of the CIS contains information on a large sample of maltreatment cases reported to child welfare agencies and opened for investigation. Child welfare workers in selected agencies across Canada completed the CIS survey for each investigation they completed within a three-month data collection period in the fall of the survey year. The participating child welfare workers were given definitions of maltreatment to ensure consistency. A multistage stratified clustered sample from all provinces and territories that included mainstream and First Nations agencies was used. Some provinces and territories provided additional funding to the CIS to obtain data (i.e. oversample) from their respective jurisdictions.

The CIS-2008 study differs from the CIS-1998 and 2003 waves because investigations of risk of future maltreatment were tracked separately from investigations of allegations of specific maltreatment. In addition, definitions of maltreatment have evolved over time. For example, the CIS-2008 tracked three subtypes of exposure to IPV: indirect exposure to physical violence, direct exposure to physical violence and exposure to emotional violence.Footnote 6 Previously, the CIS captured IPV as a subtype of emotional maltreatment. Differences between provincial and territorial legislation and changes in detection, reporting and investigation procedures over time also influence the CIS findings. This means that changes in estimates may not represent actual changes in the occurrence of maltreatment.

An earlier reviewFootnote 7  summarized and critically assessed 37 peer-reviewed analyses of CIS data published before November 2011. Roughly half of the sources were descriptive and half were multivariate. The review assessed the quality of the articles and found the multivariate articles to be generally of good quality; stronger evidence was provided for their objectives compared to the descriptive articles. The review recommended that future research use clear directional hypotheses and multivariate techniques.Footnote 7 

Considering the cyclical nature of the CIS, and especially because the Public Health Agency of Canada decided not to collect data in 2013, this is an opportune time to update the findings from the previous review and ask questions about the relevance of the CIS in terms of policy and practice. Only 3 of the articles in the previous reviewFootnote 7 utilized data from CIS-2008. In addition, the information was limited because the data had only recently been released. Furthermore, the previous reviewFootnote 7 was critiqued for not including results stemming from oversampling provinces. As more articles utilizing these and earlier waves of data have since been published, our review focusses on CIS-related literature published after November 2011 and analyses of CIS oversample data published at any time.

Knowledge users (decision makers and policy makers) find effectively summarized research useful. The surveillance reports stemming from the CIS have been used as a reference to obtain information quickly.Footnote 4 Similarly, we hope that this review will provide a quick reference for topics that have been analysed using the CIS data as well as on the quality of these articles. This review may also inspire researchers to further knowledge about child maltreatment and the responses of child welfare agencies. The information may also serve to improve the CIS as a surveillance tool by identifying some gaps in data collection and analysis. Lastly, the information has the potential to increase awareness of this important public health issue.

The specific objectives of this review were to

  • identify and retrieve all peer-reviewed studies published between November 2011 and the present day that used CIS data or CIS provincial and territorial oversampling data collected since the inception of the study;
  • assess the quality of those studies;
  • summarize the findings of those studies; and
  • illustrate the evolving uses of CIS data.

Methods

We included original research published in peer-reviewed journals as these can be expected to be of the highest quality and include sufficient information on methods and analyses to assess quality. Other sources, such as book chapters and presentations, may not follow a standardized format of presenting information or have specific objectives or hypotheses to testAs such, our quality assessment tool would not be suited to them. In addition, because the review is restricted to peer-reviewed journal articles, interested readers should find it easier to locate and access our primary sources. Table 1 lists all survey waves of the CIS, jurisdictions that oversampled and First Nations samples.

Table 1 CIS, CIS oversamples and First Nations studies
Population Cycles
Canada 1998, 2003, 2008
British Columbia 1998, 2008
Alberta 2003, 2008, 2013
Saskatchewan 2008
Ontario 1993, 1998, 2003, 2008, 2013
Quebec 1998, 2008, 2014
Northwest Territories 2003
First Nations 1998, 2003, 2008

Articles were identified through the Public Health Agency of Canada data request records. In addition, we conducted a search for articles published by authors from the CIS research team to ensure completeness. As members of the research team did not need to request permission to use CIS data, their publications would not be captured in data request records.

Figure 1 summarizes the article selection strategy (source, inclusion and exclusion criteria) and presents the list of quality assessment questions. The quality assessment questions were the same as those used in the previous review of CIS dataFootnote 7 (discussed above). Two raters independently reviewed each article for inclusion and completed the quality appraisal tool for each. Discrepancies in ratings of the articles were discussed until consensus was reached or after discussion with a third rater. Raters did not review articles they had authored.

Figure 1 Search strategy and article appraisal

This figure summarizes the article selection strategy (source, inclusion and exclusion criteria) and presents the list of quality assessment questions.

The inclusion criteria were the following: peer-reviewed article, and original analysis of CIS data or CIS oversample data.

The exclusion criteria were the following: previously included in Tonmyr et al. (2012)Footnote 7 ; comments and letters to the editor; status reports; abstracts/dissertations; focus on policy or methods.

Two raters independently reviewed each article for inclusion and completed the quality appraisal tool for each article. Discrepancies in ratings of the articles were discussed until consensus was reached or after discussion with a third rater. Raters did not review articles they had authored.

32 articles were included: 12 with a descriptive approach and 20 with a multivariate approach.

The quality assessment questions, which were the same as those used in the previous review of CIS data, were the following:

Topic

  1. What is the research question?
  2. Are the aims of the research clear?

Methods

  1. Are clear definitions used for variables?
  2. Are eligibility and exclusion criteria clearly stated?
  3. Is the sampling strategy used for the study clear/appropriate?
  4. Are the right analyses used to answer the research question?
  5. Is the number of participants sufficient?

Findings/Conclusions

  1. Is it clear how the data were analyzed?
  2. Are the results clearly presented?
  3. Do researchers take into account/discuss potential confounding factors?
  4. Are the conclusions drawn supported by the study results?
  5. Is the topic of the study relevant for practice/policy?

Results

A total of 32 studies were identified for inclusion. Of those, 20 were considered multivariate and 12 descriptive; 24 used national level data and 8 provincial or territorial level data; 5 analyzed data from Quebec and 3 from Ontario; 1 used data from the First Nations component of the CIS-2008.

Table 2 shows the objectives, methods and quality assessment results of the included studies. Ninety-one percent of authors clearly described their research aims. In contrast, only 40% clearly defined all variables. Seventy-two percent of the studies provided clear inclusion criteria for cases and 72% described clear and appropriate sampling methods. Seventy-eight percent of analyses were considered appropriate to the authors' research questions. The number of participants was considered sufficient 91% of the time. Analytical procedures and results were usually clearly explained (72%) and presented (75%). Eighty-one percent of conclusions were considered appropriate, and all but one study was judged to have been about a topic with clear applications for practice or policy.

Table 2 Characteristics of articles analyzing CIS and CIS oversample data
Ref. & CIS year Topic Sample size (N) Methods Research aims clear Variable definitions clear Inclusion criteria clear Sampling appropriate Analyses appropriate # of participants sufficient Analysis procedures clear Results clearly presented Confounds addressed Conclusions appropriate Relevant to practice/policy

Abbreviations: CIS, Canadian Incidence Study of Reported Child Abuse and Neglect; EIQ, Quebec Incidence Study of Reported Child Abuse and Neglect; FNCIS, First Nations Component of the Canadian Incidence Study of Reported Child Abuse and Neglect; IPV, intimate partner violence; OIS, Ontario Incidence Study of Reported Child Abuse and Neglect.

(8) CIS-2008 To explore the distribution of child and household characteristics for substantiated single and multiple maltreatment types, to measure whether a specific type of maltreatment is associated with greater risk of functional impairment, and to identify child and household characteristics associated with functional impairment. 7352 Generalized mixed models + + + + + + + + + + +
(9) CIS-1998, CIS-2003 To identify clinical and organizational factors (e.g. proportion of Aboriginal reports) that influence the decision to place a child in care after a child maltreatment investigation. 2059 Logistic regression + + + - - - - - - - +
(10) CIS-2003 To identify factors that predict membership in different categories of emotional maltreatment and other forms of maltreatment. 11 562 Logistic regression - - + + + + + + + + +
(11) EIQ-1998 To increase understanding of emotional maltreatment cases investigated in Quebec. 3774 Chi-square, ANOVA + + - - + + + + + + +
(12) EIQ-1998 To identify the needs of children reported to child protective services for substantiated reasons and identify profiles of children. 2079 Cluster analysis + - + + + + + + + + +
(13) CIS-1998, CIS-2003 To measure the influence of clinical and organizational characteristics on the decision to place Aboriginal children in out-of-home placements upon the conclusion of an investigation. 2059 Logistic regression + - + - + + + + + + +
(14) CIS-2008 To further explore the finding that the proportion of investigations of Aboriginal children is associated with placement for all children in an agency. 1710 Multi-level logistic regression + + + + + + + + + + +
(15) CIS-2008 To identify the profile of young parents referred to child welfare, and the factors that determine service provision for these referrals. 5222 Logistic regression + - + + + + + - + + +
(16) CIS-2008 To identify the factors that determine the provision of services at the conclusion of investigation for investigated children under one year of age. 1203 Logistic regression, Classification and regression trees + - + + + + + + + + +
(17) CIS-2008 To identify the factors that predict service provision at the end of a maltreatment-related investigation involving infants. 538 Classification and regression trees + - + + + + - - - + +
(18) CIS-2008 To explore profiles of families and children across investigations for maltreatment and risk of maltreatment. 11 925 Logistic regression + - + - + + - + + - +
(19) CIS-2008 To describe the profile of adolescents reported to child welfare. 15 980 Logistic regression + + + + + + - + + + +
(20) CIS-2003 To measure the relationship between child functioning concerns and contextual, parent, parenting, and child characteristics. 1170 Logistic regression + - + - - + + + - + +
(6) CIS-2008 To examine the assessment of risk factors and child functioning concerns by child welfare workers and their relation to subtypes of exposure to IPV. 2184 Logistic regression + + + + + + + + + + +
(21) CIS-2003, CIS-2008 To describe families with teen mothers (18 or younger) and young adult mothers (19-21) in terms of modifiable risk factors and compare them to mothers aged 22 years and older. 6836 Chi-square + + + + + + + + - + +
(22) CIS-2008 To identify the factors that predict the decision to provide services after a child maltreatment report. 15 980 Logistic regression - + - - + + + + + + +
(23) CIS-2003 To identify the frequency of reports by educators compared with other professionals, to identify the type of risk factors present in cases reported by educators, and to determine whether differences in case characteristics reported by educators account for differences in investigation outcomes. 7725 Logistic regression + + + + + + - + + + +
(24) CIS-2003 To describe characteristics of Asian children, families, and households in the Canadian child welfare system and to examine short-term service provision or transfer to ongoing services. 1798 Logistic regression + - + + + + + + + + +
(25) CIS-2008 To provide updated profile of substantiated IPV investigations and examine the differences between IPV investigations and other investigations. 15 890 Chi-square + - - + - + + + - + -
(26) CIS-2008 To examine case characteristics and service disposition for investigations of children with non-Aboriginal primary caregivers who speak neither English nor French as their primary language. 10 334 Logistic regression + - - - + + + + + - +
(27) EIQ-1998 To identify factors that predict maltreatment or behavioural problems. 2232 Logistic regression + - + + + + + - + + +
(28) EIQ-1998 To examine different family structures in substantiated cases of child neglect and their associated characteristics, and to compare maternal and paternal single parent families. 1778 Chi-square + + + + + - + + - + +
(29) CIS-2008 To identify the prevalence of supervisory neglect and physical harm and explore characteristics related to it. 4159 ANOVA, Chi-square + - + + + + - + - + +
(30) CIS-2008, FNCIS-2008 To describe the level and nature of the over-representation of First Nations children in child welfare investigations, including information on referral sources, risk factors, and child functioning concerns. 15 346 Bivariate analyses + + + + - - - + - - +
(31) CIS-2008 To identify factors associated with joint investigations by police officers and child welfare workers. 11 807 Multiple logistic regression + + + + + + - + + + +
(32) EIQ-1998 To describe the similarities and differences between Aboriginal and non-Aboriginal cases of maltreatment reported to Youth Protection services in Quebec. 430 Logistic regression + - + - - + - - + - +
(33) CIS-2008 To examine whether exposure to IPV, hitting, or neglect as sole concerns require a traditional protection response, or a family support response. 11 807 Chi-square + - - + - + + - - + +
(34) CIS-1998, CIS-2003, CIS-2008 To measure differences in response to emotional maltreatment vs. other maltreatment and whether a more specific categorization system for emotional maltreatment can differentiate between maltreatment and family problems. 30 972 Chi-square, T-tests - - - + + + + - - - +
(35) OIS-1993, OIS-1998 To identify what is responsible for changes in the rates of reported and substantiated maltreatment between 1993 and 1998. 5500 T-tests + - + + - + + + - + +
(36) CIS-1998, CIS-2003, CIS-2008 To explore the difference between risk and harm in the context of child welfare. 35 214 Chi-square, T-tests + + - - + + + + - + +
(37) OIS-1993 To describe the incidence and characteristics of reported maltreatment, the major forms of maltreatment investigated, child and family characteristics, and outcomes of investigations. 2447 Percentages + - - + + + + + + + +
(38) CIS-2008 To evaluate whether there is a specific impact of delinquency/youth criminal justice involvement on decisions to transfer cases to ongoing services above other functioning issues. 7842 Logistic regression + - - + + + + - + + +

The objectives of 28% of the articles were to do with particular types of maltreatment. Two were about neglect, three about emotional maltreatment, and two about exposure to IPV. One study was about exposure to IPV, hitting or neglect as sole concerns for investigation. None of the studies had objectives exclusively to do with physical abuse. One paper examined the frequency of joint police and child welfare investigations of sexual abuse cases compared with other maltreatment cases.

Table 3 summarizes the associations between independent, control and dependent variables measured in the articles presenting multivariate analyses. In this review, we define multivariate analyses as those that simultaneously measure multiple independent and/or dependent variables. For articles that included multiple models, only the final model is included for each dependent variable tested.

Table 3 (Section 1 of 5) Associations between independent, control, and dependent variables in the articles presenting multivariate analyses
Police involvement in investigationFootnote 31 Opening for ongoing servicesFootnote 16 Opening for ongoing servicesFootnote 15 Opening for ongoing servicesFootnote 19 Opening for ongoing servicesFootnote 26 Opening for ongoing services Footnote 8-Footnote 11,Footnote 38 Opening for ongoing services Footnote 12-Footnote 15,Footnote 38

R, reference group; -, no statistically significant relationship (p >.05); ↓, statistically significant negative relationship (p <.05); ↑, statistically significant positive relationship (p <.05).

Abbreviations: FAE, fetal alcohol effects; FAS, fetal alcohol syndrome; IPV, intimate partner violence;

Note: Where more than one symbol appears in a cell, that article featured finer grained categories of the variable in question which have been collapsed for concision and those finer categories had different relationships with the dependent variable.

a This variable combined physical and emotional harm.
b This variable combined physical and emotional harm.
c This variable was two or more moves.

Maltreatment characteristics
Physical abuse   R   R R R
Sexual abuse   -     -
Neglect       -
Emotional maltreatment   -    
Exposure to IPV   -   -   -
Indirect physical IPV exposure              
Emotional IPV exposure              
Direct physical IPV exposure              
Co-occurring forms of IPV exposure              
More than one type of maltreatment              
Risk   -       - -
Substantiated case            
Physical harm a          
Mental/emotional harm b            
Duration              
Maltreatment lasting > 6 months              
Male alleged perpetrator            
Child characteristics
Attachment issues   -        
Intellectual/learning/developmental disability   -        
Academic difficulties           -
FAS/FAE   -        
Positive toxicology @ birth          
Male sex       - -
Age 6-15 years              
Child under 6 years of age              
Child older than 15 years of age            
Aboriginal status            
Physical disability/health issue   -          
Aggression            
Depression          
Emotional/behavioural issue              
Internalizing concerns            
Externalizing concerns            
Biological functioning concerns       -      
Child functioning concerns            
Caregiver characteristics
Caregiver ≤ 8 years   R          
Caregiver < 22 years   -          
Caregiver 22-30 years            
Bio mother 22-30 years     R        
Bio father > 30 years     -        
Bio mother 17-21 years            
Caregiver 31-40 years            
Caregiver ≥ ³ 41 years   -          
No second caregiver in home   -          
Part-time/seasonal income   -     - -
Other benefits/unemployment income   -    
No income   -     -    
No employment              
Substance abuse            
Cognitive impairment            
Mental health issues            
Physical health issues            
Few social supports            
Marital conflicts              
Victim of domestic violence              
Perpetrator of domestic violence              
Caregiver functioning/risk factors    
Ethnic minority              
Asian ethnicity              
Non-English/Non-French primary language         -    
Form of punishment              
Parent uses spanking              
Parent maltreated as a child              
Household characteristics
> 1 child in the home              
Household overcrowded         -
Min one household hazard      
Household level of cooperation              
Household regularly runs out of money   -      
Owned home              
Rental           -
Public housing           - -
Other/unknown housing           - -
Household receives social assistance            
One or more moves   -↑   c
Agency/Referral characteristics
Other/anonymous referral              
Referral by school              
Referral by mother/parent       -      
Regular/emergency service              
Unknown/regular service              
≥ 1 previous case openings        
Case opened for ongoing services              
Referral to parenting program              
Referral to drug/alcohol counselling              
Referral to psychiatric/psychological services              
Any other referral              
Metropolitan location              
1/5 Aboriginal reports              
Proportion of Aboriginal reports              
45%+ investigations Aboriginal families              
Degree of centralization              
Government run agency              
Government run x Proportion of Aboriginal reports              
Minimum education of investigator              
 
Table 3 (Section 2 of 5) Associations between independent, control, and dependent variables in the articles presenting multivariate analyses
Opening for ongoing servicesFootnote 22 Retained cases for Aboriginal childrenFootnote 32 Substantiated maltreatmentFootnote 18 Substantiated riskFootnote 18 SubstantiationFootnote 23 Psychological maltreatmentFootnote 11

-, no statistically significant relationship (p >.05); ↓, statistically significant negative relationship (p <.05); ↑, statistically significant positive relationship (p <.05).

Abbreviations: FAE, fetal alcohol effects; FAS, fetal alcohol syndrome; IPV, intimate partner violence;

Note: Where more than one symbol appears in a cell, that article featured finer grained categories of the variable in question which have been collapsed for concision and those finer categories had different relationships with the dependent variable.

Maltreatment characteristics
Physical abuse          
Sexual abuse          
Neglect          
Emotional maltreatment          
Exposure to IPV          
Indirect physical IPV exposure            
Emotional IPV exposure            
Direct physical IPV exposure            
Co-occurring forms of IPV exposure            
More than one type of maltreatment            
Risk          
Substantiated case            
Physical harm            
Mental/emotional harm            
Duration            
Maltreatment lasting > 6 months          
Male alleged perpetrator            
Child characteristics
Attachment issues          
Intellectual/learning/developmental disability          
Academic difficulties            
FAS/FAE          
Positive toxicology @ birth          
Male sex          
Age 6-15 years -          
Child under 6 years of age          
Child older than 15 years of age          
Aboriginal status            
Physical disability/health issue            
Aggression          
Depression        
Emotional/behavioural issue            
Internalizing concerns            
Externalizing concerns            
Biological functioning concerns            
Child functioning concerns          
Caregiver characteristics
Caregiver ≤ 8 years            
Caregiver < 22 years          
Caregiver 22-30 years            
Bio mother 22-30 years            
Bio father > 30 years            
Bio mother 17-21 years            
Caregiver 31-40 years            
Caregiver ≥ ³ 41 years            
No second caregiver in home       -
Part-time/seasonal income            
Other benefits/unemployment income          
No income          
No employment            
Substance abuse    
Cognitive impairment          
Mental health issues      
Physical health issues            
Few social supports      
Marital conflicts          
Victim of domestic violence        
Perpetrator of domestic violence          
Caregiver functioning/risk factors          
Ethnic minority -          
Asian ethnicity            
Non-English/Non-French primary language            
Form of punishment            
Parent uses spanking            
Parent maltreated as a child          
Household characteristics
> 1 child in the home          
Household overcrowded            
Min one household hazard      
Household level of cooperation            
Household regularly runs out of money        
Owned home   -        
Rental            
Public housing          
Other/unknown housing          
Household receives social assistance            
One or more moves    
Agency/Referral characteristics
Other/anonymous referral          
Referral by school            
Referral by mother/parent          
Regular/emergency service          
Unknown/regular service   -        
≥ 1 previous case openings            
Case opened for ongoing services            
Referral to parenting program            
Referral to drug/alcohol counselling            
Referral to psychiatric/psychological services            
Any other referral            
Metropolitan location -          
1/5 Aboriginal reports -          
Proportion of Aboriginal reports            
45%+ investigations Aboriginal families            
Degree of centralization            
Government run agency            
Government run x Proportion of Aboriginal reports            
Minimum education of investigator            
 
Table 3 (Section 3 of 5) Associations between independent, control, and dependent variables in the articles presenting multivariate analyses
Direct psychological maltreatmentFootnote 11 Emotional neglectFootnote 10 Emotional abuseFootnote 10 PlacementFootnote 9 Placement (2003)Footnote 13 Placement (1998)Footnote 13 PlacementFootnote 14

-, no statistically significant relationship (p >.05); ↓, statistically significant negative relationship (p <.05); ↑, statistically significant positive relationship (p <.05).

Abbreviations: FAE, fetal alcohol effects; FAS, fetal alcohol syndrome; IPV, intimate partner violence;

Note: Where more than one symbol appears in a cell, that article featured finer grained categories of the variable in question which have been collapsed for concision and those finer categories had different relationships with the dependent variable.

Maltreatment characteristics
Physical abuse            
Sexual abuse            
Neglect            
Emotional maltreatment              
Exposure to IPV              
Indirect physical IPV exposure              
Emotional IPV exposure              
Direct physical IPV exposure              
Co-occurring forms of IPV exposure              
More than one type of maltreatment              
Risk              
Substantiated case              
Physical harm         -
Mental/emotional harm    
Duration   - -        
Maltreatment lasting > 6 months              
Male alleged perpetrator              
Child characteristics
Attachment issues              
Intellectual/learning/developmental disability              
Academic difficulties              
FAS/FAE              
Positive toxicology @ birth              
Male sex              
Age 6-15 years              
Child under 6 years of age              
Child older than 15 years of age - -        
Aboriginal status         -
Physical disability/health issue              
Aggression              
Depression              
Emotional/behavioural issue              
Internalizing concerns              
Externalizing concerns              
Biological functioning concerns              
Child functioning concerns   - -        
Caregiver characteristics
Caregiver ≤ 8 years              
Caregiver < 22 years              
Caregiver 22-30 years              
Bio mother 22-30 years              
Bio father > 30 years              
Bio mother 17-21 years              
Caregiver 31-40 years              
Caregiver ≥ ³ 41 years              
No second caregiver in home            
Part-time/seasonal income              
Other benefits/unemployment income              
No income              
No employment              
Substance abuse   - -        
Cognitive impairment              
Mental health issues   - -        
Physical health issues            
Few social supports              
Marital conflicts              
Victim of domestic violence              
Perpetrator of domestic violence              
Caregiver functioning/risk factors              
Ethnic minority              
Asian ethnicity              
Non-English/Non-French primary language              
Form of punishment              
Parent uses spanking              
Parent maltreated as a child              
Household characteristics
> 1 child in the home              
Household overcrowded              
Min one household hazard              
Household level of cooperation        
Household regularly runs out of money   - -        
Owned home              
Rental              
Public housing   -        
Other/unknown housing              
Household receives social assistance              
One or more moves          
Agency/Referral characteristics
Other/anonymous referral              
Referral by school              
Referral by mother/parent              
Regular/emergency service              
Unknown/regular service              
≥ 1 previous case openings   -        
Case opened for ongoing services   -        
Referral to parenting program   - -        
Referral to drug/alcohol counselling   -        
Referral to psychiatric/psychological services   - -        
Any other referral   -        
Metropolitan location              
1/5 Aboriginal reports              
Proportion of Aboriginal reports       -    
45%+ investigations Aboriginal families            
Degree of centralization       -      
Government run agency            
Government run x Proportion of Aboriginal reports             -
Minimum education of investigator       -      
 
Table 3 (Section 4 of 5) Associations between independent, control, and dependent variables in the articles presenting multivariate analyses
PlacementFootnote 19 PlacementFootnote 24 Learning/developmental issuesFootnote 20 Physical disability/health conditionFootnote 20 Emotional/behavioural disordersFootnote 20

R, reference group; -, no statistically significant relationship (p >.05); ↓, statistically significant negative relationship (p <.05); ↑, statistically significant positive relationship (p <.05).

Abbreviations: FAE, fetal alcohol effects; FAS, fetal alcohol syndrome; IPV, intimate partner violence;

Note: Where more than one symbol appears in a cell, that article featured finer grained categories of the variable in question which have been collapsed for concision and those finer categories had different relationships with the dependent variable.

Maltreatment characteristics
Physical abuse R -
Sexual abuse -
Neglect - -
Emotional maltreatment
Exposure to IPV
Indirect physical IPV exposure
Emotional IPV exposure
Direct physical IPV exposure
Co-occurring forms of IPV exposure
More than one type of maltreatment
Risk
Substantiated case
Physical harm -
Mental/emotional harm -
Duration
Maltreatment lasting > 6 months
Male alleged perpetrator
Child characteristics
Attachment issues
Intellectual/learning/developmental disability
Academic difficulties
FAS/FAE
Positive toxicology @ birth
Male sex
Age 6-15 years
Child under 6 years of age
Child older than 15 years of age
Aboriginal status
Physical disability/health issue
Aggression
Depression
Emotional/behavioural issue
Internalizing concerns -
Externalizing concerns
Biological functioning concerns
Child functioning concerns
Caregiver characteristics
Caregiver ≤ 8 years
Caregiver < 22 years
Caregiver 22-30 years
Bio mother 22-30 years
Bio father > 30 years
Bio mother 17-21 years
Caregiver 31-40 years
Caregiver ≥ ³ 41 years
No second caregiver in home
Part-time/seasonal income
Other benefits/unemployment income
No income
No employment
Substance abuse -
Cognitive impairment
Mental health issues -
Physical health issues -
Few social supports -
Marital conflicts
Victim of domestic violence
Perpetrator of domestic violence
Caregiver functioning/risk factors -
Ethnic minority
Asian ethnicity
Non-English/Non-French primary language
Form of punishment -
Parent uses spanking
Parent maltreated as a child -
Household characteristics
> 1 child in the home
Household overcrowded -
Min one household hazard
Household level of cooperation -
Household regularly runs out of money -
Owned home
Rental
Public housing
Other/unknown housing
Household receives social assistance
One or more moves -
Agency/Referral characteristics
Other/anonymous referral
Referral by school
Referral by mother/parent -
Regular/emergency service
Unknown/regular service
≥ 1 previous case openings
Case opened for ongoing services
Referral to parenting program
Referral to drug/alcohol counselling
Referral to psychiatric/psychological services
Any other referral
Metropolitan location
1/5 Aboriginal reports
Proportion of Aboriginal reports
45%+ investigations Aboriginal families
Degree of centralization
Government run agency
Government run x Proportion of Aboriginal reports
Minimum education of investigator
 
Table 3 (Section 5 of 5) Associations between independent, control, and dependent variables in the articles presenting multivariate analyses
Internalizing disordersFootnote 6 Externalizing disordersFootnote 6 Presence of harmFootnote 6 Child functional impairmentFootnote 8

R, reference group; -, no statistically significant relationship (p >.05); ↓, statistically significant negative relationship (p <.05); ↑, statistically significant positive relationship (p <.05).

Abbreviations: FAE, fetal alcohol effects; FAS, fetal alcohol syndrome; IPV, intimate partner violence;

Note: Where more than one symbol appears in a cell, that article featured finer grained categories of the variable in question which have been collapsed for concision and those finer categories had different relationships with the dependent variable.

Maltreatment characteristics
Physical abuse
Sexual abuse
Neglect
Emotional maltreatment -
Exposure to IPV
Indirect physical IPV exposure R R R
Emotional IPV exposure -
Direct physical IPV exposure - -
Co-occurring forms of IPV exposure
More than one type of maltreatment
Risk
Substantiated case
Physical harm
Mental/emotional harm
Duration
Maltreatment lasting > 6 months
Male alleged perpetrator
Child characteristics
Attachment issues
Intellectual/learning/developmental disability
Academic difficulties
FAS/FAE
Positive toxicology @ birth
Male sex - -
Age 6-15 years
Child under 6 years of age
Child older than 15 years of age
Aboriginal status
Physical disability/health issue
Aggression
Depression
Emotional/behavioural issue
Internalizing concerns
Externalizing concerns
Biological functioning concerns
Child functioning concerns
Caregiver characteristics
Caregiver ≤ 8 years
Caregiver < 22 years
Caregiver 22-30 years
Bio mother 22-30 years
Bio father > 30 years
Bio mother 17-21 years
Caregiver 31-40 years
Caregiver ≥ ³ 41 years
No second caregiver in home
Part-time/seasonal income
Other benefits/unemployment income
No income
No employment
Substance abuse
Cognitive impairment
Mental health issues
Physical health issues
Few social supports
Marital conflicts
Victim of domestic violence
Perpetrator of domestic violence
Caregiver functioning/risk factors
Ethnic minority
Asian ethnicity
Non-English/Non-French primary language
Form of punishment
Parent uses spanking
Parent maltreated as a child
Household characteristics
> 1 child in the home
Household overcrowded - - -
Min one household hazard
Household level of cooperation
Household regularly runs out of money - - -
Owned home
Rental
Public housing
Other/unknown housing
Household receives social assistance
One or more moves
Agency/Referral characteristics
Other/anonymous referral
Referral by school
Referral by mother/parent
Regular/emergency service
Unknown/regular service
≥ 1 previous case openings
Case opened for ongoing services
Referral to parenting program
Referral to drug/alcohol counselling
Referral to psychiatric/psychological services
Any other referral
Metropolitan location
1/5 Aboriginal reports
Proportion of Aboriginal reports
45%+ investigations Aboriginal families
Degree of centralization
Government run agency
Government run x Proportion of Aboriginal reports
Minimum education of investigator

Independent and control variables are grouped into five categories: maltreatment characteristics, child characteristics, caregiver characteristics, household characteristics and agency/referral characteristics. One article that used multivariate techniques was not included in the table because the analyses were classification and regression trees that could not be easily summarized in the structure of the table.

Most analyses that used maltreatment types (i.e. physical abuse, sexual abuse, neglect, emotional maltreatment and exposure to IPV) as independent variables included more than one maltreatment type. One study included only maltreatment variables representing different types of exposure to IPV, one included only physical abuse and two included only neglect. In addition, two studies considered emotional maltreatment as a dependent variable.

Researchers frequently used child age and sex as independent or control child-related variables. They tended to use some but not all the variables associated with child functioning concerns in their models, sometimes grouping all child functioning concerns together to create one dichotomous variable representing the presence of any of these concerns. The child functioning concerns span physical, mental, behavioural and developmental problems. Caregiver risk factors, especially substance abuse, mental health issues and lack of social support, were the most frequently used caregiver variables. Household hazards, overcrowding, household frequently runs out of money and frequency of moves were the most commonly used household variables. Finally, one or more case openings and a mother or parent as referral source were the most studied agency/referral variables.

The most commonly studied type of dependent variable, used in six articles, was the opening of a case for ongoing services. Researchers measured the associations of a variety of independent and control variables to do with maltreatment and child, caregiver, household and agency/referral characteristics with this outcome variable. The second most commonly studied outcome variable, used in four articles, was placement in out-of-home care. These articles included numerous variables to do with maltreatment, child, household and agency/referral characteristics. However, only two of four articles were about caregiver risk factors and those used few variables from the category. Other outcome variables that were studied, each in three or fewer articles, included retained cases for Aboriginal children, substantiated maltreatment, substantiated risk of maltreatment, emotional maltreatment, harm to the child, and child functioning concern variables including learning/developmental issues, physical disability/health conditions, emotional/behavioural problems, internalizing disorders, externalizing disorders, child functional impairment and police involvement in investigations.

Discussion

Our review summarized findings from the 32 identified articles on the quality and the relationships between variables in peer-reviewed journal articles using CIS data published since November 2011 and CIS oversample data. The majority of the reviewed articles had clearly stated research aims, used appropriate control variables (where studies were multivariate rather than descriptive), conducted appropriate analyses with sufficient sample sizes, had appropriate conclusions and were relevant to practice or policy.

The majority of the studies did not clearly define variables or provide rationales for the inclusion of the variales within analyses. Because most studies included a large number of variables, this may have been due to word limits imposed by publishers rather than a lack of planned rationale by the researchers. Often articles were judged to have unclear inclusion criteria and sampling methods when the authors had not explained how they selected cases for analysis from the full CIS sample and had provided only an overview of the CIS sampling methods. When the number of participants was considered insufficient, the researchers had usually used smaller subsamples of CIS cases for analysis. One strength of the CIS is that the sample size is large enough to conduct complex multivariate analyses, which is likely why this quality indicator was almost always rated positively. Analyses were frequently appropriate and results clearly presented; however, when they were not, it was usually because authors did not describe which analyses they were performing or did not clearly label tables. When analyses were gauged to be inappropriate to the research question, it was often because researchers used univariate analyses when multivariate approaches would have better addressed their research objectives. Studies that were multivariate in nature nearly always used appropriate control variables to account for potential confounding. Other studies used descriptive analyses to explore new topics, so that hypotheses could be generated and more sophisticated techniques could be used in future research. Conclusions were judged to be inappropriate when they extended beyond the scope of the analyses, such as when authors used causal language to describe an association. When an article was judged to be limited in relevance to practice or policy it was because the application of the findings was not clearly expressed, not necessarily because it had no potential application.

Regarding the associations between variables, some articles explored issues to do with substantiating maltreatment and service provision across the full sample of specific CIS waves or provincial subsamples, whereas other articles were concerned with analyses of subgroups of investigations classified by specific characteristics of children and families or specific characteristics of the investigated maltreatment or risk of maltreatment. Analyses tended to include child maltreatment types as variables when predicting opening for services, placement and child functioning concerns or harm but not when predicting substantiation. It is interesting that presence of harm was also not used to predict substantiation, although observable harm as evidence of maltreatment would presumably make maltreatment allegations easier for workers to substantiate.

This review has highlighted new areas relevant to policy. The multiple disciplines involved in child maltreatment were addressed in an article describing teachers' reporting practices and the response to these by child welfare services.Footnote 23 This expands upon our knowledge of the reporting practices of different disciplines, in the same way that a similar article in the previous review described health care professionals' reporting to child welfare agencies.Footnote 39 In addition, another article was about variables associated with joint police and child welfare worker investigations.Footnote 31

Although changes to the CIS questionnaire are kept to a minimum between cycles, changes have been made to capture changes in practice. For example, CIS-2008 was changed to collect information on risk-only investigations.Footnote 3 One of the articles focussed on untangling risk of future maltreatment from past events of maltreatment.Footnote 18

Uses of the CIS data have evolved over time. In contrast to the previous review,Footnote 7 in which 54% of studies were multivariate, 62.5% of the studies included in this review used multivariate approaches. Despite the increase in use of multivariate techniques, the objectives of most of the included studies did not include clear directional hypotheses as recommended.

The previous reviewFootnote 7  found that physical abuse was the most frequently studied category of maltreatment as a main focus and exposure to IPV was the least. In the present review, nearly all the multivariate articles included either all forms of maltreatment or no forms of maltreatment as independent or control variables. The finding that more papers have focussed on exposure to IPV, emotional maltreatment and neglect is important for policy makers because neglect and exposure to IPV were the first and second most substantiated forms of maltreatment in the CIS-2008 and emotional maltreatment was the fourth.Footnote 3  The three articles that focussed on neglect identified risk factors,Footnote 29 Footnote 33  found a low presence of harmFootnote 29  and found that fathers are less likely to be present in cases of neglect.Footnote 28  The three articles that focussed on emotional maltreatment suggested that the increased specificity of definitions in the CIS-2008 helped differentiate between the occurrence and the risk of emotional maltreatment,Footnote 34  demonstrated that emotional maltreatment often co-occurs with other forms of maltreatmentFootnote 11  and demonstrated that substantiated cases of single form emotional maltreatment are associated with more severe emotional impacts than other forms of maltreatment.Footnote 10  The three articles that focussed on exposure to IPV described the characteristics of cases of exposure to IPV in single form, exposure to IPV and other maltreatment and exposure to other maltreatment;Footnote 25  suggested that exposure to different subtypes of IPV may have different associations with child functioning;Footnote 6  and suggested that investigations of exposure to IPV or hitting only had lower risk-factors and were less likely to remain open compared with other investigations.Footnote 30 

Another notable topic for recent uses of CIS data was investigations involving First Nations children. Five articles explored topics specifically related to First Nations children, including factors influencing overrepresentation at the investigation stage,Footnote 30  placement decisionsFootnote 9 Footnote 13 Footnote 14  and differences from reports on non-Aboriginal children.Footnote 32  Compared to investigations involving non-Aboriginal children, those involving First Nations children had a greater percentage of every caregiver/household risk factor (except health issues) identified by workers.Footnote 30  This could indicate a need for increased availability of family support services for First Nations families; however, the possibility exists that increased risk factors for these families are due to assessment bias.

Strengths and limitations

This review had a number of strengths. To increase the accuracy of judgments, all articles were reviewed by at least two raters who were not the authors of the article being evaluated. We used a standardized quality assessment tool used in an earlier review of CIS data usageFootnote 7 to allow our findings to be compared to theirs. We included recent articles and articles based on oversample data, which broadened our scope.

This review also had a number of limitations. It did not include book chapters, theses, dissertations, government or agency reports, or unpublished manuscripts. It is possible that by excluding these sources we are failing to capture the breadth and depth of the research using CIS and CIS oversample data. We chose to exclude these sources, however, because they are not usually peer reviewed. Although oversample data exists for British Columbia, Alberta, Saskatchewan and the Northwest Territories, to our knowledge analyses stemming from these samples have not been published in peer-reviewed journals. Furthermore, we may have missed some articles from oversampling studies because data requests for these are not necessarily made through the Public Health Agency of Canada. In addition, because we only included published articles, our findings may suffer from publication bias (i.e. statistically significant variable associations may be overrepresented because nonsignificant findings are less likely to be published). Note, however, that most of the articles we included also presented findings of nonsignificant associations between variables. We include these nonsignificant findings in Table 3 to demonstrate the lack of relationships between some independent and dependent variables.

There are also limitations to our findings due to the nature of the CIS data. In reviewing these articles some of the limitations of the data were highlighted. Among these limitations are seasonal variations, the lack of independent verification of the data and the use of proxy-informants. In addition, CIS data only includes children who are reported to and investigated by child welfare agencies. Thus, selection bias may impact the population of children identified in the CIS as some children who experience abuse or neglect (for example, children from low-income families) may be more likely to be the subject of child welfare reports than others.Footnote 3

Furthermore, the criterion validity of the variables within the CIS data may vary. For example, child welfare workers investigating reports of maltreatment or suspected maltreatment can be expected to be better trained and have greater expertise in assessing maltreatment characteristics compared to assessing child and caregiver functioning concern variables as identifying maltreatment would be their primary objective. As previously noted,Footnote 40 child functioning concerns may be underestimated because they are assessed using a checklist of issues known or suspected by the child welfare worker rather than a standardized systematic assessment, which would not be feasible in the study. In addition, not all child functioning items are relevant to different age groups. Analyses using the full age range of investigated children could have restricted ages of those included in the analyses to account for this; however, many did not.

Finally, all variables were measured at the time of the investigation. As such, it is impossible to know whether child maltreatment preceded child functioning concerns; nor can causality be established. These limitations are important for knowledge users and for researchers. The latter could study ways of further improving the quality of the data by considering the limiting parameters of the study.

Conclusions

This review has described the evolving nature of the application of the CIS and CIS oversample data to answer questions about substantiation, placement, provision of services and the impact of maltreatment on child functioning. It is clear that a multitude of factors determine these outcomes. Researchers using CIS data have recently focussed on categories of maltreatment (exposure to IPV, neglect and emotional maltreatment) that were previously inadequately studied, and examined factors specific to First Nations children.Footnote 7 This review has highlighted newly investigated areas relevant to policy. It also suggests that data from the CIS oversamples has generally been underutilized in the peer-reviewed literature. In the future, researchers using CIS data may benefit from this analysis as it identified common pitfalls. The summary of research findings may help researchers identify unexplored topics in the CIS. Future research with CIS and CIS oversample data should continue to utilize sophisticated statistical modelling methods to take advantage of the breadth of information available to address research objectives.

Acknowledgements

The authors would like to thank Dr. Wendy Hovdestad and Dr. Anne-Marie Ugnat for their help in evaluating some of the articles included in the review.

References

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: