ARCHIVED – An Examination of the Canadian Language Benchmark Data from the Citizenship Language Survey

Canadian Language Benchmark analyses

As outlined in the original letter of agreement, the chief purpose of this analysis is to examine the relationship between scores obtained on the Canadian Language Benchmark Assessment tool for speaking and listening (the combined version) and an array of factors that might influence second language learning progress. In the sections that follow we will consider each of these factors in turn: country of origin, first language, language training in Canada, formal education in Canada, occupation in Canada, language at work, citizenship test scores, and city where the test was administered.

Country of origin

To facilitate a general analysis, we have created 8 broad geographic categories, as indicated in Table 3 (below). An examination of CLBA scores revealed the highest scoring groups were the South Pacific group (Fiji), followed by Europeans and Central/South America/Caribbean. The area showing the lowest CLBA score was East Asia; this score was significantly lower than the scores from any other region. The largest representation within this category was from China. The totals include only speakers of languages other than English or French.

First language

A factor closely related to country of origin that is more germane to this analysis is mother tongue, or first language. As noted earlier, a very large number of first languages were represented in the sample. However, small Ns in many cases do not permit us to analyze each language separately. For this reason, we reclassified the languages into 20 major categories based on the languages most frequently represented in the sample, genetic relationships among languages, and geographical region. This resulted in 20 categories, summarized in Figure 2.

Figure 2: Mean CLBA score by language group.

When we examine these CLBA scores, we see that the extremes are for speakers of Romance languages with a mean of 7.8 versus Vietnamese/Cambodian with a mean of 3.7. This indicates a wide range of proficiency across these groups. There may well be a variety of influences here. Typological similarity, for instance, may explain why speakers of Romance and other European languages scored high, whereas speakers of many non-Indo-European languages scored much lower. However, this factor cannot fully explain the outcome. In particular, Filipino/Indonesian speakers were among the highest scoring, a finding which may be related to the significant presence of English in such areas as the Philippines and the Pacific Islands. Other factors that may have affected language proficiency include degree of cultural distance, size of compatriot communities in Canadian cities, language teaching methods in the countries of origin and differences in overall levels of proficiency on arrival.

To gain insight into differences among first languages, we examined more closely the CLBA scores for the most frequent five mother tongue categories represented in the sample. Here we find the highest mean score in the Tagalog group (N = 281, M = 7.2), followed by Arabic (N = 222, M = 6.5), Mandarin (N =575, M = 6.1), Punjabi (N = 248, M = 6.0), and Cantonese (N = 289, M = 4.9). An analysis of variance revealed a significant effect of first language on CLBA scores, F(4, 1610) = 36.1, p < .001. Post hoc Bonferroni t-tests (p < .01) indicated that the Cantonese speakers scored significantly lower than any other group, whereas the Tagalog speakers scored significantly higher than all other groups. The other three groups did not differ significantly from each other. The relatively low language scores of the Mandarin and Cantonese speakers are especially striking given that these groups have more representation in the independent immigration class than do the other top 5 mother tongues (79.2% for Mandarin and 53.4% for Cantonese), which suggests a high level of education based on immigrant selection criteria. In fact, when we examine the whole data set, we find that independent class immigrants scored significantly higher (M = 6.8) on the CLBA measures than did either family (M = 6.3) or refugee (M = 6.0) classes, which also differed significantly from each other (Bonferroni, p < .05). Thus the scores of the Mandarin and Cantonese speakers, who comprise a large component of the sample, appear to be inconsistent with other members of the independent class.

It is not possible to draw any firm conclusions about why the Mandarin and Cantonese speakers seem to be at such a disadvantage. Apart from the fact that these languages are typologically distinct from English, educational practices and cultural factors in China and Hong Kong may well play a role. Future research should examine the types of English language training received by these groups before their arrival in Canada to determine how to best meet their oral language needs after they immigrate. It is unfortunate that the reading and writing CLBA scores were not done for these groups because we expect that the Chinese speakers would show greater proficiency in these skill areas than in terms of oral skills. Still another area that merits careful attention is the degree of exposure to English on a daily basis after arrival in Canada. Previous research (Derwing, Munro & Thomson, 2007) has indicated that Mandarin-speaking immigrants to Canada interact less frequently with English speakers than do immigrants from Slavic backgrounds, a tendency that may partially explain their slower oral language development.

Both Hong Kong and Taiwan have experienced more western influence than China. For this reason we undertook a further analysis comparing CLBA scores across participants from China, Hong Kong, and Taiwan. There were no statistically significant differences, in spite of a tendency for the speakers from China to score lower than those in the Hong Kong and Taiwanese groups.

Overall, a significant effect of sex was found, such that the mean CLBA score for males (6.7) was higher than that for females (6.4), t (3825) = 3.49, p < .001. It is obvious that a wide range of other variables interact with sex in determining language proficiency. For instance, it is well known that high academic achievement predicts higher attainment in a second language (Gardner, Polyzoi & Rampaul, 1996) and that education level can often differ between the sexes. However, we do not have information about the participants’ previous educational experiences before arrival in Canada. We also compared male and female CLBA scores for each of the 20 language categories as shown in Table 4.

For several countries, there were no significant differences in language proficiency in English between males and females, but we note that in five instances, females scored significantly lower than their male counterparts. In one instance, the Filipino/Indonesian group, females scored significantly higher than males. We can only surmise that these differences are related to educational practices in the home countries.

Language training in Canada

In the survey, three different sources of language training were specified: Language Instruction for Newcomers to Canada (LINC), fee-based official language training, and high school/college/university official language training. CLBA scores across the three sources differed significantly, F(2,1848) = 171.2, p < .001. Bonferroni t-tests showed that the scores in each training category were significantly different from the others: the high school/college/university category registered the highest mean score (7.2), followed by the fee-based training category (6.1) and LINC (5.0) (see Figure 3). This is not altogether surprising, since LINC is intended for low proficiency learners, and many other language providers design their programs to dovetail with LINC to avoid duplication of services. Thus the other language source categories tend to cater to higher proficiency learners.

Figure 3: Mean CLBA scores by source of language training.

Important differences emerged in the relationship between type of language training and CLBA scores when we looked at data for individual cities across Canada. In Vancouver, the pattern followed the overall Canadian data described above. In Edmonton, LINC CLBA scores were significantly lower than both fee-based and high school/college/university scores; however, the latter two categories did not differ from each other. In Ottawa, CLBA fee-based scores did not differ from LINC scores. Finally, in Montreal, the high school/college/university scores were significantly higher than those of the fee-based programs; the small number of LINC cases (N = 11) precluded any statistical analysis. The disparities across cities may reflect differences in the upper limits of LINC at the time of the study, which were lowest in British Columbia at level 3, followed by Alberta at level 4, and by Ontario at level 5. Thus, there was potentially more overlap in curriculum coverage in LINC and fee-based programs in Ontario. Figure 4 shows CLBA scores for five Canadian cities according to language training source.

Figure 4: Mean CLBA scores for five Canadian cities by language training source.

An additional examination of CLBA scores for participants enrolled in full-time versus part-time language training revealed slightly higher scores for the former group (M = 6.34 vs 6.06). However, this difference was not statistically significant (p = .07). There was a slight tendency for refugees to attend full-time more (68.2%) than members of the family (57.7%) or independent (61.0%) classes. However, this trend was not statistically significant.

Formal education in Canada

Table 5 provides mean CLBA scores according to participants’ formal education in Canada. While a significant difference in scores was observed across the categories [F(5, 1299) = 6.90, p < .001], post hoc Bonferroni tests revealed that this was due to significantly higher CLBA scores for those reporting a university or college diploma as compared with all other categories except apprenticeship training.

Occupation in Canada

We examined the NOC codes and determined that it would be most informative to run an analysis of CLBA scores against the degree of expertise required for various occupations. In an examination of the most recent occupation data, we reclassified all of the occupations into three categories of job skills: Highly skilled, professional and senior management (N = 697); Skilled, technical and middle management (N = 1453); and Low/unskilled, assisting occupations (N = 817). The mean CLBA scores for these groups differed significantly across the categories, F (2, 2964) = 42.3, p < .001.  The scores for highly skilled professional and senior management were significantly higher than the scores of the other two groups, Bonferroni, p < .05. There was no significant difference in the latter groups’ scores. We also considered the most frequently occurring recent occupations. In Table 6, the occupations in which at least 25 immigrants worked are listed, along with CLBA scores in rank order. From this information we see a wide range of average CLBA scores, from 8.48 for registered nurses, to 3.0 for sewing machine operators. As expected, there is a clear relationship between the communication demands at work and the participants’ CLBA scores. It is interesting to note, for example, that bakers (5.0) and cooks (5.0) had lower language skills than food service counter attendants/preparers (6.1). At first glance one might assume that the skill level of these occupations would be similar, but more oral interaction is required at the service counter, which requires a higher level of language proficiency.

Because the survey question on longest held occupation exhibited a very similar pattern of responses, there is little value in providing a separate analysis. It is regrettable that there was no question in the survey about occupation in country of origin. Thus we are unable to determine what percentage of these individuals have reentered their professions, and to what degree language skills have played a role in whether or not they have done so.

We performed one further analysis, in which we examined job skill level according to immigration class. As noted earlier, independent class immigrants exhibit significantly higher CLBA scores than the family and refugee classes. However, this does not appear to afford as much of an advantage as might be expected with respect to the jobs they have obtained. Figure 5 shows that within the independent group there is a larger representation of individuals working in highly skilled positions than in the other groups. Nevertheless, nearly two thirds (66%) had jobs in either the middle skilled or low skilled categories, even though most were selected to come to Canada, in part, on the basis of their superior job skills and formal education.

Figure 5: Job skill level by immigration class.

Although participants were asked whether they were currently employed, it is not possible to tell whether those who answered in the negative were actually in the labour market. Nonetheless, we have carried out an additional analysis of CLBA scores comparing employed versus not employed participants, excluding from the latter category those individuals who reported having dependents at home. This permits us to gain a rough idea of how employment status relates to language proficiency. We found a significantly higher mean CLBA score in the employed group (M = 6.78) compared to the not employed group (M = 6.19), t (1728) = 6.77, p < .001.

Language at work

As noted above, 85% of respondents identified an official language as the one they used most often at work. However, this information is limited in that it does not reveal how much people are required to communicate at work or for which audiences. Nor does it indicate the extent to which the respondents might use more than one language in their workplaces. For example, a sales person might use one language with customers and another with coworkers. What we can surmise is that lower CLBA scores are probably associated with less use of English or French at work. For example, sewing machine operators have little opportunity to interact with others in their jobs, whereas good communication skills are a bona fide qualification for customer service employees. Given the many complexities associated with language use at work, attempting to extract further information from these data is not feasible.

Citizenship test scores and CLBA scores

CLBA scores and citizenship test scores exhibited a small but statistically significant Pearson correlation (r = .391, p < .05). It is unsurprising that language skills should be somewhat predictive of performance on a written test. The effect of language skills on actual pass rates, however, appears to be minimal, given that there were no statistically significant differences among native and non-native speakers of official languages, as noted earlier. Nonetheless, immigration category did exert a significant effect on citizenship test scores, F (2, 1930) = 81.74, p < .001, such that refugees scored significantly lower (M = 17.34/20) than family class immigrants (M = 18.11), who, in turn, scored lower than members of the independent class (M = 19.16).

Differences across cities

Table 7 presents mean CLBA scores across cities. Significant differences were observed between the cities, F (5, 3821) = 37.1, p < .001. Post hoc Bonferroni tests revealed that the scores for Montreal were significantly higher than for all other cities. This finding might be partially explained by the level of language skills of participants on arrival. In Montreal, only 7% of cases who reported English language training took LINC classes (designed for beginner level or lower proficiency speakers of English). In contrast, the percentage in Edmonton was 46. This indicates that, in Montreal, a greater proportion of individuals had some proficiency in English before arriving in Canada. The average CLBA scores from Edmonton were significantly higher than those of the other English-speaking cities. In part, these differences may be explained by the relative proportions of refugee, family and independent class immigrants represented in each city’s data. Whereas Montreal has a majority of independent class respondents (61%), thus increasing the overall CLBA mean, Ottawa has a disproportionately large representation of refugees (27%) and Edmonton’s refugee percentage (14%) is the smallest of the five cities involved.

• Previous
• 1
• 2
• 3
• 4 (current)
• 5
• 6
• 7
• 8
• Next