Evaluation : Service Canada Community Offices Service Model, December 2009

Introduction

Service Canada Community Offices (SCCO) Service Model was one of four categories of points of service developed to provide service to Canadians; the other three being Service Canada Centres (SCC), Scheduled Outreach Sites and Mobile Outreach Sites. SCCO were points of service, usually in rural or remote locations, that were managed through a contract for services with community partners, or through a collaborative arrangement between Service Canada and a province or territory. The objective of the SCCO Service Model was to provide a service delivery experience for the residents of small and/or rural communities that was nationally consistent, direct, and yet flexibly tailored to the local environment.

Evaluation

In accordance with the evaluation framework and the Treasury Board of Canada Secretariat Policy on Evaluation implemented on April 1, 2009, the evaluation addressed the following core evaluation issues: relevance and performance (effectiveness, efficiency and economy) as well as design and delivery issues. As well, lessons learned were examined during the design and implementation of the SCCO Service Model. The evaluation took place between April 2008 and March 2009. The scope of the evaluation covered the implementation period of the SCCO Service Model of 2007-08. The evaluation included National Headquarters (NHQ), and the provinces and territory where the SCCO Service Model had been implemented (Alberta, Ontario, Nova Scotia, Newfoundland and Labrador and the Yukon).

The Evaluation Framework, which included a logic model and an evaluation matrix, was used to guide the evaluation of the SCCO Service Model. The evaluation utilized five main lines of evidence, integrating results from: a literature and document review; key informant interviews; a client satisfaction survey; site visits; and an operational cost analysis.

Key Findings

Performance (Efficiency, Effectiveness and Economy)

The evaluation found that a number of SCCO locations did not meet the minimum location criteria set out in either the Point of Service Strategy or the Community Partner Service Delivery Strategy. The limited financial resources available to the SCCO Service Model were being used to fund locations that should be served by another service model or to fund locations that are already served by full-service offices.

The evaluation concluded that the SCCO Service Model provided convenient service in the locations where it was offered as well as increased access to self-serve tools and information services in those locations. Although access to SCCO points of service was not consistently available across the country, SCCO met the minimum level of service set out in the generic Statement of Work (SOW) for Click-Call-Visit.

The evaluation found many of the activities and outputs identified in the SCCO Logic Model were used to support the Service Model. However, performance measurement, monitoring and communications practices could be improved to better demonstrate the linkages between activities, outputs and outcomes.

The evaluation found that the SCCO Service Model was an affordable method of providing service to remote and rural communities. Although the Outreach Service Model was less expensive to operate in Alberta and Nova Scotia, it would result in a reduction in the total hours of service being provided to these communities.

Design and Delivery

The SCCO Service Model was deemed to be functioning as designed and was found to be flexible enough to accommodate the needs of the communities being served. The evaluation found that although the design of the SCCO Service Model was suitable with supporting tools and administrative systems, the delivery was characterized by considerable variability. There was a sense that the SCCO Service Model had been "neglected" and suffered from insufficient monitoring and communication.

Lessons Learned

From a lessons learned perspective, training and communication were important to ensure that: community partners and 1 800 O-Canada operators were knowledgeable about the various service models, especially SCC and SCCO; and that all had access to current and approved documentation.

Conclusions

The findings indicate that the SCCO Service Model is making progress towards achievement of expected results and outcomes, and that the outcomes were derived through reasonable means. Additionally, the SCCO Service Model was viewed as a pertinent investment for Service Canada due to its alignment with the Government of Canada priorities.

Recommendations

It is recommended that Service Canada Citizen Service Branch:

1. review and harmonize the categories of points of service and the location selection criteria across regions to ensure national consistency on SCCO Service Model policy and procedures including the identification of service gaps in rural and remote areas and solutions to address these gaps;
2. define roles and responsibilities between NHQ and regions with respect to the management of SCCO including the implementation of a clear reporting structure; and
3. develop and implement a comprehensive performance monitoring system to track the achievement of outcomes to support decision-making on whether individual SCCO should be continued, closed or transitioned into another points of service category.

Overall Comments on Evaluation Report

The Citizen Service Branch welcomes this evaluation as part of our commitment to demonstrate accountable and responsible government. This is our first attempt at measuring the relevance, the performance, the design and delivery of a component of the In-Person Services Delivery Model.

We are pleased that the Evaluation concludes that the SCCO category is:

1. "Responsive to the needs of Canadians";
2. "Aligned with Service Canada's strategic outcome of Service Excellence for Canadians and federal government priorities"; and
3. "An economical method of providing service to remote and rural communities".

We also take note of the different challenges identified in this report regarding the implementation and management of this site category. However, many improvements have been made since 2007-08 to the SCCO category and there has been progress in the delivery of other categories in the overall In-Person model, most importantly in the use of Scheduled Outreach services. Research has demonstrated that in-person service delivery was seen by Canadians to be: "... good for general and personal information, specific questions, serious or complex issues, applications, making payments, seeking guidance/advice, dealing with issues as a last resort, and dealing with all aspects of an issue" (COMPAS Inc. Multi Channel Service Delivery Focus Group Research, 2003). We are currently exploring options to increase services in Scheduled Outreach Sites to meet the expectations of our clients in rural and remote areas for services beyond basic information and pathfinding, while providing optimal balance between affordability, safety and security.

Recommendations:

It is recommended that Service Canada Citizen Service Branch (CSB):

1. Review and harmonize the categories of points of service and the location selection criteria across regions to ensure national consistency on SCCO Service Model policy and procedures including the identification of service gaps in rural and remote areas and solutions to address these gaps.

Management Response: We agree.

The current site categories were introduced at the launch of Service Canada when the network of SCC was expanded to include scheduled outreach, mobile outreach and SCCO. After four (4) years of experience, lessons learned and feedback received from Canadians, it is time to review the Service Delivery Model to see if we can improve our network to better respond to Canadians needs and expectations regarding In-Person Service.

Service Canada has made a commitment to serve at least 90% of the Canadian population within 50 kilometres driving distance of a point of service. To meet this service standard, Service Canada uses a Point of Service Strategy which identifies service gaps across the country, ensures a nationally consistent location criteria for all site categories, and provides strategic direction for the evolution of the overall In Person Service Delivery Network.

Action to be taken:

1.1 Conduct a review of our current suite of site categories to ensure that they respond to the service needs of Canadians, to reflect the evolution of our network in the last four years and, if applicable, to incorporate best practices from other jurisdictions.

1.2 To support Action 1.1, begin to develop transition strategies for identified points of service in each category, starting with the SCCO category.

1.3 Update the Point of Service Strategy to rationalise the location criteria for all recommended site categories.

Planned Completion Date: March 2010

Responsibility: CSB

Planned Completion Date: December 2009

Responsibility: CSB

Planned Completion Date: September 2010

Responsibility: CSB

Recommendations:

2. Define roles and responsibilities between NHQ and regions with respect to the management of SCCO including the implementation of a clear reporting structure.

Management Response: We agree.

Since the creation of the SCCO category, we have improved our practices by developing a Cost Pool Framework and a database for the monitoring of the SCCO contracts. We also analyse financial documents for forecasting purposes and maintain dialogue with regional staff. Although we are reviewing this category it will continue to require sound governance in its operation for the near future.

Action to be taken:

2.1 Update information management to improve the monitoring of SCCO contracts and the forecasting of SCCO costs.

2.2 Update the SCCO Cost Pool Management Framework to clarify roles and responsibilities and implement a reporting structure.

2.3 In addition to sending the SCCO Cost Pool Management Framework to our regional contacts, we will publish it on the Intranet to ensure wider communication to all internal stakeholders.

Planned Completion Date: September 2009

Responsibility: CSB

Planned Completion Date: October 2009

Responsibility: CSB

Planned Completion Date: January 2010

Responsibility: CSB

Recommendations:

3. Develop and implement a comprehensive performance monitoring system to track the achievement of outcomes to support decision-making on whether individual SCCO should be continued, closed or transitioned into another points of service category.

Management Response: We partly agree.

As presented in Recommendation 1, after four (4) years of experience, lessons learned and feedback received from Canadians, it is time to review the model before developing a performance monitoring system for a category that may be transitioned in the near future. We recognise that the monthly reporting done by the SCCO can be minimal. Senior management approved this approach to lessen the administrative burden on small organizations and to minimize costs. We believe that the current reporting is sufficient for a category developed to deliver only basic information and pathfinding services to hard to reach communities.

Action to be taken:

3.1 Develop a Performance Management Framework for the In-Person channel. This would include performance tracking systems based on existing or improved metrics for all site categories of service identified in Action 1.1, including SCCO if applicable, updating monthly report to NHQ and ongoing analysis of performance of each category against desired outcomes.

Planned Completion Date: Fiscal Year 2010­11

Responsibility: CSB

1.1 Rationale

The creation of a Community Partner Service Delivery Model was mandated by the Deputy Head of Service Canada in 2005 and the Community Partner Service Delivery Working Group was created under the direction of two Assistant Deputy Ministers. The working group, comprised of representatives from the regions and National Headquarters, established a framework for the delivery of services by community partner organizations in response to the variety of approaches proposed by the regions on how to expand the Government of Canada's (GC) presence in smaller centers and rural communities.

The Community Partner Service Delivery Strategy was approved by the Service Canada Management Board in February 2006 and the first 12 SCCO commenced operations in March 2006. The Community Partner Service Delivery Strategy contained the following elements: a generic SCCO Statement of Work (SOW) that detailed the roles and responsibilities of service providers and Service Canada; the scope of work for deliverers; and the reporting requirements of service providers to Service Canada. The SOW also contained specific protocols on the posting of Service Canada signage, record keeping and reporting.

1.2 Description and Objectives

As stipulated in the operational policies for the in-person channel, there were four categories of points of service, including the SCCO Service Model.

• Service Canada Community Offices (SCCO) facilitated access to basic information services in communities that could not otherwise be reached by Service Canada staff. Unlike the other service types listed, SCCO were not staffed by government personnel, though the services being delivered supported the policies of Service Canada. In December 2008, there were 64 SCCO operating across Canada.
• Service Canada Centres (SCC) offered a mix of information and transactional services primarily located in urban centres. They operated either as stand-alone sites or were co-located with other organizations. SCC were managed and staffed by Service Canada employees. In December 2008, there were 330 SCC in Canada.
• Scheduled Outreach Sites involved Service Canada staff traveling to pre-determined locations on a regular basis (e.g. one day per week) to provide services that were tailored to the needs of the community (mainly information and transaction support). Of the 212 service points established in December 2008, the majority were located in rural areas.
• Mobile Outreach Sites were usually in rural or remote areas and involved mobile Service Canada staff providing services tailored to the needs of the community (mainly information and transaction support). Service Canada staff traveled to different locations, as requested or as deemed necessary and, if required, their services were communicated via local media.

By December 2008, the SCCO, SCC, and Scheduled Outreach Sites provided 606 points of service to Canadians. The geographical distribution of these points of service throughout Canada is depicted in Map 1.

Service Canada Community Offices (SCCO) were points of service, usually in rural or remote locations, that were managed through a contract for services with community partners, or through a collaborative arrangement between Service Canada and another government entity such as a province or territory. SCCO service delivery involved the following elements:

• service that was provided by persons other than Government of Canada (GC) employees;
• a contractual or collaborative partnership relationship with a non-GC entity; and
• the direct delivery of basic information and referral or pathfinding services to the public.

The objective of the SCCO Service Model was to provide a service delivery experience for the residents of small and/or rural communities in a manner that was nationally consistent, direct, and yet flexibly tailored to the local environment. In doing so, each SCCO established its own hours of operation to best suit the needs of its clientele which enabled Service Canada to partner with service providers that were well-connected and/or well-known in their communities. The generic SOW was developed to ensure that each SCCO provided services in a nationally consistent manner.

Map 1: Geographical Distribution of the 606 Points of Service to Canadians represented by SCCO, SCC, and Scheduled Outreach Sites

1.3 Services provided by SCCO

Although the partners differed for each SCCO, the services provided to citizens included all of the following:

• access to information via the internet, telephone, printed publications and from trained staff;
• Internet navigation assistance across departments and jurisdictions to GC services and benefits;
• referral to specialized expertise as required and/or to SCC for services not provided at the SCCO; and
• coaching on the use of self-serve tools such as 1 800 O-Canada and the Service Canada Internet site.

1.4 Beneficiaries and Partnerships

The main SCCO Service Model beneficiaries were clients seeking personal assistance in obtaining information on government services and programs in the communities where funded SCCO operate. Secondary beneficiaries of the Service Model were the partnering organizations that received service delivery funds enabling them to build their service delivery capacity. The objectives of sustainability, flexibility and development of long-term capacity applied to all potential contractors and community partners, including the voluntary sector.

To enter into a partnership with a community partner, a Request for Proposal was open to all community organizations or private contractors. Partners differed based on the demographics of the community in which SCCO were situated. The SCCO Service Model offered regions the flexibility to choose the most suitable partner. Examples of community partners included Public Library Boards, the municipality, Band Councils in Aboriginal communities, Community Access Program sites and other regional service offerings centers.

1.5 Governance

SCCO were managed through a procurement contract for services or a collaborative arrangement with a community partner. Service or procurement contracts were employed in cases where the partners were non-governmental organizations. These involved payment for services provided and the partner was responsible for all operating costs.

Collaborative partnership agreements, employed in cases where the partnering organization was a provincial government or a Band Council, involved funding transfers and/or in-kind resources. The development of the Community Partner Service Delivery Strategy involved a review of the Treasury Board Alternate Service Delivery (ASD) Guidelines and determined that procurement contracts were the most appropriate mechanism for funding contractors and community partners to deliver Service Canada services.

1.5.1 Role of Service Canada

The Service Canada Service Charter committed Service Canada to provide easy access to government services. To fulfil this commitment, Service Canada developed various Service Delivery models, which include Service Canada Centers (SCC), Scheduled Outreach, Mobile Outreach and Service Canada Community Offices (SCCO).

To enable partners to fulfill their responsibilities under the SCCO Service Model, Service Canada provided one initial training session to SCCO staff at the beginning of the service contract on how to navigate government Web sites and direct clients to information on government programs and services. In addition, signage bearing the Service Canada brand and a supply of GC core documents and forms were provided.

1.5.2 Role of Partnering Organizations

The generic SCCO statement of work stipulated that delivery organizations were responsible for:

• ensuring that SCCO staff have the knowledge, skills and experience required to provide services to clients including navigating Web sites, referring clients to services and programs, coaching clients to use self-serve tools and protecting clients' privacy;
• providing tools and resources to clients such as a list of core government documents and forms, public access computers, telephones and furniture;
• overseeing operating costs, staffing and other capital expenditures; and
• developing a Business Continuity Plan and monthly Client Activity Reports.

1.6 Eligibility Criteria

In March 2007, the Service Canada Management Board approved the principles of an overarching Point of Service Strategy that included baseline criteria for choosing or renewing a SCCO location.

In order to establish a SCCO in a community or to renew an existing SCCO partnership for another term, regions had to meet two key criteria. The first criterion for renewal was related to the location and approval as a point of service based on strategic objectives and a commitment to community presence. The second criterion required that sufficient funds were available within the centrally-managed cost pool. Once these two key criteria had been satisfied, the approved funds were transferred to the region from the In-Person Channel office at National Headquarters.

1.7 Logic Model

The following exhibit presents a draft of the SCCO Logic Model with its associated activities, outputs and outcomes. The SCCO Logic Model was jointly developed by In Person Channel Business Integration staff within the Citizen Service Branch and Service Canada Evaluation. It was designed to illustrate the results chain or how the activities and outputs of the SCCO Service Model are expected to lead to the achievement of the immediate, intermediate and final or ultimate outcomes. The logic model is presented in Exhibit 1-1.

2.1 Purpose of the Evaluation

The purpose of the evaluation was to examine major evaluation issues such as relevance, the extent to which the SCCO Service Model was on track to meeting its stated objectives and outcomes with respect to performance targets, efficiency, economy as well as design and delivery. The evaluation took place between April 2008 and March 2009.

2.2 Scope of the Evaluation

The scope of the evaluation covered the implementation and operational period of the SCCO Service Model of 2007-08. The evaluation included NHQ, and the provinces and territory where the SCCO Service Model had been implemented (Alberta, Ontario, Nova Scotia, Newfoundland and Labrador and the Yukon). There were no community offices located in Prince Edward Island, New Brunswick, Quebec, Manitoba, Saskatchewan, British Columbia, Northwest Territories and Nunavut.

2.3 Evaluation Issues

In accordance with the evaluation framework and the Treasury Board of Canada Secretariat (TBS) Policy on Evaluation implemented on April 1, 2009, the evaluation addressed value for money by examining the five core evaluation issues within relevance and performance (effectiveness, efficiency and economy) as well design and delivery. In addition, lessons learned were examined during the design and delivery of the SCCO Service Model. The questions grouped under each evaluation issue were developed by Service Canada Evaluation in consultation with In-Service Channel Business Integration. These questions, framed around outcomes identified in the logic model, were incorporated into the Evaluation Framework.

2.3.1 Relevance

Is there a continued need for the SCCO Service Model?

To what extent does the SCCO Service Model continue to address a demonstrable need and is responsive to the needs of Canadians.

Is the SCCO Service Model aligned with Government Priorities?

To what extent are the SCCO Service Model objectives linked to: (1) federal government priorities; and (2) departmental strategic outcomes.

Is the SCCO Service Model aligned with Federal Roles and Responsibilities?

Assessment of the role and responsibilities for the federal government in delivering the SCCO Service Model.

2.3.2 Performance (effectiveness, efficiency and economy)

To what extent has the SCCO Service Model achieved its Expected Outcomes?

To what extent has there been progress toward achievement of expected outcomes (including immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes. (Effectiveness - Success).

• To what extent are SCCO providing convenient service across sites?
• To what extent are SCCO providing consistent service across sites?
• To what extent are performance management systems and monitoring strategies supporting the SCCO Service Model in achieving its outcomes?
• Were there any unintended impacts?

To what extent has the SCCO Service Model demonstrated efficiency and economy?

Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.

• To what extent are the SCCO Service Model outcomes being achieved in an affordable manner?
• To what extent are the costs of achieving the program outcomes minimized?
• To what extent are some types of community partners providing better value than others?

2.3.3 Design and Delivery

• What are the variations in SCCO Service Model implementation? What were the challenges and benefits?
• Is the SCCO Service Model design flexible enough to accommodate the needs and socio-demographic profiles of communities being served?
• Are eligibility requirements appropriate and clear? Are eligibility requirements being applied consistently across provinces and territories?
• Is the partner selection process competitive?
• Are the roles and responsibilities between NHQ and the Regions well defined?
• To what extent are partner organizations delivering bilingual services in minority language communities.

2.4 Evaluation Approach and Methodology

The Evaluation Framework for the SCCO Service Model, including the above-mentioned SCCO Service Model logic model and an evaluation matrix, was used to guide the evaluation. The five main lines of inquiry used as sources of information were: a literature and document review; key informant interviews; a client satisfaction survey; site visits; and an operational cost analysis.

2.4.1 Document Review

One of five lines of evidence was the document review. The document review provided structure to the evaluation and background information to assist in the development of other components of the evaluation such as interview guides and cost-effectiveness analysis.

Documents reviewed were foundation pieces for the effective management and structure of the Service Model. They were also used to evaluate delivery and performance management. The document review consisted of, but was not limited to the following:

• Community Partners Service Delivery Strategy;
• Generic Service Canada Community Office SOW and individual SCCO contracts;
• Point of Service Strategy;
• Centrally-Managed Cost Pool Management Framework; and
• Financial and operational information.

2.4.2 Key Informant Interviews

Another line of evidence for this evaluation was key informant interviews. Key informants were selected on the basis of their knowledge and expertise relative to the range of activities associated with design, development, implementation, and monitoring of the SCCO Service Model.

Key informants were selected from three main groups which included: service managers and analysts at National Headquarters; regional coordinators; and SCCO staff. An interview guide was developed for each key informant group, and individuals were interviewed separately. Both in-person and telephone interviews were conducted in each of the key informant's preferred official language. The key informants provided explanations and other in-depth information that served to corroborate or clarify findings from other lines of evidence. Content analysis was undertaken by reviewing responses to each question with a view to identifying areas of concurrence and areas of divergence within and across the three main groups. The interviews were conducted in November and December 2008 with a total of 27 key informants as planned.

2.4.3 Client Satisfaction Survey

Another source of evidence was the SCCO client satisfaction survey. This survey was conducted to determine user communities' satisfaction with the Service Model and to identify design and delivery features that were working particularly well. The following information was collected in the client satisfaction survey:

• number of visits to the SCCO;
• convenience of the location of the office;
• how clients became aware of the office;
• reasons for visiting the office;
• services clients used at the office;
• satisfaction with the assistance received from staff and the services; and
• over all satisfaction with the SCCO.

Clients accessing Service Canada information were invited to participate in the voluntary survey by SCCO staff. Follow-up telephone interviews were conducted with clients. All SCCO in Alberta, the Yukon, Newfoundland and Labrador, Ontario and Nova Scotia were to ask only clients that accessed the Service Canada information and services to participate. A client list was compiled from 27 SCCO. It should be noted that Service Canada Evaluation Division did not receive client names from any Ontario SCCO.

The original plan was to obtain a sample of at least 400 SCCO clients and to complete the survey shortly after individuals had visited the SCCO. However, because of the October 2008 federal election, the survey of individuals who had visited a SCCO in August 2008 did not begin until mid-November 2008. Due to this delay, coupled with a relatively small number of clients' names supplied by the SCCO, Service Canada Evaluation Division asked for another sample in December 2008; these clients were called in January 2009.

For telephone numbers for which there was no answer, several calls were made at different times of the day to reach potential shift workers and others who were not home during the evening. Up to 10 telephone attempts were made with each potential respondent. Interviewers conducted the surveys in either English or French, depending on the preference of the respondents.

2.4.4 Site Visits

Four SCCO were visited as part of the five lines of evidence. On-site observations allowed for in-person key informant interviews with project proponents. In addition, it provided the evaluation team an opportunity to observe the manner in which the services at the location were being delivered. A checklist was used to assess the SCCO and collect information on its facilities, usage, staff competence as well as general comments on the SCCO.

2.4.5 Cost Analysis

The purpose of the costing analysis as one of five lines of evidence was to determine the cost of operating SCCO. In addition to capturing existing differences in the cost of operating SCCO between regions, the cost analysis was also used to compare SCCO to other service models in order to identify advantages and disadvantages of small SCC and scheduled outreach services.

The analysis involved collecting data relating to cost, outputs (such as units of service), service point attributes (such as organization, hours of operation and types of services provided) and service area attributes (such as population served and distance to nearest SCC). Each of these was hypothesized to affect the cost of service delivery in a community, in addition to the Service Model employed there. The costing analysis drew on the following sources:

• monthly Service Canada activity reports;
• data pertaining to the value and length of contracts between Service Canada and each SCCO service provider;
• information relating to service point attributes and service area characteristics for each SCCO and for the community in which it was located;
• SCC annual cost data from fiscal year 2007-08, reporting costs related to salaries, non-salary expenses and rent for four SCC; and
• information relating to Outreach service provision, including accommodation allowances and kilometric rates from the TBS Travel Directive.

Two SCCO were excluded from the dataset used for the cost comparisons because one or more of their features were non-representative of the remaining SCCO. The inclusion of these SCCO was found to distort the results obtained from the costing analyses. The two SCCO excluded were the Calgary Centre for Newcomers and the Banff SCCO. The Calgary SCCO was excluded because the population within 50 km of that location, which included the entire city of Calgary, did not represent the clientele the office was expected to serve; moreover, it was in close proximity to numerous SCC. The Banff SCCO was excluded because client figures were unusually high, making it an atypical site.

2.4.6 Methodology Strengths and Limitations

Strengths

The approach used resulted in a reliable evaluation, which allowed for the provision of evidence to conclude on all issues. In addition, because multiple lines of evidence were used, as well as qualitative and quantitative data, the evaluation methodology was relatively strong. Other factors that contributed to the overall strength of the methodology were:

• the resources allocated to the evaluation were reasonable;
• the key informants and stakeholders contacted were instrumental in ensuring that the evaluation team had all of the information required in a timely manner; and
• individuals contacted were interested in the study and were agreeable to participating in the evaluation interviews and in providing information.

Limitations

Conclusions on the effectiveness of SCCO that provided basic service compared to service models providing full service should be drawn with caution because of factors such as the number of clients seen and served, and the location of the SCCO in regions.

The interpretation of client satisfaction was limited to the sampling technique, its size and distribution. Though random sampling technique was used to collect feedback from SCCO clients, the number of potential respondents to the survey varied widely, with some SCCO providing no names and others providing many. ^Footnote2

In total, 203 out of the planned 400 clients completed the survey with a response rate slightly over 50%. The overall results can be considered to be accurate to ±7%, 19 times out of 20. The interpretation was limited by the range of locations, difference in hours of operation and number of visitors served.

2.5 Out-of-Scope Issues

Although this evaluation addressed issues included in the evaluation framework and the TBS Policy on Evaluation (April 2009) it did not consider the following additional issues that surfaced during the evaluation.

• Although some key informants discussed the idea of expanding services offered by SCCO to include transactions such as Social Insurance Number, Employment Insurance or Canada Pension Plan applications, it was not included in the evaluation.
• Several regions have co-located SCCO in locations providing services under contribution agreements. The evaluation did not consider whether there were controls in place to identify the purposes and application of funds paid out under each instrument in accordance with all applicable policy requirements.
• SCCO locations were compared to the baseline criteria set out in the Point of Service Strategy. However, locations that could be potentially added to the network were not identified.

It is significant to note that, to avoid duplication, the evaluation did not follow-up on the CSB internal Community Partner Service Delivery Strategy study and its 17 recommendations contained therein. Nonetheless, many of the issues contained in this internal study were raised during the conduct of the evaluation. The status of the implementation of the recommendations contained in the Community Partner Service Delivery Strategy study was not considered as part of this evaluation.

3.1 Relevance

4. Conclusions

The SCCO Service Model was created as one of four categories of points of service developed to improve service to Canadians by working with partners to provide access across multiple service delivery channels, including in person, by telephone, Internet or mail. In general, the SCCO Service Model was found to contribute to this objective by offering pathfinding assistance and access to call centres and the Internet in rural and remote communities.

In that the majority of people visiting a SCCO was very satisfied with their experience and was able to obtain the information they were seeking, the SCCO Service Model was deemed to be responsive to the needs of Canadians. Furthermore, the SCCO Service Model was found to align with federal government priorities in that it improved service delivery through community partnerships and increased federal presence. Overall, the SCCO Service Model contributed to the Service Canada strategic outcome of Service Excellence for Canadians by improving service delivery to Canadians.

On the whole, the evaluation found that progress was made towards achieving the immediate outcomes of the SCCO Service Model, specifically: addressing service gaps; improving access and use of government services across communities; and delivering service in a cost-effective manner.

The evaluation found that the SCCO Service Model provided convenient service in the locations where it was offered and increased access to self-serve tools and information services in those locations. Although access to SCCO points of service was not consistent across the country, where they were located, SCCO did meet the minimum level of service set out in the generic SOW for Click-Call-Visit. Progress was made towards providing service in the official languages and, further, towards improved access to government services and information that was easy to understand by offering service in other languages.

In general, access to SCCO points of service was not consistent across the country. The evaluation determined that a number of SCCO locations did not meet the minimum location criteria set out in the Point of Service Strategy. The limited financial resources available to the SCCO Service Model were being used to fund locations that ought to have been served by another service model or to fund locations that were already being served by full-service offices. Improvements in this area would result in increased effectiveness and efficiency and a more consistent level of service across service types.

The findings detailed above indicate that the SCCO Service Model was making progress towards achievement of expected results and outcomes, and that the outcomes were derived through reasonable means. Additionally, the SCCO Service Model can be viewed as a pertinent investment for Service Canada due to its alignment with the Government of Canada priorities.