Evaluation of Ministerial Instructions (Implementation)

1.1 Key stakeholders identify continuing need for the MI

1.2 Stakeholders feedback regarding need to manage application intake

1.3 Consideration and analysis of alternative methods to manage intake (pass mark, etc.)

• Key-informant interviews (Immigration Branch, OMC, International Region, CPR, Legal Services, CVOA)
• Document review (2008 Budget, OAG report, Action Plan, Briefing material, “Pressure notes”)
• CIO site visit

2.1 Stakeholders’ perception of the ability to make adjustments to the system (i.e. controlling the FSW intake with the first set of MI)

2.2 Evidence of regular monitoring and related adjustment as required

• Key-informant interviews (Immigration Branch, OMC, International Region, CPR, Legal Services, CVOA)
• Document review
• Data Analysis (Monthly monitoring reports)
• CIO site visit

3.1 Level of stakeholders and prospective immigrant awareness of the first set of MI

3.2 Stakeholder perception and understanding of MI objectives

• Key informant interviews (Communications, Call Centres, Immigration Branch, International Region, OMC, CPR, CIO, CVOA)
• Document review
• Media analysis/reports
• CIO site visit

4.1 Completeness and accuracy of information provided by clients (application)

• % of incomplete applications (disaggregated by cost-recovery, etc.)
• Number of applications intake (compared to previous years)
• % of non eligible applications
• Number of positive referrals to CVOA that resulted in negative decisions

4.2 Application Intake (CIO)

• Lapsed time for completeness check
• Time for assessment against MI
• Lapsed time between application received and referral to CVOA
• Application volumes/inventory at CIO

4.3 Processing and Final assessment (CVOA)

• Wait time between each step application assessment
• Time between application received at the CIO and final decision at CVOA
• Time between application received at the CVOA and final decision/visa issuance
• Wastage rates and related trends
• Application volumes/ inventory in Missions
• Consistency of CIO and CVOA decisions (applications sent to CVOA where the decisions were reversed)

4.4 Trends in consistency, transparency and timeliness

• Trends in processing times
• Number of complaints and legal challenges
• Perceptions of consistency, transparency, and timeliness of processing

• Data analysis (CAIPS, FOSS – CIO, OMC Stats)
• CIO site visit
• Document Review
• Key-informant interviews (Communications, Call Centres, Immigration Branch, International Region, OMC, CPR, CIO, CVOA)
• CVOA survey

5.1 Number of applications in the MI inventory (CIO and CVOA)

5.2 % reduction of pre-MI backlog

5.3 Evidence of whether the backlog reduction target was met or not

5.4 Stakeholders’ perception of the effectiveness of MI in reducing the backlog

• Data analysis
• Document Review
• Key-informant interviews (CVOA, CIC, CPR, CIO, OMC, Immigration Branch, International Region)
• CIO site visit
• CVOA survey

6.1 Costs to process MI / FSW application (pre/post MI)

6.2 Stakeholders view on possible improvements to the application process

6.3 Evidence of processing efficiency for FSW applications with the CIO compared to previous approach (benefits re: level of effort/cost/fees/transparency)

6.4 Evidence of rationale for CIO (evidence-based decision making)

• Data analysis (Cost-Management Model)
• Key-informant interviews (Immigration Branch, OMC, CPR, Finance, CVOA)
• Document Review (Financial data, CIO documents, budget)
• CIO site visit
• CVOA survey

7.1 Evidence of outcomes not accounted for in original plan and design (positive or negative)

7.2 Stakeholders identify potential processing challenges

• Key-informant interviews (Immigration Branch, International Region, CPR, CIO, OMC, CVOA)
• Document Review
• CIO site visit
• CVOA survey