Evaluation of the Grain Inspector Development Program
On this page
Executive Summary
This report presents the results of a program evaluation of the Canadian Grain Commission’s Grain Inspector Development Program (GIDP). The Grain Inspector Development Program is an internal training program for grain inspectors. This program was selected for evaluation as it is considered a key control activity for the successful fulfillment of the Canadian Grain Commission’s mandate as set out in the Canada Grain Act and its Quality Assurance Program’s objectives.
The evaluation was conducted in accordance with the Treasury Board’s Policy on Results, which requires departments to measure and evaluate performance and use the resulting information to manage and improve programs, policies and services. The Audit and Evaluation Services team of the Canadian Grain Commission conducted this evaluation during the 2022 to 2023 and 2023 to 2024 fiscal years, as part of the Canadian Grain Commission’s Risk-Based Audit and Evaluation Plan.
Evaluation findings and recommendations are intended to inform current and future program and policy decisions. This is the first time this program has been evaluated independently by the Canadian Grain Commission’s Audit and Evaluation Services team.
Methodology
This evaluation assessed the program’s relevance, performance and efficiency. To ensure reliability and validity of data and information, the evaluation used multiple lines of evidence and triangulated findings to support the evaluation. The evaluation team investigated multiple lines of evidence, including program information review, document review, existing data, key informant interviews and an anonymous online trainee survey.
Key findings and recommendations
Details of findings, recommendations, and management action plan responses can be found in Appendix 1.
| Key finding | Recommendation |
|---|---|
| Relevance | |
| There is a continued need for the program to support CGC service delivery and to support external stakeholders. Evaluator was unable to identify any other training programs offering a similar level of training. | Continue to improve the program with the following recommendations for improvement and or enhancement. |
| Program aligns with government priorities roles and responsibilities. | Continue to improve the program with the following recommendations for improvement and or enhancement. |
| Performance | |
| Overall, those surveyed and interviewed felt that the program could be shortened to potentially increase graduation rates to support grain inspection without compromising the quality of training. | Explore shortening the program based on the evidence-based potential opportunities outlined. |
| The main reason trainees exit the program is resignation. Key informants reported that this was primarily related to factors external to the program. The nature of the work, the hiring process, and compensation were the main factors in resignations, rather than the training program itself. Key informants felt that recruitment processes could better provide candidates with information about the realities of the work and the work environment. | Update GIDP recruitment related outreach (external to the program) to incorporate a more realistic description of the work and the work environment. Examine new ways to appeal to and engage potential best fit candidates. Continue to value previous experience such as industrial and or agricultural related. Improve program information on the Canadian Grain Commission website. Targeted recruitment and building awareness of this program online and using social media is underway. |
| Analyze GIDP trainee retention data to better understand at which stage of the program GIDP candidates are exiting the program. Explore and further investigate the loss of GIDP candidates including the already highlighted potential link between compensation and loss of GIDP candidates. | |
Trainees that responded to the survey felt that more time and attention should be given to waterfront and shipping duties. In addition, the majority of inspectors and the specialists/operations supervisor groups identified 2 areas for improvement:
| Add structured modules and formal assessment of waterfront duties, including vessel loading scenarios, shipping and export duties (including export tolerances in addition to inward tolerances), administration, paperwork, forms and use of OSCAR, which may vary by terminal. |
|
Once trainees graduate from the program, they often transition into a leadership or mentor role in operations work and begin to mentor trainees. Although they lead/mentor others, some do not always receive formal supervisor training. Internal stakeholders working directly with trainees identified a need for increased focus on behavioural competencies and additional training in leadership, supervising, and soft skills. |
Addition of training modules in the areas of mentoring, communication and feedback to enhance support for the transition of GIDP graduates to mentors and leaders. Ensure GIDP graduates who are in a supervisory role receive formal supervisory training. |
| Program Efficiency & Enhancement | |
| Overall, trainees feel supported by their trainers and are happy with mentoring received on the waterfront. The program could be improved by adding more trainers overall and more trainers in both official languages. There is little in the way of formal training offered to program trainers. | Continue to hire additional trainers in both official languages (this is currently underway). |
| Add formal training for trainers in the areas of training, and learning. If needed, add custom training modules for ongoing refresher training. | |
| Inconsistency and lack of clarity in documentation. All of the pieces exist but their use may vary by location. Program could benefit from compiling one competency map for clarity and consistently. | Re-organize existing program documents into a competency map, filling in any gaps where necessary. This document should outline the required competencies, the criteria for assessing the competencies, and the assessment tools used for each. The competency map should give all parties more clarity to ensure consistency regarding competency and proficiency needed to advance. The competency map should reflect recommended updates implemented as part of this program evaluation. |
| The evaluator found evidence to warrant a comprehensive review of the terms of the Grain Inspector Development Program Framework and Roadmap. Some program practices may have evolved, and the impact of emergent technologies may have created new opportunities. Recommendations made within this evaluation may necessitate review and update of the governance documents. |
Review and update program governance documents focusing on these additional areas that have not already been covered in previous MAPs:
|
| Program does not collect regular feedback from trainees and internal stakeholders for the purpose of continuous program improvement. | Implement a formal feedback mechanism to collect feedback from trainees and inspectors for purposes of continuous program improvement. A confidential feedback mechanism will ensure that everyone has the opportunity to provide feedback. |
| Program evaluation identified further opportunities to update and modernize training materials while acknowledging the significant recent progress and high level of satisfaction with training materials. | Continue to modernize training materials (including self-directed materials) and explore use of new training technology. Update crop samples and provide full set of training kits for each permanently staffed location. |
| Evaluator found a lack of easily available program data. | Assign additional administrative support to develop and maintain a comprehensive program database to further strengthen program oversight and decision making. System to provide up-to-date, easily accessible student information including trainee retention and loss. Given the other recommendations including modernization and program organization there may be benefit to considering the use of a learning management system. |
Conclusion
The program evaluator found evidence of continued need for the program, the program is being utilized and aligns with Canadian Grain Commission priorities, roles and responsibilities. This evaluation contains recommendations for program improvement in the areas of performance and efficiency that require a management action plan response.
1.0 Introduction
1.1 Evaluation purpose
This report presents the results of a program evaluation of the Canadian Grain Commission’s Grain Inspector Development Program (GIDP). The Grain Inspector Development Program is an internal training program for grain inspectors. This program was selected for evaluation as it is considered a key control activity for the successful fulfillment of the Canadian Grain Commission’s mandate as set out in the Canada Grain Act and its Quality Assurance Program’s objectives.
The evaluation was conducted in accordance with the Treasury Board’s Policy on Results, which requires departments to measure and evaluate performance and use the resulting information to manage and improve programs, policies and services. The Audit and Evaluation Services team of the Canadian Grain Commission conducted this evaluation during the 2022 to 2023 and 2023 to 2024 fiscal years, as part of the Canadian Grain Commission’s Risk-Based Audit and Evaluation Plan. The evaluation focused on the following areas:
- relevance
- performance
- efficiency and enhancement
In addition, the Business Process Owner identified the following areas of interest:
- recommendations regarding current program length; including opportunities to streamline the program without impacting quality of training outcomes
- recommendations regarding the merit-based practice of “fast tracking” or shortening the program length for students who excel
- recommendations regarding aligning current training practices to localized differences (currently training remains the same regardless of geographical location)
- recommendations regarding the vulnerabilities of the program regarding attrition both prior to completion and after graduation and the resulting loss in investment in training and HRFootnote1
Evaluation findings and recommendations are intended to inform current and future program and policy decisions. This is the first time this program has been evaluated independently by the Canadian Grain Commission’s Audit and Evaluation Services team.
1.2 Evaluation scope
The evaluation considered data and information for the period between program launch in January 2014 and 2021 to 2022 fiscal years while also making use of more recent learnings where applicable.
1.3 Background
The Canadian Grain Commission’s mandate as set out in the Canada Grain Act is to, “in the interests of the grain producers, establish and maintain standards of quality for Canadian grain and regulate grain handling in Canada, to ensure a dependable commodity for domestic and export markets.”
CGC’s Inspection Program
The Canadian Grain Commission’s grain inspectors play a critical role in delivering on the Canadian Grain Commission’s mandate. They are responsible for ensuring that all grain exported from Canadian ports is safe, reliable and marketable and for providing services for Canadians and Canadian grain producers through our Quality Assurance Program.
Official grain inspection is required by the Canada Grain Act for all grain being exported from Canada by ship and must be inspected by the Canadian Grain Commission. Shipments to the United States are exempt from this requirement. As the terminal elevator loads each vessel, a representative from the Canadian Grain Commission:Footnote2
- collects an official sample every 2000 metric tonnes during the course of loading
- performs an inspection on every 2000 tonne increment sample
2.0 Program profile
2.1 Program activities
The primary objective of the Grain Inspector Development Program is to ensure that the Canadian Grain Commission attracts, develops and retains the talent necessary to meet its grain inspection duties and fulfill its mandate under the Canada Grain Act.Footnote3 Staff enter the program through a competitive hiring process. The program is designed to graduate fully trained PI-03 level inspectors after 4 years of practical training. It is not possible to remain indefinitely at the PI-01 or PI-02 level.Footnote4
2.1.1 Program logic model
A logic model is a visual roadmap of how the program is expected to achieve its intended outcomes. It illustrates the intended causal relationships between the program activities, outputs and outcomes. The program logic model in Figure 1 was developed jointly by the evaluator and the evaluation working group. It outlines the program activities and outputs as well as immediate, intermediate and long-term outcomes.
Figure 1: Grain Inspection Development Program logic model
Note: intake/hiring decisions are made externally to the program – Director IS and Regional Management Team makes these decisions
Note: not all inspectors trained through the Grain Inspector Development Program; some completed their training prior to the implementation of the program in 2014
2.2 Program description
Trainees joining the program are hired into entry-level PI-01 positions. Trainees advance through the program to PI-02 and finally PI-03 level grain inspectors without the need for further staffing processes. Trainees remain on probationary status for the duration of the program, typically spending 2 years at the Trainee PI-01 level and another 2 years at the Intermediate PI-02 level. The program must be completed within 4 years.
The program tracks and facilitates trainee progress in meeting the Industry Services Inspection Technical and Behavioural Competencies.Footnote5 Each trainee has a personalized learning plan that outlines the competencies the trainee must develop. Training is conducted by the program manager and trainers, with onsite training and mentorship support from experienced grain inspectors. Training cycles between in-class study, review, and assessment and hands-on learning in the work environment.
Self-study is a key component of the program. Trainees are expected to act on their own initiative to advance their own learning. Advancement is based on achievement of the necessary competencies. Motivated trainees can potentially move through the program more quickly than the maximum 48 months.
Formal assessment methods include practice tests, and barrier exams at years 1 and 2. Barrier exams are typically completed in months 12 and 24. In order to advance past these barrier tests and move on to the next year of the program, trainees must achieve:
- a grade of 70% in the barrier test (with 1 reassessment offered to those who score between 60% and 69.9% on their first attempt)
- ratings of “operational” for technical proficiencies and “good” for behavioural proficiencies
Trainee development is periodically tracked through performance learning agreements. This gives trainers and trainees an opportunity to identify which competencies need additional development. Participants also receive continual feedback from grain inspectors, supervising grain inspectors, specialists and trainers.
Trainees are assessed for graduation by their team of trainers, mentors, and managers. This team meets when the trainee is in month 40 (on average) of the program to discuss how the participant is progressing. This team meets again at month 48 to discuss each participant’s overall performance in the program to determine if the trainee should graduate to the level of a PI-03 grain inspector.Footnote6 Because hiring into the program is targeted to meet operational needs, all successful participants are promoted to the PI-03 level.
As of end of the 2021 to 2022 fiscal year, the program has graduated 29 fully qualified grain inspectors.
2.3 Program resources and funding
The program manager is housed within the Industry Services division and reports to the Chief Grain Inspector, Industry Services. Additional administrative staff, trainers, Human Resources staff and Inspection staff support the program. At the time of this evaluation there were a total of 7 trainers: 2 in the Western Region, 3 in the Eastern Region, and 2 in Winnipeg. Trainees are also supervised and mentored by grain inspectors, supervising grain inspectors, inspection specialists/operations supervisors, and regional managers housed outside of the program but within the Industry Services division.
The GIDP and official grain inspection are closely linked and difficult to separate, as inspection staff are heavily involved in mentoring trainees and providing feedback. Trainees are also a key component of inspection service delivery. Several key decisions affecting the Grain Inspector Development Program are made outside of the program, including: hiring, budgeting, succession planning, staffing decisions, advancement, scheduling, travel and disciplinary actions.
2.4 Program budget
Program costs are not tracked under a separate cost centre. For purposes of this evaluation, the Industry Services division estimated program costs making an effort to allocate costs between service delivery and training. This estimate includes a portion of trainee salaries, trainer salaries, and travel costs for training purposes and training materials.
This high-level estimate indicates that the program has cost approximately $8 million over a 9-year period. Annual costs in the most recent year included in the evaluation (2021 to 2022 fiscal year) were approximately $1 million. Detailed estimated historical program costs are found in Appendix 2.
3.0 Methodology
This evaluation used multiple lines of evidence and triangulated findings to ensure reliability and validity of data and information. The evaluation team investigated the following lines of evidence:
- program information review
- document review
- existing data
- key informant interviews with trainers, internal and external stakeholders
- trainee survey
The review of existing data and documents included internal documents, relevant legislation and guidance documents, reports and articles.
New data was collected via key informant interviews and an email survey sent to all trainees.
For a complete list of data sources and additional insight into the information gathered, refer to Appendix 3: Methodology.
4.0 Evaluation findings
4.1 Relevance
4.1.1 Continued need for the program
The program evaluation found significant evidence from multiple sources that the program continues to be well utilized and that there is a continued need for the program at the Canadian Grain Commission.
Trainees, trainers, inspectors, specialists/operations supervisors, and external stakeholders all consistently reported that the program is working well and accomplishing its stated objectives. All key informant groups consistently felt that there is a continued need for the program.
Overall, trainees surveyed reported that they were well prepared for their assessments and were fully prepared for the duties they perform at the waterfront.
External stakeholder views were collected via a review of past public consultation submissions and key informant interviews directly with stakeholders. Several external stakeholders noted interest in seeking training assistance and support from the Canadian Grain Commission’s program in the future. Many of the external stakeholders that were interviewed reported that their internal staff training programs were based on the Canadian Grain Commission’s materials, external website, and/or former or retired employees who developed and/or delivered grain inspection training to their employees. Internal and external stakeholders interviewed also felt that the Grain Inspector Development Program is unique and does not duplicate any other programs or initiatives.
Finding: There is a continued need for the program to support Canadian Grain Commission service delivery and to support external stakeholders. The evaluator was unable to identify any other training programs offering a similar level of training to the Canadian Grain Commission’s.
Recommendation: Continue to improve the program with the following recommendations for improvement and or enhancement.
4.2 Alignment with government priorities, roles and responsibilities
The program supports Canadian Grain Commission mandate and federal government roles and responsibilities.
The Canada Grain Act is the federal legislation that defines and outlines the roles and responsibilities of the Canadian Grain Commission. The Canadian Grain Commission’s mandate, as outlined in section 13 of the Act, is to:
“…in the interests of grain producers, establish and maintain standards of quality for Canadian grain and regulate grain handling in Canada, to ensure a dependable commodity for domestic and export markets.”
The Canadian Grain Commission’s 2021-2022 Departmental Results Report identifies the organization’s Core Responsibility as:
“… domestic and international markets regard Canadian grain as dependable and safe and that farmers are fairly compensated for their grain.”
The Grain Inspector Development Program directly supports the mandate and core responsibility. Trainees support inspection service delivery both during their training and upon graduation as fully qualified grain inspectors, ensuring Canadian grain is dependable and safe for domestic and export markets.
Through its training and mentorship approach and interaction with the inspection program, the Grain Inspector Development Program also supports the Canadian Grain Commission’s following key areas of focus as identified in the 2023-24 Strategic Plan.Footnote7
- Modernize the Canadian Grain Commission regulatory framework programs, and services
- Position the Canadian Grain Commission as a global leader in grain science
- Strengthen the Canadian Grain Commission’s stakeholder relationships, with a focus on Canadian producers
- Attract and retain employees in a competitive market
Finding: Program aligns with government priorities roles and responsibilities.
Recommendation: Continue to improve the program with the following recommendations for improvement and or enhancement.
The following section of the report identifies areas for program improvement and or enhancement.
4.3 Performance
4.3.1 Inspection program is sustained and supported by the Grain Inspector Development Program
Overall, key informants felt that the program struggles at times to provide a sufficient number of fully trained inspectors in some regions/areas to meet current and future demand for Canadian Grain Commission trained inspectors. There is a risk that this may lead to shortages of fully trained inspectors in some areas and during the peak grain-shipping season and lead to increased use of overtime. When queried, external stakeholders overall did not have enough familiarity with the training program nor indicated any impact in service.
The ability of the program to provide sufficient graduates is impacted by the following:
- the number of new hires into the program
- upcoming retirements
- the suitability of new hires
- program length
- the retention rate
The majority of these factors are not within the control of the program. Two main factors identified, that are within the control of the program, include program length and the retention rate.
4.3.1.1 Program length
Almost all of the trainees surveyed, and the majority of trainers, inspectors, and specialists/operations supervisors interviewed, felt the program could be shortened from the current 4-year period without compromising the quality of training outcomes by moving trainees through the steps more quickly avoiding skipping any critical competencies. A small number of respondents felt that the full 4 years are needed.
Overall finding: Overall, those surveyed and interviewed felt that the program could be shortened to potentially increase graduation rates to support grain inspection without compromising the quality of training.
Overall recommendation: Explore shortening the program based on the evidence-based potential opportunities outlined below.
Opportunity 1: Competency-based advancement
Key informants reported that some trainees could be fast-tracked by allowing trainees to advance at their own pace once a competency is mastered, rather than on a set schedule. Those who excel at both behavioural and technical competencies or those with a background in agriculture or other relevant experience may be able to move more quickly through the program milestones. Currently, trainees are selected for fast-tracking based on their performance or previous experience. The internal processes for this decision making are not well defined or communicated.
Evidence from key informant interviews supports the view that trainees progress at different rates and that the program should accommodate this. Any opposition to this process was based on a lack of transparency and clarity about how fast-tracking works. Key informants (internal stakeholders) noted fast-tracking must not skip any steps. Competency-based advancement allows trainees with sufficient performance/proficiency or previous experience to advance when they are proficient.
Program legacy documents support a competency-based, fast-tracking approach. Advancement is based on merit and timelines are flexible, depending on the region and skill level of each participant.Footnote8
Observation: Trainers unanimously support the practice of fast-tracking. Inspectors and Specialists/Operations supervisors also expressed an almost unanimous level of support.
Potential opportunity: Explore a move to competency-based advancement that expands the current practice of fast-tracking to allow interested trainees to apply for advancement. Develop a clear and transparent process for application and guidelines for eligibility. Ensure that program steps are not skipped but that trainees are permitted to advance more quickly through the milestones when appropriate.
Opportunity 2: Tailoring the program to specific regions
Key informant groups gave mixed responses when asked about this practice. Many supported this approach or felt that it was already happening in practice due to the differing commodities, schedules, speeds of inspection, etc., that are workplace realities in different locations. The minority did not support this practice or felt it has a downside because the Canadian Grain Commission loses the flexibility to transfer staff between regions.
While trainers, inspectors and specialist/operations supervisor groups were divided on this issue, 81% of inspectors in the Eastern Region favoured tailoring the program based on crop type. This would allow trainees to act for corn and soybeans only. Corn and soybeans are considered less challenging to grade when compared with other grains such as wheat. Potential drawbacks to tailoring the program include losing the ability to transfer staff between regions and equity/compensation issues. The potential benefit would be the ability to expedite training and reduce staffing shortages. Inspectors in the Western Region were less supportive of this option, with only 44% supporting it.
Observation: There is some support to tailor the program to specific reasons for the purpose of expediting acting inspectors in the Eastern Region for corn and soybean inspection only.
Potential opportunity: Explore phased training in the Eastern Region for corn and soybean grading. Trainees would still complete the entire program, but also qualify to act as inspector in the interim for only corn and soybean shipments until they complete their training on all 21 regulated grains.
Opportunity 3: Use of Grain Inspection Assistants
Another common theme from key informants was to use grain inspection assistants during the peak grain shipping season to allow trainees to focus on training and mentoring on the waterfront. To date, grain inspection assistants have only been used in the Western Region. Trainer views on this topic were mixed. Inspectors supported this practice. The specialist/operations supervisor group was also split.
Overall, the majority of key informants felt that the addition of grain inspection assistants during the busy season lessens the impact of the busy season on training. Key informants reported that training is impacted negatively during the busy season due to staffing shortages and increased workloads, and that the use of grain inspection assistants can help mitigate that impact. Key informants who were opposed to this practice tended to cite the Canadian Grain Commission’s need for fully qualified grain inspectors rather than those at a PI-01 level. Others felt there is a need and a role for career PI-01 level staff.
Key informants also reported that grain inspection assistants can be a good source of candidates for the Grain Inspector Development Program, and could be used as a stepping stone into the program. Hiring temporary, 90-day grain inspection assistants allows both the Canadian Grain Commission and the candidate to assess interest and suitability prior to entering the Grain Inspector Development Program.
Observation: There is some support for the continued use of grain inspection assistants as a feeder to the Grain Inspector Development Program and as seasonal support. Key informants indicated that the presence of grain inspection assistants allowed trainees to focus on training and mentoring on the waterfront.
Potential opportunity: Consider limited use of grain inspection assistants.
Opportunity 4: Program intensity
Key informants tended to support incorporating additional formal training sessions to shorten the program. Internal stakeholders felt that the level of self-directed learning and initiative displayed by trainees has declined over time. Suggestions to address this include the following:
- more structured use of downtime and supervision at the waterfront to intensify the program
- encouraging self-study of the Official Grain Grading Guide
- more communications between trainers and waterfront staff
- more trainer presence on the waterfront
- implementing e-learning/virtual options
The specialist/operations supervisor group expressed the view that while technical training could be expedited, continuing to ensure sufficient waterfront experience was a priority.
Observation 1: Key informants felt that the program length could be reduced by incorporating additional formal training sessions during downtime between vessel loading and during the off-season. Key informants suggested the program should rely less on trainees using their downtime for self-study and focus more on increasing program intensity with more program structure and supervision. This was more of a concern in the Western Region than the Eastern Region.
Potential opportunity 1: Increase the number of trainers with a focus on implementing the following changes:
- More supervision and structured learning during downtime on the waterfront
- Assigned work and required modules available to trainees for self-directed learning
- Focus on initiative and work ethic, as trainees do not always use downtime effectively without supervision or assigned study; key informants expressed that more structure is needed
- Increased trainer presence on the waterfront, with trainers travelling more frequently to sites that do not have a dedicated trainer
- Dedicated scheduled sessions with inspectors and specialists
- Explore expanding training beyond only day shifts, being mindful of the negative impact of graveyard and afternoon shifts on training; this may not be feasible with current training resources
Observation 2: Trainers and Inspectors interviewed were split as to whether the busy season has a negative impact on training. Over 75% of trainees surveyed felt that training was negatively impacted during the busy season. Those in the Eastern Region reported a greater impact when compared with the Western Region. During the busy season, trainee advancement was sometimes slowed or impacted by higher workloads, less availability of mentorship, and rescheduling of formal training.
Potential opportunity 2: Advance planning and scheduling of training during the off-season to mitigate this impact. Focus on formal training during the off-season and vessel loading/service delivery during the busy season, with a renewed focus on making mentors available to trainees. Limit onboarding new trainees during the busy season.
Observation 3: Formal training is consistent amongst trainees while the practical, on-the-job training can vary due to unexpected incidents and by region, shift, elevator, weather, commodity, schedule, and operational demands. Scheduling can also impact time spent with the trainee’s supervising PI-04 and time rotation to different ports (exposure to different commodities and environments). Graveyard and afternoon shifts were noted to have a negative impact on learning.
Potential opportunity 3: Renew focus on scheduling to optimize training. Rotate trainees between different locations and ensuring trainees are scheduled with their PI-04 supervising grain inspector as much as possible. The negative impact and limited availability of training on graveyard shifts and afternoon shifts should be considered during scheduling.
Observation 4: Trainees receive similar training opportunities and have different levels of initiative and use of downtime and self-study on the waterfront.
Potential opportunity 4: Increase program intensity, structure and supervision to encourage all trainees to make good use of self-learning opportunities.
4.3.1.2 GIDP trainee retention
Trainee retention also influences the program’s ability to sustain and support inspection. Trainee retention is a key factor in graduating enough qualified inspectors to meet service delivery demands.
Key informants interviewed reported that trainees leave primarily due to the nature of the work and, more specifically, misconceptions about the nature of the work. This was consistent across all key informant groups. Misconceptions about the nature of the work lead to hiring trainees who leave the program because they are not well suited to the work or the work environment.
Continuing to improve transparency around the nature of the work and the work environment in hiring processes and more up-front communication and exposure (videos, etc.) to the waterfront environment and the nature of the work would help prevent this. One option may be to target candidates with previous industrial or agricultural experience to help select “best fit” candidates for the program. Recruiting at high schools and job fairs may help broaden the pool of prospective trainees. Online targeted ads and social media could be used to increase overall awareness of the program.
Inadequate compensation was also reported as a reason that trainees leave the program, with some candidates leaving for better-paying positions within the federal government or other sectors. Over half of the trainees surveyed (55%) felt their compensation was low compared to other opportunities available to them. This may differ by location of the terminal.
Data provided by the program manager indicate retention rates for the program over the life of the program are approximately 60% in the Eastern Region and 73% in the Western Region (including Winnipeg). Trainee loss from the program is primarily due to resignations. It is unclear at what stage of the program resignations are occurring.
Employee departures from the program were compared against the Treasury Board Secretariat of Canada’s Demographic Snapshot of Canada’s Public Service, 2021.Footnote9 This snapshot provides a comparison of typical government-wide retention rates. Over the life of the program, 30% of trainees in the Eastern Region and 20% in the Western Region (including Winnipeg) were lost to resignation, including those who moved to another position with the Canadian Grain Commission. Only 4% of candidates have exited the program due to failed barrier exams since the program launched.
Finding: The main reason trainees exit the program is resignation. Key informants reported that this was primarily related to factors external to the program. The nature of the work, the hiring process, and compensation were the main factors in resignations, rather than the training program itself. Key informants felt that recruitment processes could better provide candidates with information about the realities of the work and the work environment.
Recommendation a: Update GIDP recruitment-related outreach (external to the program) to incorporate a more realistic description of the work and the work environment. Examine new ways to appeal to and engage potential best-fit candidates. Continue to value previous experience such as industrial and/or agricultural related. Improve program information on the Canadian Grain Commission website. Targeted recruitment and building awareness of this program online and using social media are underway.
Recommendation b: Analyze GIDP trainee retention data to better understand at which stage of the program GIDP candidates are exiting the program. Explore and further investigate the loss of GIDP candidates including the already highlighted potential link between compensation and loss of GIDP candidates.
4.3.2 Graduates demonstrate sufficient technical and behavioural competencies
The program focuses on technical and behavioural competencies. Technical competencies are developed via a series of learning modules and mentorship on the waterfront. Behavioural competencies are developed primarily via mentorship on the waterfront. All informant groups were asked to identify any gaps in the program or any competencies that needed more focus. Almost 60% of respondents felt there were some competencies that needed more focus. A review of responses from trainee surveys and key informant interviews indicated the following.
Finding: Trainees that responded to the survey felt that more time and attention should be given to waterfront and shipping duties. In addition, the majority of inspectors and the specialists/operations supervisor groups identified two areas for improvement:
- Add formal training modules and assessment of waterfront, vessel loading and shipping duties, including administration and paperwork, forms, OSCAR, (many of which can vary by terminal).
- Add more structure and increase program intensity to make the best use of downtime on the waterfront and shorten the program. Inspection staff noted an increasing lack of initiative regarding self-study.
Recommendation a: Add structured modules and formal assessment of waterfront duties, including vessel loading scenarios, shipping and export duties (including export tolerances in addition to inward tolerances), administration, paperwork, forms, and use of OSCAR, which may vary by terminal.
Finding: Once trainees graduate from the program, they often transition into a leadership or mentor role in operations work and begin to mentor trainees. Although they lead/mentor others, some do not always receive formal supervisor training. Internal stakeholders working directly with trainees identified a need for increased focus on behavioural competencies and additional training in leadership, supervising, and soft skills.
Recommendation b: Addition of training modules in the areas of mentoring, communication, and feedback to enhance support for the transition of GIDP graduates to mentors and leaders. Ensure GIDP graduates who are in a supervisory role receive formal supervisory training.
4.3.3 High stakeholder satisfaction levels with the Grain Inspector Development Program
Overall, internal stakeholders interviewed reported a high level of satisfaction with recent program graduate performance on the waterfront. When asked how satisfied they were with recent graduates on a scale of 1 to 5, internal stakeholders typically responded with a 4 or higher. When asked whether they felt that new graduates of the program had received sufficient training to perform inspections independently in the field, they reported high levels of satisfaction overall, as illustrated in Figure 2.
Figure 2: Reported satisfaction with Grain Inspector Development Program graduate performance
Figure 2
| Reported satisfaction with Grain Inspector Development Program graduate performance | West/HQ | East |
|---|---|---|
| Performs inspections independently in the field/waterfront for service delivery | 3.6 | 4.3 |
| Meets the needs of the CGC inspection program | 4.1 | 4.4 |
| Provide a high level of satisfaction to the CGC inspection program | 3.9 | 4.2 |
| Meet the needs of external stakeholders | 4.0 | 4.0 |
| Provide a high level of satisfaction to external stakeholders | 4.0 | 4.0 |
Source: Key informant interviews with internal stakeholders, 2023
When asked, external stakeholders were not familiar enough with the program to distinguish the trainees or have a sense of trainee performance on the waterfront.
4.4 Efficiency and enhancements
4.4.1 Trainee perspectives
Overall, trainees reported a higher level of satisfaction with their training environment in the Western Region than the Eastern Region.
Figure 3: Trainees’ perspective on the training environment
Figure 3
| Trainees’ perspective on the training environment | West/HQ | East |
|---|---|---|
| You receive enough mentoring from the people you work with on a daily basis? | 4.1 | 4.4 |
| You receive enough training on shipping aspects of your job? | 3.5 | 3.6 |
| You receive enough variety and exposure to different grains and terminal operations? | 4.0 | 2.6 |
| Are you adequately prepared for your exams? | 4.3 | 3.4 |
Source: GIDP Trainee Survey, 2023
Suggested areas for improvement included the following:
- Lack of exposure to a variety of different grains and terminal operations in the Eastern Region
- Preparation for exams in the Eastern Region
- Training on shipping aspects in the Western Region
Trainees also reported a high level of satisfaction with the quality of feedback, mentorship and instruction received from mentors and supervisors on the waterfront. Again, satisfaction levels were higher in the Western Region than the Eastern Region.
Figure 4: Trainee satisfaction with quality of feedback
Figure 4
| Trainee satisfaction with quality of feedback | West/HQ | East |
|---|---|---|
| Trainers | 4.8 | 4.3 |
| Supervisors | 3.9 | 3.8 |
| Grain Inspectors | 4.0 | 4.0 |
Source: GIDP Trainee Survey, 2023
Figure 5: Trainee satisfaction with quality of instruction and mentorship
Figure 5
| Trainee satisfaction with quality of instruction and mentorship | West/HQ | East |
|---|---|---|
| Trainers | 4.8 | 3.4 |
| Supervisors | 4.5 | 3.6 |
| Grain Inspectors | 4.5 | 3.4 |
Source: GIDP Trainee Survey, 2023
Trainers are selected via a competitive process and are developed via mentorship, on the job training, and self-learning. Each module taught by the trainers includes trainer guidelines/instructions that trainers follow for comprehensive and consistent training delivery. Trainers reported a high level of satisfaction with the support they receive from the program manager and other co-workers. Trainers also reported that self-learning opportunities are an important part of their development. Key informant groups and trainees reported a high level of satisfaction overall with trainers and the program as a whole.
Key informants also identified potential benefit to adding "train the trainer" onboarding and ongoing refresher training for all trainers to enhance the quality of training and to support trainers and their ongoing development.
Over half of all trainers interviewed (57%) felt there were enough trainers in both official languages. Of those interviewed by the evaluator, only 17% of inspection staff who work closely with GIDP trainees on the waterfront felt there were enough trainers in both official languages.
Finding: Overall, trainees feel supported by their trainers and are happy with mentoring received on the waterfront. The program could be improved by adding more trainers overall and more trainers in both official languages. There is little in the way of formal training offered to program trainers.
Recommendation a: Continue to hire additional trainers in both official languages (this is currently underway).
Recommendation b: Add formal training for trainers in the areas of training, and learning. If needed, add custom training modules for ongoing refresher training.
4.4.2 Competency map
Trainers were clear and knowledgeable on the following:
- the documents used to outline competencies
- the modules used for each competency
- the criteria used to assess competencies
- the assessment tools and instruments used for all barrier, marker and practice exams
However, not all key informant groups (who are responsible in some capacity for leadership, mentoring, training or assessment of trainees) gave consistent responses in this area. A competency map, or master document that clarifies what is expected and how to meet the criteria for advancement, was not found for the program.
Finding: Inconsistency and lack of clarity in documentation. All of the pieces exist but their use may vary by location. Program could benefit from compiling one competency map for clarity and consistently.
Recommendation: Re-organize existing program documents into a competency map, filling in any gaps where necessary. This document should outline the required competencies, the criteria for assessing the competencies, and the assessment tools used for each. The competency map should give all parties more clarity to ensure consistency regarding competency and proficiency needed to advance. The competency map should reflect recommended updates implemented as part of this program evaluation.
4.4.3 Governance
The Commission approved the Grain Inspector Development Program Framework and accompanying roadmap on December 10, 2013. These documents recommend the program be assessed periodicallyFootnote10 to ensure current practices align with these guidance documents. During this evaluation, the evaluator found evidence that some practices may have evolved since the GIDP was designed.
While barrier exam failure over the life of the program was only 4%, the overall impact on morale was significant. Actions to mitigate this could include:
- offering a pre-test or practice test to confirm that trainees are ready for the barrier test
- exploring options other than immediate termination upon barrier exam failure, such as allowing the trainee to spend up to 1 year working as a PI-01 before resuming the program or re-taking the test
More formal assessment of vessel loading and waterfront skills may also be necessary. The roles and responsibilities around competency-based advancement may need be better defined.Footnote11
Finding: The evaluator found evidence to warrant a comprehensive review of the terms of the Grain Inspector Development Program Framework and Roadmap. Some program practices may have evolved, and the impact of emergent technologies may have created new opportunities. Recommendations made within this evaluation may necessitate review and update of the governance documents.
Recommendation: Review and update program governance documents focusing on these additional areas that have not already been covered in previous management action plans:
- incorporation of new and emerging technologies
- probationary period and program
- termination and advancement rate
- promotion process, and roles and responsibilities
These documents should link to and reference the competency map.
4.4.4 Formal program feedback mechanism
Key informant interviews and trainee survey responses consistently indicated that there is no formal program review or feedback mechanism in place to collect regular (annual or quarterly) feedback about how the program is functioning. Overall, there was a high level of satisfaction with the trainers and program manager. Internal stakeholders interviewed consistently reported that the program manager and trainers are accessible and open to feedback. However, not everyone is comfortable speaking up without a formal and confidential feedback mechanism. Trainees who responded to the survey reported that there was no formal program feedback mechanism in place, but that their waterfront supervisors did solicit feedback informally and occasionally.
Finding: Program does not collect regular feedback from trainees and internal stakeholders for the purpose of continuous program improvement.
Recommendation: Implement a formal feedback mechanism to collect feedback from trainees and inspectors for purposes of continuous program improvement. A confidential feedback mechanism will ensure that everyone has the opportunity to provide feedback.
4.4.5 Training and assessment materials and resources
The evaluator found that while the program is working effectively, there are opportunities to improve the quality and availability of training materials and resources.
Overall, amongst trainees there was a higher level of satisfaction with training materials in the Western Region than the Eastern Region.
Figure 6: Satisfaction with training materials
Figure 6
| Satisfaction with training materials | West/HQ | East |
|---|---|---|
| Trainers | 4.4 | 2.8 |
| Grain Inspection (PI-03 and PI-04) | 4.1 | 3.4 |
Source: Key Informant Interviews, 2023
Most respondents felt that training materials had improved significantly over time, and identified following areas for improvement.
Trainers consistently felt that the training offered matches the work performed on the waterfront. The majority of internal stakeholders interviewed, however, felt that although the training was well delivered and effective in the grading aspects, the program would benefit from additional focus on waterfront, vessel loading and shipping duties, including administration and paperwork, forms, OSCAR, conflict resolution, and communication with internal and external parties. For example, vessel troubleshooting scenarios could be added to the training materials. Key informants noted that these skills are taught via mentorship but not formally assessed. This view was fairly consistent between the regions, with slightly more emphasis in the Western region.
In addition, respondents felt that every terminal should have their own set of training sample kits for all grains. These respondents found it problematic to share and ship training kits. Some of the self-directed training materials available to trainees could be updated and modernized using new training technology to promote virtual training and self-directed training and study.
Finding: Program evaluation identified further opportunities to update and modernize training materials while acknowledging the significant recent progress and high level of satisfaction with training materials.
Recommendation: Continue to modernize training materials (including self-directed materials) and explore use of new training technology. Update crop samples and provide full set of training kits for each permanently staffed location.
4.4.6 Program database
The program does not accurately measure and track key metrics in a centralized database. It is not clear whether tracking is a program responsibility or whether this is done outside of the program. As a result, it was difficult and time consuming for the evaluator and program manager to source basic information about the program participants, program budget and costs, and status of program activities. This can result in a diminished ability to accurately assess program benefits and make key program decisions.
Finding: Evaluator found a lack of easily available program data.
Recommendation: Assign additional administrative support to develop and maintain a comprehensive program database to further strengthen program oversight and decision making. System to provide up-to-date, easily accessible student information including trainee retention and loss. Given the other recommendations including modernization and program organization there may be benefit to considering the use of a learning management system.
5.0 Conclusion
There is significant evidence to support the continued need for the Grain Inspector Development Program to produced fully qualified grain inspectors and support Canadian Grain Commission inspection. There was no evidence of duplication by another comparable program that can provide the same level of training. There was significant evidence that external stakeholders rely on Canadian Grain Commission resources in some way for their own inspection training. The program aligns with government priorities, roles and responsibilities.
Overall, there was significant evidence that the program is well-utilized and is achieving its intended outcomes but that the program struggles to produce enough fully-trained inspectors in some regions to meet the Canadian Grain Commission’s current and future service delivery demands. Shortages may exist in some areas during peak grain shipping seasons. Recommendations are provided to address this.
This evaluation also provides several recommendations for program efficiency and enhancement, including areas of interest that were identified by the business process owner.
6.0 Acknowledgement
The Chief Audit and Evaluation Executive would like to thank the Evaluation Working Group, evaluation team and individuals who contributed insights and input to this evaluation.
Appendix 1: Findings, recommendations and management action plans
The table below summarizes findings and recommendations identified during the program evaluation that require management action plans.
The Director, Industry Services is responsible for implementation of MAPs other than where specifically noted. Management agrees with all recommendations.
| Relevance (2) | Observation and findings | Potential impact | Recommendations | Management action plans |
|---|---|---|---|---|
| 4.1.1 Continued need for program | There is a continued need for the program to support CGC service delivery and to support external stakeholders. Evaluator was unable to identify any other training programs offering a similar level of training. | Continuous program improvement | Continue to improve the program with the following recommendations for improvement and or enhancement. | Management agrees with the recommendation. |
| 4.2 Alignment with government priorities, roles and responsibilities | Program aligns with government priorities roles and responsibilities. | Continuous program improvement | Continue to improve the program with the following recommendations for improvement and or enhancement. | Management agrees with the recommendation. |
| Performance (5) | Observation and findings | Potential impact | Recommendations | Management action plans |
| 4.3.1.1 Program Length | Overall, those surveyed and interviewed felt that the program could be shortened to potentially increase graduation rates to support grain inspection without compromising the quality of training. | Service delivery impacts. | Explore shortening the program based on the evidence-based potential opportunities outlined. | Management agrees with the recommendation to explore shortening the program based on the evidence-based potential opportunities. Timeline for approved implementation plan is July 31, 2025. Implementation to be completed by February 28, 2026. |
| 4.3.1.2a Trainee Retention | The main reason trainees exit the program is resignation. Key informants reported that this was primarily related to factors external to the program. The nature of the work, the hiring process, and compensation were the main factors in resignations, rather than the training program itself. Key informants felt that recruitment processes could better provide candidates with information about the realities of the work and the work environment. | Service delivery impacts. | Update GIDP recruitment related outreach (external to the program) to incorporate a more realistic description of the work and the work environment. Examine new ways to appeal to and engage potential best fit candidates. Continue to value previous experience such as industrial and or agricultural related. Improve GIDP program information on the Canadian Grain Commission website. Targeted recruitment and building awareness of this program online and using social media is underway. | Management agrees with the recommendation. IS management will seek HR expertise to lead the initiative to make improvements related to GIDP recruitment and outreach focusing on areas highlighted in the recommendation. Work to be completed by March 31, 2025. Director IS accountable for completion of this MAP and Executive Director HR to provide guidance, SME and resources. |
| 4.3.1.2b Trainee Retention | Service delivery impacts. | Analyze GIDP trainee retention data to better understand at which stage of the program GIDP candidates are exiting the program. Explore and further investigate the loss of GIDP candidates including the already highlighted potential link between compensation and loss of GIDP candidates. | Management agrees with the recommendation and will continue to monitor, analyze and track the already highlighted potential link between compensation and loss of GIDP candidates to support and backup meaningful discussion with the union during contract negotiation. Completion date of June 30, 2025. Director IS accountable for completion of this MAP and Executive Director HR to provide guidance on compensation, SME and resources. | |
| 4.3.2a Sufficient technical competencies |
Trainees that responded to the survey felt that more time and attention should be given to waterfront and shipping duties. In addition, the majority of inspectors and the specialists/operations supervisor groups identified 2 areas for improvement: 1. Add formal training modules and assessment of waterfront, vessel loading and shipping duties, including administration and paperwork, forms, OSCAR, (many of which can vary by terminal). | Risk of decline in quality of level of service delivery in future. | Add structured modules and formal assessment of waterfront duties, including vessel loading scenarios, shipping and export duties (including export tolerances in addition to inward tolerances), administration, paperwork, forms and use of OSCAR, which may vary by terminal. | Management agrees with the recommendation. A comprehensive implementation strategy to be completed by June 30, 2025, in order to facilitate implementation of 4.3.1.1 of February 28, 2026. |
| 4.3.2b Sufficient behavioural competencies |
Once trainees graduate from the program, they often transition into a leadership or mentor role in operations work and begin to mentor trainees. Although they lead/mentor others, some do not always receive formal supervisor training. Internal stakeholders working directly with trainees identified a need for increased focus on behavioural competencies and additional training in leadership, supervising, and soft skills. | Risk of decline in quality of level of service delivery in future. |
Addition of training modules in the areas of mentoring, communication and feedback to enhance support for the transition of GIDP graduates to mentors and leaders. Ensure GIDP graduates who are in a supervisory role receive formal supervisory training. |
Management agrees with the recommendation. Additional training modules in the areas identified will be added. IS will collaborate with HR. To be completed with 4.3.1.1 February 28, 2026. |
| Program Efficiency & Enhancement (7) | Observation and findings | Potential impact | Recommendations | Management action plans |
| 4.4.1a Trainee Perspectives | Overall, trainees feel supported by their trainers and are happy with mentoring received on the waterfront. The program could be improved by adding more trainers overall and more trainers in both official languages. There is little in the way of formal training offered to program trainers. | Trainee satisfaction levels. | Continue to hire additional trainers in both official languages (this is currently underway). | Management agrees with the recommendation and will continue to monitor the number of trainers needed in both official languages on an ongoing basis and monitor the impact of any potential change to the GIDP program on the trainers in both official languages required. This will be implemented August 31, 2026 (based on meeting Feb 28, 2026 4.3.1.1 implementation date) and ongoing and part of our annual people plan related to the sustainability of our inspection program. |
| 4.4.1b Trainee Perspectives | Impacts to trainer professional development and quality of training. | Add formal training for trainers in the areas of training, and learning. If needed, add custom training modules for ongoing refresher training. |
Management agrees with this recommendation. HR and IS will add development opportunities for trainers. Will explore the GOC learning system (CSPS) and potentially other government departments and learning sources for an appropriate “train the trainer” course for new and existing IS trainers. To be implemented by Feb 28, 2026. Director IS responsible for this MAP with supporting resources from Executive Director HR. | |
| 4.4.2 Competency Map | Inconsistency and lack of clarity in documentation. All of the pieces exist but their use may vary by location. Program could benefit from compiling one competency map for clarity and consistently. | Increased risk to the organization / adherence to best practices | Re-organize existing program documents into a competency map, filling in any gaps where necessary. This document should outline the required competencies, the criteria for assessing the competencies, and the assessment tools used for each. The competency map should give all parties more clarity to ensure consistency regarding competency and proficiency needed to advance. The competency map should reflect recommended updates implemented as part of this program evaluation. |
Management agrees with the recommendation. A comprehensive competency map will be developed as per the recommendation. Completion date February 28, 2026 (to align with completion of 4.3.2b). Director IS responsible for this MAP with supporting resources from Executive Director HR and Communications (Executive Director I&S). |
| 4.4.3 Governance | The evaluator found evidence to warrant a comprehensive review of the terms of the Grain Inspector Development Program Framework and Roadmap. Some program practices may have evolved, and the impact of emergent technologies may have created new opportunities. Recommendations made within this evaluation may necessitate review and update of the governance documents. | Increased risk to the organization / adherence to best practices |
Review and update program governance documents focusing on these additional areas that have not already been covered in previous MAPs.
These documents should link to and reference the competency map. | Management agrees with this recommendation and agrees to review and update the program governance documents as part of program updates in MAPs above and additional areas noted in the recommendation. Will collaborate with HR to ensure proper application of labour relations processes. Updated program governance documents to be completed by February 28, 2026. Director IS responsible with support from Executive Director HR and Executive Director I&S. |
| 4.4.4 Program Feedback | Program does not collect regular feedback from trainees and internal stakeholders for the purpose of continuous program improvement. | Increased risk to the organization / adherence to best practices | Implement a formal feedback mechanism to collect feedback from trainees and inspectors for purposes of continuous program improvement. A confidential feedback mechanism will ensure that everyone has the opportunity to provide feedback. | Management agrees with this recommendation and agrees to develop an annual feedback collection process for trainees and internal stakeholders for continuous program improvement. Process to be implemented by July 31, 2026 to allow one year with program updates in place (4.3.1.1. completed July 31, 2025). |
| 4.4.5 Training & Assessment materials/resources | Program evaluation identified further opportunities to update and modernize training materials while acknowledging the significant recent progress and high level of satisfaction with training materials. | Risk of falling behind current trends and techniques. | Continue to modernize training materials (including self-directed materials) and explore use of new training technology. Update crop samples and provide full set of training kits for each permanently staffed location. | Management agrees with this recommendation and will continue to modernize training materials and technologies. To be completed as part of 4.3.2.a (to be completed July 31, 2025). Full set of training kits to be provided to all permanently staffed locations by December 31, 2024. |
| 4.4.6 Program Database | Evaluator found a lack of easily available program data. | Diminished ability to accurately assess program benefits and measure efficiency and make effective program decision. | Assign additional administrative support to develop and maintain a comprehensive program database to further strengthen program oversight and decision making. System to provide up-to-date, easily accessible student information including trainee retention and loss. Given the other recommendations including modernization and program organization there may be benefit to considering the use of a learning management system. |
Management agrees with this recommendation and agrees to explore database options such as learning management system (LMS) or other built in database offerings that may be available off the shelf. Implementation of a data base and/or LMS to be completed by December 31, 2026. Director, IS responsible for completion of this MAP with supporting resources from Executive Director I&S. |
Appendix 2: Estimated program budget
| Program Year | FTEs | Salary Costs | Travel | Training Materials | Total Annual Costs |
|---|---|---|---|---|---|
| 2013/14 FY | 11 | $639,557 | $5,700 | $300 | $645,557 |
| 2014/15 FY | 27 | $741,880 | $7,920 | $1,200 | $751,000 |
| 2015/16 FY | 35 | $807,773 | $19,500 | $1,500 | $828,773 |
| 2016/17 FY | 48 | $902,742 | $16,200 | $2,150 | $921,092 |
| 2017/18 FY | 46 | $863,366 | $24,900 | $2,550 | $890,816 |
| 2018/19 FY | 47 | $937,223 | $34,950 | $2,350 | $974,523 |
| 2019/20 FY | 52 | $882,762 | $24,300 | $2,600 | $909,662 |
| 2020/21 FY | 46 | $843,760 | $- | $2,200 | $845,960 |
| 2021/22 FY | 48 | $1,025,202 | $- | $2,500 | $1,027,702 |
| Total | 360 | $7,644,265 | $133,470 | $17,350 | $7,795,085 |
Source: Industry Services administration in conjunction with the program manager
FTEs include: trainee and trainer resources
Salary costs include: 10% GIDP trainee salary (does not include trainee time spent supporting service delivery) and 100% GIDP trainer salary is included in program budget
Travel costs: reflect travel for training purposes only
Training Materials includes: the shipping costs associated with sending materials to trainee's locations and the materials used to create the training packages
Note: the program uses mentoring resources and administrative support from the inspection program, which was not included in the budget above but is attributed to service delivery requirements. The one-time purchase of trainee supplies was also attributed to service delivery. Costs of infrequent relocations were also not included in the budget as they are attributed to service delivery.
Appendix 3: Methodology
Program information review
- Grain Inspector Development Program Framework and flowchart
- Training and assessment materials
- Competency documents and performance measures
- Staffing information
- Discussions with the program manager
Document review
The documents reviewed include:
- Departmental Plan and Departmental Results Framework
- Strategic Planning Infographic 2022-2023
- Human Resources planning documents
- Publicly available Canadian Grain Commission mandate letters from the period in scope (2018, 2022)
- Public consultation documents
- Demographic Snapshot of Canada’s Public Service 2021, Treasury Board Secretariat
- New articles
- Job postings
Existing data
- Program staffing data
- Canadian Grain Commission demographic data
- Program budget
Key informant interviews
Key informant interviews were conducted with relevant groups to capture a wide range of views and opinions regarding the program. Interviews were conducted during July and August 2023.
- Current and former trainers (80% response)
- Internal stakeholders
- Inspectors and supervising grain inspectors (56% response)
- Specialists and operations supervisors (80% response)
- Please note: of internal key informant interview conducted, approximately 45% were with union representatives
- Regional managers (all regional and senior managers)
- External stakeholders familiar with the program (over 60% of those invited) including representation from industry (grain companies and third-party grain inspection service providers)
Virtual interviews with key informants consisted of both open-ended and scale questions. The interview questions were designed to gather key informant views on relevance and program performance.
GIDP Trainee Survey
A confidential email survey of Grain Inspector Development Program trainees was conducted between July 21st and August 9th, 2023. The survey was designed to collect trainee views and opinions about the program. The survey was distributed to all trainees.
Out of approximately 37 trainees at GIDP, 21 submitted complete responses. This reflects an organizational response rate of approximately 57%. Respondent breakdown is as follows:
- 61.9% are trainees at level PI-01 (n=13)
- 23.08% are trainees at level PI-02 (n=5)
- 14.2% are trainees that left their program level blank or selected “Prefer not to say” (n=3)
Responses were collected, analyzed, and summarized for reporting. Responses were also categorized by demographic to provide more context and detail. Names of survey respondents were not collected to ensure confidentiality. The survey asked respondents to provide basic demographic data such as gender, region and city of work, if they were a member of a vulnerable population, and their program level. Demographic analysis is useful to show trends in survey response data to inform recommendations made in this report.
Constraints and limitations
The following methodological constraints and limitations were identified during the evaluation. Steps taken to mitigate their impacts are included.
Constraint: Program resources and costs have not been fully tracked for this program in a centralized database.
Mitigation: The current acting program manager provided a high-level budget estimate for the period in scope.
Constraint: The Grain Inspector Development Program and grain inspection are intertwined and it was very difficult to separate the two in terms of resources, budget and external views of the program.
Mitigation: We estimated and attempted to separate the program where possible.
Constraint: External stakeholders typically did not have a high level of awareness of the program, and it was difficult to collect their views on the program as distinct from grain inspection.
Mitigation: When collecting external views on the program we attempted to clarify the difference between the Grain Inspector Development Program and grain inspection services.
Constraint: Potential for self-selection bias in the employee survey. Due to self-selection bias in the trainee survey, the demographic characteristics of the survey responses could potentially be an inaccurate reflection of the program’s population. The following comparison was completed.
Demographic comparison of trainee survey to the program population (region)
| Question | Trainee - Program % | Trainee - Survey % |
|---|---|---|
| Western / HQ Region | 64% | 71% |
| Eastern Region | 37% | 30% |
Demographic comparison of trainee survey to the program population (gender)
| Question | Trainee - Program % | Trainee - Survey % |
|---|---|---|
| Male | 76% | 75% |
| Female | 24% | 25% |
Source: Program data and survey data
Note: Comparison was limited to region and gender due to data constraints
Mitigation: The demographic table above compared the demographics of the survey to the program population to assess whether the survey results accurately reflect the views and opinions of the program population overall. Comparison indicated that the trainee survey is representative of the population. It should be noted that the Western Region is slightly over-represented in the survey data and the Eastern Region is slightly under-represented. This was considered when interpreting survey results.