Review Topics for Enquiry: A summary of issues by project gate

ISBN: 978-1-100-15690-3
Catalogue No. BT53-18/2010E-PDF

This document is available in alternative formats upon request.

Foreword

Review Topics for Enquiry is intended to be used by independent reviewers. This document is part of the guidance on independent project reviews developed by the Chief Information Officer Branch at the Treasury Board of Canada Secretariat (TBS). Reviews of information technology (IT)-enabled projects include full reviews, quick reviews, workshop reviews, and health check reviews.

This document has been developed as a companion piece to The Independent Reviewer's Handbook to provide project reviewers with specific areas to investigate, broken down into topics and linked to the issues that are most likely to be relevant at each gate in the gating framework.

1. Reviewer's Master Outline of Topics for Enquiry

1.1 Topics for enquiry tables: An overview

The topics for enquiry tables are intended to support an experience-based project review. The word "topic" is used to indicate that each entry in the table is able to generate a variety of specific questions, depending on the situation. The tables themselves, however, are not designed as questionnaires or checklists.

Reviewers are expected to be able to formulate review-specific questions for each relevant topic, based on the initial briefing, the review reading, and the extensive experience of the review team. Furthermore, because each review is a voyage of discovery, so to speak, the questions for each topic are expected to evolve as the review progresses. Questions asked about a topic in early interviews are likely to be different from questions asked about the same topic in later interviews.

The tables serve as a repository of information about the most significant issues that may be encountered at key decision points, or gates, in a project. These issues may need to be investigated during a project gate review. As emphasized in The Independent Reviewer's Handbook, the project gate review is only one of the inputs considered at a gate decision meeting. It is not a substitute for other forms of due diligence and monitoring during the course of the project, such as quality assurance (QA), Independent Verification and Validation (IV&V), and audits. Therefore, the tables are not meant to be used for these other functions.

The review team can tailor the content of the tables to support a variety of gate definitions. The basic principle in the development of the tables is the expectation that each successive gate would exhibit the following:

A greater degree of detail for project definition and planning;
Fewer uncertainties and unknowns;
An increased number of mitigated risks;
More precise estimates; and
A greater understanding of the specific business outcomes.

Reviewers can adapt this document's structure and move topics forward or backward so that they appear where needed, based on the specific gating plan used on a project.

The content in the tables is based on the review of large projects and several years of project review experience. The outline is highly reflective of topics that have proven to be significant for large IT-enabled projects undertaken by the Government of Canada in recent years.

Though IT project management literature offers a number of excellent project review guides, none would appear to provide an exhaustive list of all the challenges a project might encounter. The following tables are also not exhaustive. Nevertheless, their content is based on solid project review experience, concrete interviews, and a thorough study of industry publications. It can be said with confidence that they present the priority topics to be considered when reviewing federal government projects.

Finally, this material is not intended to constrain the enquiring mind. Review team members may find it necessary to explore uncharted areas and will have to decide for themselves what degree of detail is required for the particular project situation they encounter.

1.2 How to use this outline

The outline is divided into two main tables or groups:

The first table, called the "Primary Issues Table," describes the primary issues that affect a project from inception to completion and provides a synopsis of the significance and importance of each issue. The table is divided into 12 topics, and for each topic there is a list of issues. The table's "Significance and Importance" column explains the relevance of each issue.
The second group is a series of individual tables representing each of the seven gates usually encountered in the life cycle of a major project. The tables set out gate-by-gate review points for each of the primary issues and suggest appropriate lines of enquiry for these review points. When two or more gates are collapsed into one in a project review, the tables can be collapsed accordingly.

Each gate's table includes all the topics and all the issues within the topics—the same structure as in the Primary Issues Table. Content is provided for a particular gate when relevant, along with suggested lines of enquiry that reviewers can choose to pursue at their discretion.

Every review has to be tailored to the nature and scope of the project, the project's environment, and the particular stage in its life cycle. Reviewers must exercise good judgment when applying the information in the tables on issues, review points, and lines of enquiry. Reviewers may wish to develop a checklist of primary issues from these tables to use in interviews, to conduct analysis, or to ensure they have completely covered all aspects of each topic.

1.3 Primary issues to be reviewed

Primary Issues Table: Topic 1—Business proposition

Issue	Significance and Importance
The business imperative	Initiatives that are seen as important to the enterprise have the greatest likelihood of success, particularly when they are closely aligned with program and business objectives, are visibly sponsored, and have clearly assigned accountability for results. Mission-critical initiatives are rarely allowed to fail in government. Initiatives that are not perceived to be critical to the mission are less likely to receive the executive direction and attention they need to succeed. The business imperative must be based on an understanding of the current business environment and the business problem at issue. The business imperative and circumstances that broadly affect the business proposition can change while a project is in its definition phase, as well as during its execution phase. It is essential to revisit and reconfirm this issue as the project proceeds, and to question the project appropriately, should the business imperative itself change.
Vision and scope	Project sponsors, stakeholders, and project management must share a common vision of the project and a corresponding understanding of the project's scope. Otherwise, the project will lack cohesiveness and suffer from continuing debate and uncertainty about the desired outcome. Imprecise project scope and boundaries threaten a project from its launch. There has to be clarity on what the project is and what the project is not. Scope is concerned not only with the functionality of the system under development but also with the community of users to be served.
Expected business outcomes	Outcomes expressed in business terms are the means by which the value of the business case is established. If the new system does not achieve the planned outcomes, the underlying business case is invalidated and the project cannot be considered successful. Outcomes need to be tangible and measurable. It must be possible to easily validate the alignment of the project with stated business expectations and outcomes. Expected business outcomes have to be absolutely clear among project sponsors, stakeholders, and project managers. Without a common understanding of the outcomes, reaching agreement on requirements will be difficult and the project may drift from its intended objectives during the execution phase. Ultimately, the initiative may be at risk of failing to deliver an acceptable solution to the business problem at hand. Within the federal government, a formal outcomes management model and plan are recognized management tools designed to assist project groups in realizing business outcomes through performance indicators and measurements. Effective project management requires that all deliverables be mapped to the outcomes model and plan.
Alignment with other programs and business strategies	There should be a clearly established relationship and alignment with government and departmental business strategies—that is, with the whole-of-government framework and departmental program activity architecture. Business cases must also align with, conform to, and respect approved government-wide and cross-departmental initiatives, as applicable. Examples include TBS shared systems, Service Canada, and the Secure Channel Network, among others.
Assumptions, constraints, and dependencies	The assumptions upon which early project decisions are based must be clearly understood so that deviations from assumptions can be recognized immediately and timely remedial actions taken. Unclear, faulty, or changing assumptions do not provide an appropriate basis for key decisions at future project stages. In particular, costs are often underestimated in the early stages because there are still too many unknowns, and optimistic or best case assumptions are used or attempts made to fit within a predetermined budget. Choosing one assumption over another can easily make a 100-per-cent difference in cost estimates, and cumulative errors can dramatically change the cost of the project. Sensitivity analysis is important in the costing process. Similarly, failure to identify and take into account constraints and dependencies may fundamentally alter the underlying conditions for project success from the outset.

Primary Issues Table: Topic 2—Sponsorship, leadership, and governance

Issue	Significance and Importance
Project sponsorship, ownership, and commitment	A project should proceed only when there is a clear project sponsor who understands his or her role and is prepared to take ownership of the results. In general, projects are led by the business owner (normally, the departmental executive responsible for overall program delivery), who is supported and enabled by the departmental IT organization. Typical senior executive misunderstandings include: Legislating or mandating end dates; Failing to clearly assign a project sponsor and managers; Failing to understand the notion of "project time"; Failing to assess the potential impact of unanticipated events and decisions on the project; and Committing the project to absorb or contain the effect of some eventuality or change, before determining whether this is feasible.
Leadership and management	Project leaders and managers need to understand the expectations and obligations placed on them. They should bring qualifications and experience commensurate with the scale and complexity of the project undertaken. Project managers should be dedicated to project activities and not be expected to perform other management responsibilities. This way, they can give their undivided attention to the project. Because enterprises differ in their management cultures and principles, it is difficult to establish standard criteria for project management. Allowances have to be made for the management culture of an organization. Adjusting for practices that are considered unusual by some people but have demonstrated a good record of success in comparable ventures is a good idea. For example, in some organizations, extensive matrix management with vague accountabilities works, even though this is generally not a recommended practice for large IT-enabled projects.
Stakeholder roles and responsibilities	All stakeholders in the project should be identified. Their responsibilities and accountabilities have to be appropriately assigned and precisely stated, and their commitment to a successful project outcome must be beyond question. Clear, documented stakeholder responsibilities and accountabilities are essential to ensure timely decisions and enable progress. Stakeholders have to understand what needs to be done, and they must be capable of doing it.
Decision making and governance	"Project time" means taking action and making decisions that support a project within the time frame required by the project. Otherwise, work stops and costs mount. Senior officials involved in a project who do not understand the concept of project time sometimes expect that projects are able to wait in a holding pattern, without incurring costs, while decisions are made. It is critical that the project's governance framework work effectively—that it be responsive and not onerous. This is particularly true for horizontal or multi-jurisdictional projects. Governance mechanisms that are too complex for the circumstances obscure accountability and encumber the project manager. Projects are often burdened with excess governance that diffuses accountability and adds marginal value. On the other hand, governance frameworks that are too simple or do not engage the right parties will not serve the project well. Getting it right is critical, especially in situations where project governance bodies are mandated by policy. The governance framework and process need to be clearly articulated. Approved terms of reference and appropriate membership for essential governance bodies have to be established—particularly for the business area. Meetings should be carefully orchestrated to ensure timely decisions, and records should clearly communicate results to project participants, as appropriate to their responsibilities.

Primary Issues Table: Topic 3—Concept and approach

Issue	Significance and Importance
Project classification and business change management	IT-enabled projects are classified according to the extent of change they bring to the business line they support; namely, projects can be sustaining, tactical, evolutionary, or transformational. There are widely different management issues associated with each class of project, and experience has shown that it is vital the project be set up and managed according to its classification. If the classification that fits the project initiative is not completely understood, it is difficult to determine whether the department has the capacity to manage it successfully. The issues associated with each class of project, such as capacity and risk, differ. Projects with major transformation and business process change components generally represent the greatest challenge. At times, managers have initially classified some projects in one group and subsequently attempted to manage them accordingly, only to realize later that the projects actually belonged in a different class and required a very different management approach (for example, projects classified as sustaining often turn out to be transformational).
Alternatives and options	During project conception, there are usually many ways to approach the business requirement. The failure to assess these options fully or the pursuit of inappropriate options carries significant risk with major implications, because everything that follows will be affected by these decisions. Frequently, project managers predetermine the approach without fully considering alternatives that might offer better value or lower risk. Make-versus-buy options (e.g., commercial-off-the-shelf (COTS) versus custom development, outsourcing, etc.) are often fundamental to developing the project approach. They greatly affect important areas such as procurement, requirements definition, and data conversion. Projects involving the replacement, integration, or evolution of legacy systems often involve considerable risk. The process is like changing an aircraft engine mid-flight. Projects that appear to be a straightforward replacement of systems often evolve into enhancement projects, though without the corresponding adjustment to the management approach. Aging mission-critical legacy systems are often poorly documented, and few staff members with the expertise to understand them are still with the organization. Another option requiring careful consideration is the decision to undertake a "greenfield" project. This is a project that builds an entirely new system from scratch without reusing components from existing systems. While appealing to developers, these projects are often high-risk as a result of the many unknowns. A technical design review conducted as soon as high-level business requirements are known can help narrow technology choices.
Project sizing and packaging	An assessment should be made early on to determine whether the size of a proposed initiative makes it: 1) too expensive for the level of available funding, 2) beyond the capability of the department, 3) excessively risky, or 4) unachievable within the required time frame. The proposed initiative may subsequently be repackaged into more manageable entities, each representing a project in its own right.
System transition strategy	In choosing the project approach, the feasibility and risk of various transition and implementation scenarios must be carefully considered. A big-bang or single-stage system migration approach for implementing a new or replacement system is always a high-risk undertaking because there is seldom a viable fallback plan. A gradual system migration requires parallel operations and support. This is costly but carries significantly lower risk. Transition implications must be clearly understood from the outset and the consequences widely accepted. Business factors include data migration, process re-engineering, training, temporary staffing, support, documentation, etc. It is often costly and sometimes impossible to change the transition approach once a project is under way.
Service delivery and management strategy	Understanding how service delivery is to be managed helps stakeholders understand how the business will change, what costs to anticipate, and how to capture the benefits. A concept of operations statement is useful for describing roles, responsibilities, and performance objectives. As the project progresses, this statement typically evolves to include service levels, support strategies, and performance measurement.
Gating and exit strategies	At the conceptual stage, major gates representing high-level approval milestones and logical off-ramps or exit points should be identified. Exit strategies need to take into account the residual business value achieved, should the project have to be terminated at some logical decision point or gate.

Primary Issues Table: Topic 4—Organizational readiness and capacity

Issue	Significance and Importance
Organizational capacity	Undertaking projects that are beyond the capability of the enterprise or project participants is a principal reason for project difficulties and failures. Unless the department has a project culture and proven project management capacity derived from repeated experience with systems development, even a modest project undertaking can be a formidable challenge. Apart from meeting the demand for specific project delivery infrastructure, the department has to set up an overarching supportive environment in areas such as human resources (HR), finance, procurement, etc. This may prove to be difficult. Departments must be prepared—success depends on the readiness of the departmental corporate, business, and project environments.
Executive and management capacity	Some government executives reach the level of assistant deputy minister or deputy minister without ever having managed or been directly involved in a large project. These executives often do not understand projects and may struggle or fail to create and maintain an environment in which the project can succeed. The stability and continuity of executive roles and accountabilities throughout the project are critical, and their importance should not be underestimated.
Business capacity	Business or program groups that have experience in the role of business owner in an IT-enabled project usually understand their responsibilities and are prepared for them. They accept responsibility for decisions on the program delivery approach, the delivery channels to be used, and the extent of automation that will be involved. Further, they have the required ability to design the program delivery approach and associated business processes, to map the client interactions and information flows, and to specify the business rules. They are familiar with the discipline and process that this requires and are accustomed to the various tools that the IT project team uses to document these requirements. Business groups also need to be prepared to participate in the testing and verification of systems to ensure that business requirements are met prior to implementation. As well, the business area is responsible for forecasting business volumes and providing growth projections to IM/IT so that the required infrastructure can be planned and developed according to schedule. The business owner also manages relationships and communication with stakeholders. When these skills are missing, or when the business group does not see it as its responsibility to fulfill these functions, even a straightforward project is placed at risk. In some departments and agencies, program management within headquarters is weak. When this happens, business process design and even business rules can sometimes be left ungoverned, allowing individual operating regions to informally develop their own business processes and rules. In such cases, the headquarters program function may not have the strength to force closure on the processes and rules, making it extremely difficult to wrap up requirements. If the business group perceives that the project is something it must submit to rather than lead, then the entire power balance shifts toward the IT group, which, out of desperation, volunteers to design the business processes and specify the rules. At this point, the business effectively relinquishes control, losing accountability for business operations, and the project becomes IT-led.
IM/IT capacity	The IM/IT organization has a supporting role in relation to the business of the department, but a full functional authority over IM/IT matters. The IM/IT function is expected to provide advice on possible automation approaches, expertise to assist business in specifying requirements, estimates on development times and costs for various approaches, and strategies for integration with and transition to existing operations. The IM/IT group also maintains security and integrity of the overall computing environment through implementation and production. Departmental IM/IT organizations that have done strictly maintenance work for a prolonged period and not regularly undertaken major systems development are often not ready in terms of experience, processes, infrastructure, and discipline to tackle the rigours of a major development project. An IM/IT group that lacks the experience to undertake a large project cannot quickly improve its qualifications by means of staffing, training, or injecting external contracted expertise.

Primary Issues Table: Topic 5—Risk management

Issue	Significance and Importance
Identification of risks	From conception to delivery, large IT-enabled projects are exposed to a variety of factors that may potentially jeopardize planned business outcomes. Risks have to be identified, impacts assessed, and mitigation strategies formulated. A comprehensive risk management plan identifies major risks and ensures that appropriate mitigation measures are covered. The first step is to identify areas of potential risk. Risks encountered in the government environment can either be external to the department or within departmental control. The following are examples of external risk factors: Political sensitivities—Projects are often associated with or in direct support of politically sensitive or controversial government programs and initiatives. Projects connected to politically hot files are at risk of unplanned changes in direction. They may also suffer from underfunding because of an arbitrary funding cap. Policy or program synchronization—If projects are conceived and launched while unresolved legislative or government policy issues are pending, mid-course changes may be required or the entire initiative may have to be abandoned if legislation, regulations, or policy changes do not materialize as expected. Interdepartmental or multi-jurisdictional factors—Cross-departmental or multi-jurisdictional projects often have complex governance processes with widely spread authority. As a result, making decisions in project time is difficult. Achieving agreement on a common set of requirements can be exceedingly time-consuming and involve onerous compromises. Changing conditions or instability—The rate and impact of change during the life of the project are significant risk factors. Once conceived, the general environment in which the project is to be implemented is assumed to be known and relatively stable. However, any project that lasts more than 12 months is likely to experience changes beyond project or departmental control. Failure to properly assess the impact of changes on the project can be a major contributor to project overruns. Examples of risk factors within departmental control include: Interference or distraction from normal departmental operations—Project work does not mix well with ongoing program responsibilities. Project groups that are not properly insulated from the day-to-day business of the host department often get distracted and diverted into non-project work, resulting in less time and energy spent on the project. Introduction of new technology—When the proposed business solution is based on the introduction of new or unfamiliar technologies, great care should be taken to ensure that all implications are fully considered. Software, data, and communications compatibility, backup, and recovery are some of the major risk areas due to changes in technical architecture or infrastructure. Geographical factors—Project activities sometimes span different parts of the country or even the world. In extraordinary cases, projects may have to be conducted with a team that is scattered across several centres. The challenges associated with dispersed project work and deployment to remote areas can contribute significantly to the project's risk. Having business and IT resources located in the same centre is highly desirable. Otherwise, complications can arise, often requiring strong management discipline to compensate for them.
Impact assessment	Project executives and managers should not underestimate the impact of identified project risks. The probability of their occurrence and potential impact on the costs and schedule can be assessed at each stage in the project. From these assessments, mitigation strategies and plans can be developed and refined as circumstances change. Excessive risk at any point demands a revision, or outright rejection, of the proposed approach.
Mitigation and contingency plan	In a supportive project environment, management seeks to insulate and protect the project from changes (other than those that are absolutely necessary). The immediate response of project-savvy senior departmental executives to an externally imposed change is to determine how it could affect the project schedule and budget. There should be no reluctance to replan the project or seek revised project authorities when change will materially affect the project. When the probability and impact of risks are underestimated, project sponsors and stakeholders may have a false sense of security about the project. Contingency planning is the management tool used to deal with the inevitable risks stemming from changes and unknown factors that arise over the life of the project. For each identified risk that threatens the delivery of the project or the achievement of business outcomes, there needs to be a formal, achievable, and accepted countermeasure. A fully elaborated risk management plan that properly identifies the risks, assesses their impact, and proposes contingency measures is the vehicle through which to manage risks.

Issue

Significance and Importance

Identification of risks

From conception to delivery, large IT-enabled projects are exposed to a variety of factors that may potentially jeopardize planned business outcomes. Risks have to be identified, impacts assessed, and mitigation strategies formulated.

A comprehensive risk management plan identifies major risks and ensures that appropriate mitigation measures are covered. The first step is to identify areas of potential risk.

Risks encountered in the government environment can either be external to the department or within departmental control. The following are examples of external risk factors:

Political sensitivities—Projects are often associated with or in direct support of politically sensitive or controversial government programs and initiatives. Projects connected to politically hot files are at risk of unplanned changes in direction. They may also suffer from underfunding because of an arbitrary funding cap.
Policy or program synchronization—If projects are conceived and launched while unresolved legislative or government policy issues are pending, mid-course changes may be required or the entire initiative may have to be abandoned if legislation, regulations, or policy changes do not materialize as expected.
Interdepartmental or multi-jurisdictional factors—Cross-departmental or multi-jurisdictional projects often have complex governance processes with widely spread authority. As a result, making decisions in project time is difficult. Achieving agreement on a common set of requirements can be exceedingly time-consuming and involve onerous compromises.
Changing conditions or instability—The rate and impact of change during the life of the project are significant risk factors. Once conceived, the general environment in which the project is to be implemented is assumed to be known and relatively stable. However, any project that lasts more than 12 months is likely to experience changes beyond project or departmental control. Failure to properly assess the impact of changes on the project can be a major contributor to project overruns.

Examples of risk factors within departmental control include:

Interference or distraction from normal departmental operations—Project work does not mix well with ongoing program responsibilities. Project groups that are not properly insulated from the day-to-day business of the host department often get distracted and diverted into non-project work, resulting in less time and energy spent on the project.
Introduction of new technology—When the proposed business solution is based on the introduction of new or unfamiliar technologies, great care should be taken to ensure that all implications are fully considered. Software, data, and communications compatibility, backup, and recovery are some of the major risk areas due to changes in technical architecture or infrastructure.
Geographical factors—Project activities sometimes span different parts of the country or even the world. In extraordinary cases, projects may have to be conducted with a team that is scattered across several centres. The challenges associated with dispersed project work and deployment to remote areas can contribute significantly to the project's risk. Having business and IT resources located in the same centre is highly desirable. Otherwise, complications can arise, often requiring strong management discipline to compensate for them.

Impact assessment

Project executives and managers should not underestimate the impact of identified project risks. The probability of their occurrence and potential impact on the costs and schedule can be assessed at each stage in the project. From these assessments, mitigation strategies and plans can be developed and refined as circumstances change. Excessive risk at any point demands a revision, or outright rejection, of the proposed approach.

Mitigation and contingency plan

In a supportive project environment, management seeks to insulate and protect the project from changes (other than those that are absolutely necessary). The immediate response of project-savvy senior departmental executives to an externally imposed change is to determine how it could affect the project schedule and budget. There should be no reluctance to replan the project or seek revised project authorities when change will materially affect the project. When the probability and impact of risks are underestimated, project sponsors and stakeholders may have a false sense of security about the project.

Contingency planning is the management tool used to deal with the inevitable risks stemming from changes and unknown factors that arise over the life of the project. For each identified risk that threatens the delivery of the project or the achievement of business outcomes, there needs to be a formal, achievable, and accepted countermeasure. A fully elaborated risk management plan that properly identifies the risks, assesses their impact, and proposes contingency measures is the vehicle through which to manage risks.

Primary Issues Table: Topic 6—Project structure and mechanics

Issue	Significance and Importance
The Project Management Office (PMO)	The PMO is the control and decision support centre for the project as well as the centre of expertise for project disciplines. Too often regarded as simply an administrative overhead function, an effective PMO actually plays a vital and proactive support role in the successful execution of large IT-enabled projects.
Project organization	A current project organization chart is an important management tool. It is a focal point that helps ensure responsibilities, accountabilities, and resource commitments are clearly defined and understood throughout the life of the project.
Facilities and working conditions	Satisfactory working conditions are important for ensuring and maintaining project resource productivity and morale. Such considerations as location and the availability of office space, workstations, meeting rooms, equipment, materials, and services (including a cafeteria and recreation) should be priorities right from the start. Given that the assignment may extend over several years, an office environment that is comparable to permanent conditions is an understandable expectation.
Project management methodology and discipline	While process is not a substitute for effective management, in a large project it is extremely important that the PMO properly coordinate and enable informed decision making. Sophisticated tools and methodologies will assist in areas such as scheduling, estimating, change management, quality assurance, problem tracking and resolution, risk management, progress tracking, status reporting, and continuous improvement. The cost-estimating methodology has to be comprehensive and based on realistic assumptions, and it needs to consider post-implementation support and operations. The PMO is responsible for maintaining a detailed and up-to-date project schedule with appropriate, clearly defined effort and cost estimates for each project stage. The schedule must be grounded on valid assumptions, constraints, and dependencies. The cost model must be linked to the work breakdown structure (WBS) to provide a rational basis for estimating levels of effort and assuring that estimates are defensible. Also essential is a financial reporting system capable of timely expenditure tracking and assisting with earned value tracking (i.e., a process that shows the cost of work done relative to the value of the work done). PMO processes have to deal with scope, schedule, and financial changes, while factoring them into an effective risk management program. To minimize business impact, effective issues management and problem management systems have to be organized and deployed to rapidly resolve issues and problems and produce solutions. Similarly, the ability to manage change in the project is one of the greatest contributors to success. An effective issues management process recognizes and captures change requests, provides an assessment of impacts, directs proposals to the appropriate approval levels, and makes appropriate adjustments to schedules and estimates. QA and test strategies, supported by tool sets when practical, play a significant role in large-scale development and integration projects. An IV&V function is normally a feature of the PMO QA process. Effective status reporting is essential to managing the project at the executive level and at the level of the project team. Key reporting metrics focus on costs, time, risks, and issues affecting the schedule.
Estimating and related assumptions	Estimating can contribute greatly to project difficulties. Sometimes estimates are inappropriately prepared to fit into a predetermined project budget. At other times, detailed estimates are prepared before enough information is available to accurately do so. Staff developing the estimates may be inexperienced or the models used for estimating may not be suitable for the project situation. Some people argue that many project overruns are actually cases of underestimating. Since estimating inherently involves making assumptions, the reasonableness of the assumptions can be problematic. Finally, estimates are best when they are cross-checked for reasonableness and calibrated against early results. These two steps are missing from many project environments.
Review and audit	Reviews and audits are a proactive way of determining whether the project is on track and, if not, what remedial action is necessary. Using external parties to provide an independent assessment of the project's progress and status can often assist in detecting major problems early and making timely mid-course corrections. Reviews can be synchronized with key decision or approval points to provide timely inputs. Care should be taken to prevent the review process from burdening or disrupting the project's progress. This can be achieved by building review activities into the project schedule at the outset.

Primary Issues Table: Topic 7—Business requirements

Issue	Significance and Importance
Business model and architecture	Business outcomes need to be mapped to an overall business model and business architecture, which in turn should map to business requirements as dictated by the degree of detail required for each project phase. The business model and business architecture must be confirmed before detailed business requirements can be specified.
Requirements definition	The most significant area of early activity in large IT-enabled projects involves processes for analyzing, defining, and approving business requirements. In most large projects, there is much at stake in getting requirements completely, accurately, and unambiguously specified, documented, and accepted. Weakness in this area leads to repeated reworking, not only of the requirements themselves but of systems components already under development. Reworking in later stages of development and deployment causes extremely costly delays. Diligence and care in establishing requirements are essential to prevent delays, reworking, and cost overruns in large projects. Business requirements have to be sufficiently detailed to support project costing and planning. If an iterative development methodology is used, checkpoints to constrain time and costs need to be established and closely monitored, pending finalization of the full set of requirements. Once the statement of requirements is approved, the process for managing requests for changes to the approved requirements must be rigorously controlled.
Security and privacy	While business requirements are being defined, it is also important to identify and carefully consider the privacy and security requirements associated with the proposed system. Experience has shown that these can be very difficult to retrofit later in the project without major costs and schedule delays.

Issue

Significance and Importance

Business model and architecture

Business outcomes need to be mapped to an overall business model and business architecture, which in turn should map to business requirements as dictated by the degree of detail required for each project phase. The business model and business architecture must be confirmed before detailed business requirements can be specified.

Requirements definition

The most significant area of early activity in large IT-enabled projects involves processes for analyzing, defining, and approving business requirements. In most large projects, there is much at stake in getting requirements completely, accurately, and unambiguously specified, documented, and accepted. Weakness in this area leads to repeated reworking, not only of the requirements themselves but of systems components already under development. Reworking in later stages of development and deployment causes extremely costly delays. Diligence and care in establishing requirements are essential to prevent delays, reworking, and cost overruns in large projects.

Business requirements have to be sufficiently detailed to support project costing and planning. If an iterative development methodology is used, checkpoints to constrain time and costs need to be established and closely monitored, pending finalization of the full set of requirements.

Once the statement of requirements is approved, the process for managing requests for changes to the approved requirements must be rigorously controlled.

Security and privacy

While business requirements are being defined, it is also important to identify and carefully consider the privacy and security requirements associated with the proposed system. Experience has shown that these can be very difficult to retrofit later in the project without major costs and schedule delays.

Primary Issues Table: Topic 8—Technology

Issue	Significance and Importance
Technology architecture and standards	In the majority of project situations, technology issues are secondary to the business considerations at play. However, the extent to which the technology under consideration conforms to federal government and departmental standards for technology architecture is an important issue, particularly for solutions that involve networking and information sharing with other organizations.
Maturity	When the primary technologies involve tried and proven familiar products that build upon existing infrastructure, the risk of deployment may be relatively small. The situation is far riskier, however, when the project approach to the business problem depends on leading-edge technology and when experience in this technology at project inception is limited or does not exist. In other cases, projects may involve replacement of legacy systems that call for expertise in obsolete technologies for which supply is limited. This brings a different kind of risk into play.
Performance and availability	Clearly understanding the project's technical performance parameters at an early stage will greatly aid the technology solution. Failure to prove whether the solution can meet performance objectives has caused several large project failures. Changing the technology platform solution for the project is very expensive once construction has started. Even adding additional capacity can trigger procurement and funding issues as well as delays. In projects where performance is treated as an afterthought, retrofitting and reconfiguring technologies to satisfy performance requirements can lead to costly delays and overruns, not to mention unacceptable levels of dissatisfaction among systems users.

Primary Issues Table: Topic 9—Procurement

Issue	Significance and Importance
Procurement strategy and plan	Procurement remains one of the most difficult project variables to manage and predict in the public sector environment. A key factor in successful project delivery is early assessment of the project's dependency on procurement to determine how this critical external risk will be managed throughout the project life cycle. Projects that depend on procurement outcomes will likely experience problems affecting schedule and cost, particularly when project and departmental staff are not experienced in managing these activities or when Public Works and Government Services Canada (PWGSC) is not appropriately engaged and committed. An effective procurement strategy and plan considers: Goods and services to be acquired; Procurement vehicles available for use; Contracting models available (turnkey project delivery, systems integrator or project manager, additional staff, etc.); Time and effort to develop a request for proposal (RFP), including the time required for vendor response, to evaluate RFP responses with a view to engaging a capable supplier, and to negotiate and award contracts; and Capability of available resources to undertake the procurement work and manage the resulting contract(s). Procurement and contracting activities need to be closely linked and integrated with the project plan and schedule. Experience has shown that delays in these areas are a frequent source of project delays and cost overruns.
Contract management	The responsibilities of the supplier(s) and the project group should be completely understood when the contract is awarded. A healthy and constructive relationship with the supplier(s) is fundamental to the project's success and can be achieved only when expectations on the part of all parties are clearly set out. The contract management process must be clearly articulated and accepted by all parties and must cover such areas as progress reporting, change and issues management, dispute resolution, acceptance and sign-off for performance and deliverables, milestone payments, and extensions.

Issue

Significance and Importance

Procurement strategy and plan

Procurement remains one of the most difficult project variables to manage and predict in the public sector environment. A key factor in successful project delivery is early assessment of the project's dependency on procurement to determine how this critical external risk will be managed throughout the project life cycle. Projects that depend on procurement outcomes will likely experience problems affecting schedule and cost, particularly when project and departmental staff are not experienced in managing these activities or when Public Works and Government Services Canada (PWGSC) is not appropriately engaged and committed.

An effective procurement strategy and plan considers:

Goods and services to be acquired;
Procurement vehicles available for use;
Contracting models available (turnkey project delivery, systems integrator or project manager, additional staff, etc.);
Time and effort to develop a request for proposal (RFP), including the time required for vendor response, to evaluate RFP responses with a view to engaging a capable supplier, and to negotiate and award contracts; and
Capability of available resources to undertake the procurement work and manage the resulting contract(s).

Procurement and contracting activities need to be closely linked and integrated with the project plan and schedule. Experience has shown that delays in these areas are a frequent source of project delays and cost overruns.

Contract management

The responsibilities of the supplier(s) and the project group should be completely understood when the contract is awarded. A healthy and constructive relationship with the supplier(s) is fundamental to the project's success and can be achieved only when expectations on the part of all parties are clearly set out. The contract management process must be clearly articulated and accepted by all parties and must cover such areas as progress reporting, change and issues management, dispute resolution, acceptance and sign-off for performance and deliverables, milestone payments, and extensions.

Primary Issues Table: Topic 10—Human factors

Issue	Significance and Importance
Staffing	Projects happen because of people—the right people at the right time. It is critical that HR processes respond to project needs, which means that staffing needs to occur in project time. For this to happen, project management must engage the departmental HR organization early on and maintain a close working relationship to ensure that the HR group understands the personnel requirements and is prepared to deal with these requirements as they arise. A project team that has the skills and experience to do the required work, particularly in the key positions, will significantly boost productivity and greatly help mitigate risk. Learning on the job in a larger project is expensive. The cohesiveness and productivity that come from team members having worked together on other projects can be a major benefit.
Training	Even with careful attention to project staffing, rarely will project team members bring the full set of skills and competencies needed to perform their assigned roles. This is particularly true in the deployment of new or unfamiliar processes and technologies. This situation can be addressed through a training plan that is fully integrated with the overall project plan, meaning that budget and time allocations are included in the schedule.
Teamwork, morale, and communication	The efficiency and effectiveness of the project is highly dependent on teamwork within the small working units and across the entire project. Decision making through consensus is not necessarily an effective management style in projects, and certain conflicts and tensions can be expected. To a degree, disagreements can actually be healthy if the various opinions are expressed respectfully and brought to bear in an effective manner. Care should be taken by managers, however, not to allow differences of opinion to escalate unreasonably to the detriment of the common objective. Having working units in the same location is highly desirable; however, when impossible, management visits, proactive team communications, regular project meetings, and other measures need to be taken to compensate for this. A project environment does not offer the stability associated with permanent organizational structures. This can lead to staff members feeling isolated as well as uncertain and insecure about their immediate and future roles. Project management must communicate frequently, openly, and honestly with team members about the project as a whole and about the units and individuals involved. Unsubstantiated rumours can lead to poor staff morale, which has proven to be a significant factor in the failure of large projects.

Issue

Significance and Importance

Staffing

Projects happen because of people—the right people at the right time. It is critical that HR processes respond to project needs, which means that staffing needs to occur in project time. For this to happen, project management must engage the departmental HR organization early on and maintain a close working relationship to ensure that the HR group understands the personnel requirements and is prepared to deal with these requirements as they arise.

A project team that has the skills and experience to do the required work, particularly in the key positions, will significantly boost productivity and greatly help mitigate risk. Learning on the job in a larger project is expensive. The cohesiveness and productivity that come from team members having worked together on other projects can be a major benefit.

Training

Even with careful attention to project staffing, rarely will project team members bring the full set of skills and competencies needed to perform their assigned roles. This is particularly true in the deployment of new or unfamiliar processes and technologies. This situation can be addressed through a training plan that is fully integrated with the overall project plan, meaning that budget and time allocations are included in the schedule.

Teamwork, morale, and communication

The efficiency and effectiveness of the project is highly dependent on teamwork within the small working units and across the entire project. Decision making through consensus is not necessarily an effective management style in projects, and certain conflicts and tensions can be expected. To a degree, disagreements can actually be healthy if the various opinions are expressed respectfully and brought to bear in an effective manner. Care should be taken by managers, however, not to allow differences of opinion to escalate unreasonably to the detriment of the common objective.

Having working units in the same location is highly desirable; however, when impossible, management visits, proactive team communications, regular project meetings, and other measures need to be taken to compensate for this.

A project environment does not offer the stability associated with permanent organizational structures. This can lead to staff members feeling isolated as well as uncertain and insecure about their immediate and future roles. Project management must communicate frequently, openly, and honestly with team members about the project as a whole and about the units and individuals involved. Unsubstantiated rumours can lead to poor staff morale, which has proven to be a significant factor in the failure of large projects.

Primary Issues Table: Topic 11—Funding

Issue	Significance and Importance
Funding strategy and plan	It is important from the outset of the project that a close working relationship be established between project management and departmental financial authorities. Project, departmental, and TBS representatives must fully understand the preparation and approval of estimates and submissions. This can assist greatly in expediting funds as needed to support the project schedule. In cases where project funding is not secure and approvals for project authorities and the release of funds are dealt with phase by phase, a significant amount of project management time may be sidetracked to managing the funding and approval process.

Issue

Significance and Importance

Funding strategy and plan

It is important from the outset of the project that a close working relationship be established between project management and departmental financial authorities. Project, departmental, and TBS representatives must fully understand the preparation and approval of estimates and submissions. This can assist greatly in expediting funds as needed to support the project schedule.

In cases where project funding is not secure and approvals for project authorities and the release of funds are dealt with phase by phase, a significant amount of project management time may be sidetracked to managing the funding and approval process.

Primary Issues Table: Topic 12—Implementation and deployment

Issue	Significance and Importance
Organizational and business readiness	The business or program group must complete all transformational and process re-engineering activities and have a deployment readiness plan to be properly positioned to receive and use the new system. The business group, right from the outset, must pay attention to deployment of the solution prior to system implementation. Among the areas requiring attention are: Business transformation and process re-engineering; Data conversion and cleanup; User and stakeholder communication; User procedures and training; System deployment and migration, including fallback provisions; and Outcomes evaluation and follow-up. The timely and informed participation of business managers and staff in planning, scheduling, and executing these items is essential to avoid confusion and frustration among users, who must continue to maintain service levels throughout the transition to the new system. The capacity to handle the peak workloads during implementation and deployment, particularly on the business side, is easily underestimated.
Construction, deployment, and support	Completion of construction is determined through testing and acceptance criteria and procedures and, ultimately, through formal acceptance of all specified project deliverables. These include such items as system components, data conversion, system and user documentation, training courses and materials, contracted goods and services, and security certification. Phased deployment can begin before construction has been completed, but project management must not lose sight of the point at which full system development will be complete. Project credibility is based on how smoothly deployment goes. This is largely dependent on thorough planning that has made allowances for almost any eventuality. Poor performance in transition may lead to user rejection or outright system avoidance and failure. Support involves a well-orchestrated combination of business and technical resources, often backed by third-level support from a supplier. Service level agreements need to be well defined, communicated, and accepted, and execution must proceed accordingly, especially during the peak transition period. Otherwise, users will quickly become frustrated and ultimately hostile to the system. Recovery from this kind of situation is difficult. Data conversion is often a challenging undertaking and can easily become an Achilles heel, so to speak, when implementing a large project. Depending on how much conversion of historical material is involved and the quality of the existing data, data requirements for most new systems call for careful planning and should be given early attention by both business and technical areas. They must consider what data is to be captured or acquired, whether the quality of existing data or data collection processes is sufficient, and which data, and how much, should be converted or archived. If quality needs to be improved or validated, this must be accommodated in the schedule. Data requirements for new systems (particularly COTS products) are typically more demanding and are frequently underestimated in project planning. The processes needed to manage and support the new system in production must be ready prior to implementation. These include standard processes such as change and problem management and release management. If these processes are not in place or are incomplete, service quality is certain to suffer and business value will be compromised.

Issue

Significance and Importance

Organizational and business readiness

The business or program group must complete all transformational and process re-engineering activities and have a deployment readiness plan to be properly positioned to receive and use the new system. The business group, right from the outset, must pay attention to deployment of the solution prior to system implementation. Among the areas requiring attention are:

Business transformation and process re-engineering;
Data conversion and cleanup;
User and stakeholder communication;
User procedures and training;
System deployment and migration, including fallback provisions; and
Outcomes evaluation and follow-up.

The timely and informed participation of business managers and staff in planning, scheduling, and executing these items is essential to avoid confusion and frustration among users, who must continue to maintain service levels throughout the transition to the new system. The capacity to handle the peak workloads during implementation and deployment, particularly on the business side, is easily underestimated.

Construction, deployment, and support

Completion of construction is determined through testing and acceptance criteria and procedures and, ultimately, through formal acceptance of all specified project deliverables. These include such items as system components, data conversion, system and user documentation, training courses and materials, contracted goods and services, and security certification. Phased deployment can begin before construction has been completed, but project management must not lose sight of the point at which full system development will be complete.

Project credibility is based on how smoothly deployment goes. This is largely dependent on thorough planning that has made allowances for almost any eventuality. Poor performance in transition may lead to user rejection or outright system avoidance and failure.

Support involves a well-orchestrated combination of business and technical resources, often backed by third-level support from a supplier. Service level agreements need to be well defined, communicated, and accepted, and execution must proceed accordingly, especially during the peak transition period. Otherwise, users will quickly become frustrated and ultimately hostile to the system. Recovery from this kind of situation is difficult.

Data conversion is often a challenging undertaking and can easily become an Achilles heel, so to speak, when implementing a large project. Depending on how much conversion of historical material is involved and the quality of the existing data, data requirements for most new systems call for careful planning and should be given early attention by both business and technical areas. They must consider what data is to be captured or acquired, whether the quality of existing data or data collection processes is sufficient, and which data, and how much, should be converted or archived. If quality needs to be improved or validated, this must be accommodated in the schedule. Data requirements for new systems (particularly COTS products) are typically more demanding and are frequently underestimated in project planning.

The processes needed to manage and support the new system in production must be ready prior to implementation. These include standard processes such as change and problem management and release management. If these processes are not in place or are incomplete, service quality is certain to suffer and business value will be compromised.

1.4 Gate-by-gate review points and lines of enquiry

This section is organized by gate. A table for each gate sets out the appropriate topics, issues, and suggested lines of enquiry.

Complete descriptions of each gate, including their respective critical issues, are set out in Section 3 of The Independent Reviewer's Handbook.

Gate 1—Strategic assessment and concept

This gate should answer the key questions "What do we want to do?" and "Why?"The objectives at this early stage are to test the wisdom and appropriateness of the proposed undertaking, and to ensure that key stakeholders are identified, and that everyone understands what is to be done and why. A half- to one-day workshop session is typical, preceded by reading of any available early project documents. The Gate 1 review seeks to arm the review sponsor with considerations that should be addressed before the next phase of the project and, in some cases, may actually dictate going back to the drawing board before proceeding further.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	What business problem will this initiative address? Is the initiative politically driven or program-driven? Is the initiative a stated government priority? Is it tied to legislation or regulations? When will the legislation or regulations be approved and finalized? Is the initiative considered mission-critical? To whom—the federal government? The department? What are the consequences of not undertaking the initiative? To whom? Will the IT-enabled project under consideration help the initiative achieve its proposed outcomes? Has a business imperative and project concept document been prepared to address broad business questions such as "What is to be done?" and "Why?"
	Vision and scope	Is the undertaking clearly and concisely stated so that all levels of the organization understand it? Is the project's scope clearly defined? Is it clear what the project will and will not do? What business model or processes will change? Who will be affected by the project? Are all the stakeholders and users clearly identified? Are these parties aware of the proposed undertaking and how it will affect them?
	Expected business outcomes	Are the expected business outcomes clearly set out? Is the value anticipated from these outcomes clearly linked to the business imperative? If this project fails, which government and departmental expectations will not be met?
	Alignment with policies, programs, and strategies	Is the project aligned with any of the strategic outcomes listed in departmental reports to Parliament? How well is the project aligned with government-wide strategies and cross-departmental initiatives (e.g., Service Canada, TBS shared systems, Secure Channel Network)? Is the alignment with departmental portfolio and business strategies well understood? Is alignment with the IM/IT portfolio well understood?
	Assumptions, constraints, and dependencies	What underlying assumptions are fundamental to the project's success? Are they realistic and reasonable? Are constraints and their effects well understood? Are these limitations acceptable? Have external dependencies that could materially affect the project been taken into account? Have the associated risks been evaluated? Are they manageable?
2. Sponsorship, leadership, and governance	Ownership and commitment	Can the project sponsor and owner of the initiative be readily identified? Is the project sponsor's experience appropriate for the scale of the project? Are the roles and responsibilities associated with ownership well understood and accepted? Does the business area fully understand the extent of its required involvement, or do business managers think that the IM/IT group will manage everything? Is the organizational culture such that the business area readily assumes responsibility? Is the business area prepared to be the final arbiter on the business rules that describe the business solution and support the IT system under construction?
	Leadership and management roles	Are the roles of project leader and project manager clearly understood? Have business managers previously been clients for a systems development or systems integration initiative? Are business leaders truly committed and will they do whatever is necessary to make the project succeed—even if it puts their career on the line? Have IM/IT managers had prior experience with large-scale systems development? Are the assigned leadership and management roles appropriate for this project? Could the project management structure be problematic—for example, in terms of the business relationship with IM/IT or matrix versus line management? If so, are responsibilities and accountabilities clear?
	Stakeholder roles and responsibilities	Are all stakeholders identified and appropriately represented? Are roles and responsibilities clear, documented, and accepted? Are stakeholders committed to the business outcomes and effectively engaged in governance?
	Governance and decision making	Have the project governance framework and process been adequately considered? Have participants been identified? Are all valid interests adequately represented?
3. Concept and approach	Project classification and business change management	Is the extent of business change recognized? Has the project been classified accordingly and put into the appropriate project category (i.e., sustaining, tactical, evolutionary, or transformational)? Do the project sponsor, management, and stakeholders understand the implications of this class of project? Does the department have recent experience with this class of initiative or a comparable project? If so, how did it perform, and what lessons were learned?
	Alternatives and options	Has a reasonable range of options been considered before arriving at the project concept? What decision criteria are used in selecting an option? To what extent have make-versus-buy and technology options been considered at this stage?
	Project sizing and packaging	Is there an order of magnitude sizing and time frame associated with the project concept? How well do these correspond to the business imperatives? How does the scale and complexity of the proposed project compare with other federal government projects? How do they compare with the other projects that the department successfully carried out? Will the project have enough staff, financing, and other resources under its current limits?
4. Organizational readiness and capacity	Organizational capacity	Given what we know about the proposed project at this stage, does the department have the requisite project culture and proven project management capacity derived from undertaking comparable initiatives? Is the size of the initiative reasonably proportional to the size of the department? Can the department provide the required project support services (e.g., HR, finance, procurement, logistics, training)? Does the project group regularly undertake similar projects? Does it have PMO structures and disciplines in place, or the equivalent? Is there an appropriate project delivery infrastructure? Is the department presently involved in other major project undertakings? What comparable project has the department undertaken in the past several years? What were the results and lessons learned?
	Executive and management capacity	What is the level of IT project literacy among departmental executives? Do key executives understand project management vocabulary? Is there a risk that they might commit the project to unreasonable objectives or make decisions without consulting project managers and stakeholders? What experience has departmental management had in working directly with large projects? Do the executives and managers have a sound understanding of the prevailing business issues? Is this understanding from hands-on experience? Will there be significant turnover of key managers during the life of the project?
	Business capacity	Does the business program clearly see itself as the project business owner? Does the responsible business program manager have experience managing an IT-enabled project of this scale? Does the business line (or lines) involved have well-established centralized program management policies, discipline, and capabilities? Is the business program within headquarters ready and able to help decentralized operating units reach agreement and closure on business requirements?
	IM/IT capacity	Given what we know about the proposed project at this stage, are the size and strength of the IM/IT organization proportional to the scale of the proposed undertaking? Does the IM/IT organization have recent experience in delivering large projects? Do the backgrounds and experience of the leaders and managers in the systems development group include significant project development experience? Do the key systems development staff members have project management certification? Are there other major IM/IT initiatives competing for project resources?
5. Risk management	Identification of risks	Has project complexity been fully taken into account? Have business managers identified high-level risks as a basis for a preliminary risk assessment?
	Impact assessment	For the risks identified, has a proper impact assessment been performed? Have formal risk assessment tools been used?
	Mitigation and contingency plan	For each high-level risk assessed, have potential contingencies and mitigation measures been identified? Are there any potential showstoppers for which mitigation measures are unclear?

Gate 2—Project approach

This gate should confirm that the approach to address the business problem or opportunity is wise and sound by testing its feasibility and appropriateness. As in the Gate 1 review, this gate is not intended to be an overly burdensome undertaking and can often be performed in a one-day workshop session. The project group is expected to provide information on the approach envisaged for the undertaking—a discussion that should flow logically from the focus on the business problem and alignment issues of the Gate 1 review.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	Revalidate lines of enquiry for Gate 1.
	Vision and scope	Revalidate lines of enquiry for Gate 1. As well, ask: Are the vision and scope consistently and widely understood and accepted? Does the scope clearly state what is within the project's range and what is outside it?
	Expected business outcomes	Revalidate lines of enquiry for Gate 1. As well, ask: Is it clear how the project will help achieve business outcomes, i.e., are key project deliverables sufficiently detailed to be clearly linked to outcomes? Has a means been established to measure progress and overall success in qualitative and quantifiable terms? How will project deliverables link to the business requirements at each stage (e.g., a requirements traceability matrix tool)? What approval process will validate outcomes at each stage? Is the approval authority clear?
	Alignment with policies, programs, and strategies	Revalidate lines of enquiry for Gate 1. As well, ask: Have all prevailing legislative, regulatory, and policy issues affecting the project been identified and considered? Will policy interpretations make it onerous to carry out the project? Should more favourable interpretations be pursued? Is the relationship of the project to other programs fully understood? Can associated risks be resolved within the project time frame without adverse effects?
	Assumptions, constraints, and dependencies	Revalidate lines of enquiry for Gate 1.
2. Sponsorship, leadership, and governance	Ownership and commitment	Revalidate lines of enquiry for Gate 1.
	Leadership and management roles	Revalidate lines of enquiry for Gate 1.
	Stakeholder roles and responsibilities	Revalidate lines of enquiry for Gate 1.
	Governance and decision making	Revalidate lines of enquiry for Gate 1. As well, ask: Have terms of reference been established and membership designated for governance bodies? Are the differences between the decision-making responsibilities and accountabilities of project management and those of the governance process clear? Do all participants in the governance process appear to have a legitimate interest and a role in decisions affecting the project?
3. Concept and approach	Project classification and business change management	Revalidate lines of enquiry for Gate 1. As well, ask: Do all phases of the project fit within the project classification selected in Gate 1? If not, has the proposed approach taken this fully into account? Will this systems development be created from scratch with little or no use of existing systems components? What were the deciding factors—for example, was it difficult maintaining and upgrading an aging technology platform? If a greenfield approach is selected, are the business rules formally documented apart from the existing application code? Can they be easily derived or extracted from the existing code? What are the implications for older legacy systems—for example, data conversion, continuity of service, phase-out? Will staff familiar with their operation and maintenance support them? While legacy systems are being replaced, can their service delivery standards continue to be maintained? Have the technology architecture and infrastructure to be used for the project solution been considered? In particular, is this likely to involve mature, proven technology understood by the IM/IT organization? Is the adoption of a leading-edge solution a possibility? Does the project team understand in principle how data for the new system will be converted, captured, populated, and validated? Have currency, integrity, and fallback issues been given preliminary consideration? Have business and technical managers thought about the transition and system migration approach to the proposed system—for example, phased-in or big bang? Have business and IT people close to the existing system operations been involved in practical considerations of the various approaches? Have high-level project milestones been defined at this point? Has a preliminary project exit strategy been set out? What business value could be expected at each stage? Should the project be suspended or terminated? Are the project approach and rationale described in enough detail to assess the approach's feasibility and appropriateness?
	Alternatives and options	Revalidate lines of enquiry for Gate 1. As well, ask: Was the decision made objectively? Without bias? Or were any options prematurely rejected? Have possible trade-offs been well articulated? Is the rationale for the choice of approach properly documented?
	Project sizing and packaging	Revalidate lines of enquiry for Gate 1. As well, ask: How is the project to be phased, and what are the approximate time frames? What systems development approach is being considered—for example, make-versus-buy considerations such as outsourcing, customized in-house development, systems integration with prime contractor, extent of COTS usage? Have management implications been taken into account for the approaches chosen? Have business and IT personnel close to the operation of the existing system reviewed how the various approaches being considered could be implemented? Using the requisite government procurement mechanisms, how much time and resource effort is needed to purchase what the project needs? What staffing is needed, approximately? Roughly how much funding is required? What facilities are required, in general terms? Is all of this feasible?
	System transition strategy	Have the feasibility and risks of the transition and implementation strategies been considered—for example, phased versus big-bang system migration? Does the project group understand how much more money and how many more resources will be needed when it becomes necessary to run parallel operations at times?
	Service delivery and management strategy	How will service delivery be managed? Are stakeholders aware of how the business will change? Do they know what costs to anticipate? Do they know how to plan to capture benefits? Has a concept of operations statement been prepared for service roles, responsibilities, and performance objectives?
	Gating and exit strategies	Have major gates representing high-level approval milestones and logical exit points been considered? Do proposed exit strategies take into account the residual business value achieved?
4. Organizational readiness and capacity	Organizational capacity	Revalidate lines of enquiry for Gate 1.
	Executive and management capacity	Revalidate lines of enquiry for Gate 1.
	Business capacity	Revalidate lines of enquiry for Gate 1. As well, ask: Is the business area familiar with the discipline, processes, and tools required to develop and document business rules, information flows, client interactions, and a service delivery approach? Does the business area understand and accept its lead role and responsibilities in key project areas such as budget submissions, requirements statement, workload forecasts, change management, data conversion, testing, training, and business resumption planning? Does business management understand and accept the different roles and responsibilities that it and IM/IT have? Does the business area accept responsibility for managing overall stakeholder relationships and communications for the project?
	IM/IT capacity	Revalidate lines of enquiry for Gate 1. As well, ask: Does IM/IT's capacity support and satisfactorily align with the systems development approach selected and also with the approximate project sizing and packaging? Is there an established project infrastructure available in terms of proven processes and tools? Are the development methods and tool sets to be used already in use? Or will they be acquired and skills developed during the project.
5. Risk management	Identification of risks	Revalidate lines of enquiry for Gate 1. As well, ask: Would further refinement of the business proposition, as well as the concept and approach, raise any implications?
	Impact assessment	Revalidate lines of enquiry for Gate 1. As well, ask: Would further refinement of the business proposition, as well as the concept and approach, have any implications?
	Mitigation and contingency plan	Revalidate lines of enquiry for Gate 1. As well, ask: Would further refinement of the business proposition, as well as the concept and approach, have any implications?

Gate 3—Business case and general readiness

This gate should answer the key question "How?" The most feasible project options are considered here and the preferred option recommended. The project approach is now fully articulated. A high-level project plan has been tabled, upon which preliminary costing is based. The business case should include an investment rationale and describe outcomes to be achieved, as well as their justification relative to the proposed cost. Project cost and schedule estimates for the entire project should be in the 40-per-cent range. (Ranges assume a typical development or integration project where there are many unknowns at the early stages. An infrastructure replacement project might have a much smaller estimating range. See Section 3, "The Gating Framework," in The Independent Reviewer's Handbook.) This gate calls for a workshop review; however, in very large or complex projects, it may be a quick review.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	Revalidate lines of enquiry for previous gate(s). As well, ask: Has anything occurred in the project environment to force the original business imperative to be altered? If so, what adjustments have been made?
	Vision and scope	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the vision for the approved project still serve the intended purpose? Is the vision unambiguous and fully accepted by stakeholders and project members? If amendments have been proposed and accepted, has their significance been fully factored into plans and activities up to this point? Are the scope's boundaries clear and complete? Have any questions arisen about whether certain items are out of the project's scope? Have those questions been resolved satisfactorily?
	Expected business outcomes	Revalidate lines of enquiry for previous gate(s). As well, ask: Is the TBS outcome management process being followed? Can senior executives confirm that expected outcomes have remained consistent as the project proceeded—for example, is the value proposition still valid? Is this a good investment of public funds? Have any adjustments to the business proposition or project plans been necessary? Have plans been developed and approved in cases where gaining benefits involves realigning resources or downsizing? Will this involve changing the way in which service is delivered to the public, to other levels of government, or to third-party communities? Are all parties in agreement? Do project outcomes translate into business outcomes? Do performance indicators and measurements remain valid?
	Alignment with policies, programs, and strategies	Revalidate lines of enquiry for previous gate(s).
	Assumptions, constraints, and dependencies	Revalidate lines of enquiry for previous gate(s).
2. Sponsorship, leadership, and governance	Ownership and commitment	Revalidate lines of enquiry for previous gate(s).
	Leadership and management roles	Revalidate lines of enquiry for previous gate(s).
	Stakeholder roles and responsibilities	Revalidate lines of enquiry for previous gate(s).
	Governance and decision making	Revalidate lines of enquiry for previous gate(s). As well, ask: Has project governance been fully described? Is it in effect? Have all necessary oversight mechanisms been defined? Has it been determined when independent reviews and audits of the project should occur?
3. Concept and approach	Project classification and business change management	Revalidate lines of enquiry for previous gate(s). As well, ask: Are the concept and approach now formally embodied in a complete business case? Is there a shared view and acceptance of the transformational aspects of the project? At this stage, does the business case provide sufficient detail on the following: Selected approach and its justification? How the project will be developed? How risks are to be managed? Service delivery and management strategy, i.e., has a concept of operations document been prepared? Deliverables, performance, and expected outcomes, and how they will be measured and validated? Does the business case strongly support a decision to proceed? If not, what additional areas need attention?
	Alternatives and options	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the business case clearly set out the analysis of alternatives and options? Does it present a sound rationale for the project approach selected? Is there bias toward one option?
	Project sizing and packaging	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the business case satisfactorily cover the points raised on sizing and packaging under the preceding gates? Does the project sizing take into account the full costs from conception to implementation?
	System transition strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the business case present the transition and implementation strategy to be deployed? If parallel operations are required, have cost and schedule changes been taken into account?
	Service delivery and management strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the business case address the general concept of operations for service delivery and management?
	Gating and exit strategies	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the business case set out the major gates for the project, taking into account the residual business value proposition at each gate? Are review points synchronized with major funding approvals?
4. Organizational readiness and capacity	Organizational capacity	Revalidate lines of enquiry for previous gate(s).
	Executive and management capacity	Revalidate lines of enquiry for previous gate(s).
	Business capacity	Revalidate lines of enquiry for previous gate(s).
	IM/IT capacity	Revalidate lines of enquiry for previous gate(s). As well, ask: Does IM/IT's capacity support and satisfactorily align with the development approach selected and also with estimated project sizing and packaging?
5. Risk management	Identification of risks	Revalidate lines of enquiry for previous gate(s). As well, ask: Has the TBS project complexity and risk assessment tool been run? What was the outcome?
	Impact assessment	Revalidate lines of enquiry for previous gate(s). As well, ask: Have potential project delays for major risks been realistically assessed? What could those delays cost?
	Mitigation and contingency plan	Revalidate lines of enquiry for previous gate(s). As well, ask: How do the project group and, in particular, senior executives view contingency reserves? Do they see them as a cushion to compensate for poor planning, or as genuine insurance against unknown costs and events outside of the project's control?
6. Project structure and mechanics	PMO	Do project participants fully understand and appreciate the role and importance of the PMO in achieving project success? Have the PMO model and structure been established? Does the PMO have a sufficient number of qualified employees or contractors in relation to the size and complexity of the project? Have project management methodologies and tools been considered and selected?
	Project management methodology and discipline	Is a proven cost model to be used, and are PMO staff familiar with its use? Does the project approach use several estimating methodologies such as top down, bottom up, or rules of thumb? Have trade-offs been assessed with respect to cost, schedule, scope, and quality? Have all goods and services been included? Has the full project life cycle been considered? Is the strategy to constrain the project on cost, time, or scope? Is the life cycle expenditure profile realistic—for example, not front-end loaded?
	Estimating and related assumptions	Do assumptions that affect costing provide a sufficient basis upon which to develop estimates?(This may require revalidation of the Gate 1 line of enquiry for assumptions, constraints, and dependencies.) Are estimates that are sensitive to changes in conditions based on assumptions? How sensitive are they? Has a sensitivity analysis been performed? Consider salaries, contractor rates, inflation, ratio of employees to contract staff, productivity, staff retention and turnover, and other representative areas.
7. Business requirements	Business model and architecture	Are business team members who are well-grounded in the business processes involved in the proposed solution? Do they have experience in preparing a statement of business requirements for a comparable project? To what extent will they have to live with or be accountable for results? Has a conceptual view of the business model and architecture been proposed to serve as the foundation for the development of business requirements? Does it have the endorsement of the appropriate business authority?
	Requirements definition	Has a methodology for gathering and elaborating business requirements been selected? Do project business members have experience with the proposed approach or one similar to it? Is the definition of requirements based on current business approaches or on a future higher-value business approach? Do those responsible for requirements definition clearly understand the goals of the project in this regard? Has the process and authority for approval of business requirements at each level of definition been established?
	Security and privacy	Up to this stage, has it been recognized that security and privacy are part of project planning? Are qualified and dedicated project resources assigned to these areas of responsibility? Have preliminary threat and risk assessment and privacy impact assessment exercises been conducted? Have the results been appropriately considered in business and technical planning up to this point?
8. Technology	Architecture and standards	Has the full range of technology required for the project solution been considered? This includes computing, storage, network, utilities, and more. Does the proposed technology architecture conform to federal government enterprise standards? Does it conform to departmental standards? Is the technology solution available immediately, or does it have to be acquired?
	Maturity	Is the proposed solution using existing technologies? To what extent? Does the solution involve products and services familiar to the organization? Are they currently in use? Are IM/IT training requirements associated with new products and services being considered? Is there full appreciation for the benefits of supplier maturity and the maturity of associated support dependencies?
	Performance and availability	Have preliminary systems performance and availability specifications been mapped to the expected business outcomes? Do the operating parameters of these systems differ radically from the systems that the IM/IT group is accustomed to supporting?
9. Procurement	Procurement strategy and plan	Have the following been adequately considered: Goods and services to be acquired per the project schedule? Procurement vehicles to be used? Level of effort and expertise involved in developing RFPs, bid evaluations, contract negotiation, and ongoing contract management? Departmental procurement support and services available to the project? Risks and impacts inherent in the procurement process?
10. Human factors	Staffing	At this stage, have the following been adequately considered: Availability and scheduling of required expertise? Contracting and scheduling of external expertise? Training requirements for new methods and products? Departmental HR support and services available to the project?
11. Funding	Funding strategy and plan	Have the following been adequately considered: Departmental project budget processes and approvals schedule? Treasury Board funding submission processes and approvals schedule? Departmental financial support and services available to the project?
12. Implementation and deployment	Organizational and business readiness	Is the organization ready to receive and absorb the new system with minimal disruption to the business? For example, are the following areas being considered: Business organizational transformation and process re-engineering? Deployment and support, including user and stakeholder communication and documentation, and user training? Data clean-up, conversion, and error handling? Problem resolution and change management? Acceptance and sign-off procedures?

Gate 4—Project charter / PMP

This gate should address business case unknowns. A complete project charter, high-level project management plan (PMP), and definition of the business solution are prerequisites for this gate. Any business case unknowns should be resolved here (or a plan in place to resolve them). Project cost and schedule estimates for the entire project should be ±25 per cent (see Section 3, "The Gating Framework," in The Independent Reviewer's Handbook). In the case of larger projects, this gate calls for a full review; however, a quick review may be appropriate for smaller, low-risk initiatives.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	Revalidate lines of enquiry for previous gate(s).
	Vision and scope	Revalidate lines of enquiry for previous gate(s).
	Expected business outcomes	Revalidate lines of enquiry for previous gate(s). As well, ask: Has the outcomes realization model for the project been finalized and approved? Has a preliminary outcomes realization plan been prepared? Have all project deliverables been specified in high-level terms? Do they map to the outcomes realization model and plan?
	Alignment with policies, programs, and strategies	Revalidate lines of enquiry for previous gate(s).
	Assumptions, constraints, and dependencies	Revalidate lines of enquiry for previous gate(s). As well, ask: Are all underlying assumptions affecting the project clearly stated? Has their validity been challenged? Are they appropriately recognized in the charter and preliminary PMP? Have all the project constraints and dependencies been fully recognized, analyzed, and clearly stated? Are they appropriately considered in the charter and preliminary PMP? Is an appropriate management process in place to track the continuing validity and implications of these factors and to respond accordingly?
2. Sponsorship, leadership, and governance	Ownership and commitment	Revalidate lines of enquiry for previous gate(s).
	Leadership and management roles	Revalidate lines of enquiry for previous gate(s).
	Stakeholder roles and responsibilities	Revalidate lines of enquiry for previous gate(s).
	Governance and decision making	Revalidate lines of enquiry for previous gate(s).
3. Concept and approach	Project classification and business change management	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the project charter clearly confirm and back up the concept and approach with a rationale and supporting detail? Is the preliminary PMP consistent with the accepted concept and approach?
	Alternatives and options	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the project charter confirm the results of business case decisions on alternatives and options? Is the degree of detail provided sufficient to support project planning such as the preliminary PMP?
	Project sizing and packaging	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the project charter confirm the results of business case decisions on project sizing and packaging? Is the degree of detail provided sufficient to support project planning such as the preliminary PMP?
	System transition strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the PMP confirm the transition and implementation strategy to be deployed?
	Service delivery and management strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the PMP provide a concept of operations?
	Gating and exit strategies	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the PMP cover project gating and exit strategies? Does it indicate how they relate to residual business value and funding approvals?
4. Organizational readiness and capacity	Organizational capacity	Revalidate lines of enquiry for previous gate(s).
	Executive and management capacity	Revalidate lines of enquiry for previous gate(s).
	Business capacity	Revalidate lines of enquiry for previous gate(s).
	IM/IT capacity	Revalidate lines of enquiry for previous gate(s).
5. Risk management	Identification of risks	Revalidate lines of enquiry for previous gate(s).
	Impact assessment	Revalidate lines of enquiry for previous gate(s).
	Mitigation and contingency plan	Revalidate lines of enquiry for previous gate(s). As well, ask: Are risks being carried forward without mitigation? As risks become less probable, are mitigation plans being adjusted and contingency provisions reduced accordingly?
6. Project structure and mechanics	PMO	Revalidate lines of enquiry for previous gate(s).
	Project organization	Does the project have a complete and current organization chart describing organizational units, designated functions, reporting relationships, and resourcing? Are key project resources identified and their availability committed? Are part-time personnel involved? How are the associated commitments assigned and tracked?
	Facilities and working environment	Is the project office environment, i.e., its location, space, furniture and workstations, equipment, materials, meeting facilities, and services, conducive to productivity? Is physical or geographical separation of project units a factor? If so, how is this being addressed?
	Project management methodology and discipline	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan provide for a formal project management methodology? Are major project milestones set out in the preliminary PMP? Is the sequencing of critical activities logical, and are the estimates for elapsed time realistic? Are interdependencies clearly represented? Is a formal project scheduling tool, such as Primavera or MS Project, to be used? Are PMO roles and responsibilities specified? Is the project change management process in place? Is there a high-level WBS with approximations of effort? Are cost estimates linked to work packages? Is the project supported by a suitable financial reporting system? Can this system provide project expenditures to date? Is it possible to establish and maintain linkages between the project schedule, cost estimates, and work packages? How is earned value to be tracked? Will estimates for completion costs be available? Has a process been established for status reporting to regularly provide management with a clear picture of what has been accomplished, where the project is in its life cycle, and what remains to be done? Does reporting cover the following factors: Progress against schedule (slippage, delays)? Meaningful budget and expenditure tracking (cost overruns)? Impact of cost and schedule variations? Remedial actions to be taken?
	Estimating and related assumptions	Review the validity of the project cost model and the refinement of estimates, as appropriate for this stage. Ask questions about the estimating methodology used, the past experience of the estimators, the reasonableness of the underlying assumptions, and any techniques in place to cross-check estimates for reasonableness.
	Review and audit	Does the preliminary PMP allow for project reviews and audits as an integral planning element? Does the project schedule indicate the points or gates at which these independent examinations should occur? How will the required services be obtained in a timely manner? These include factors such as secondments and procurement. Are certain funding approvals conditional on the results of project reviews or audits? Will governance decisions be synchronized with the availability of review and audit results to ensure that recommendations are dealt with expediently and project delays prevented?
7. Business requirements	Business model and architecture	Revalidate lines of enquiry for previous gate(s). As well, ask: Has the business model been sufficiently developed to provide the following: A high-level solution description and functional design? Transformation and process re-engineering strategies, as applicable? High-level functional and performance requirements? A high-level data model? Does the project charter include the preliminary business model and architecture?
	Requirements definition	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the preliminary PMP describe how business requirements will be defined and elaborated upon? Does this task have the appropriate priority and resources? Is the methodology available, and is staff prepared?
	Security and privacy	Revalidate lines of enquiry for previous gate(s). As well, ask: Are security and privacy requirements covered in the project charter and in the preliminary PMP requirements definition and elaboration processes?
8. Technology	Architecture and standards	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the project charter provide a description of the proposed technical solution, including how it will meet performance and availability requirements? If the solution involves a COTS product, do those developing business requirements understand the implications of this approach? Are they clear to those developing the technical specifications? Does the preliminary PMP address high-level technical specifications? Does it propose how and when these specifications will be satisfied to meet the project schedule—for example, with upgrades or new technologies?
	Maturity	Revalidate lines of enquiry for previous gate(s).
	Performance and availability	Revalidate lines of enquiry for previous gate(s).
9. Procurement	Procurement strategy and plan	Revalidate lines of enquiry for previous gate(s). As well, ask: Is the procurement plan fully linked to the preliminary PMP in terms of schedule, level of effort, and risk? Has PWGSC been involved in the development of the procurement strategy and plan? Is PWGSC firmly committed to the schedule?
10. Human factors	Staffing	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the preliminary PMP estimate the required expertise over the project's life cycle in terms of full-time equivalents, contract resources, etc.? Have key positions been filled or committed?
10. Human factors	Training	Have project training needs, such as career development and job training, been assessed? Have appropriate provisions been made in the budget and schedule? Are these needs covered in the preliminary PMP?
11. Funding	Funding strategy and plan	Revalidate lines of enquiry for previous gate(s).
12. Implementation and deployment	Organizational and business readiness	Revalidate lines of enquiry for previous gate(s).
12. Implementation and deployment	Deployment and support plans	Have strategies been developed and has preliminary planning started for high-level deployment activities? This includes problem and change management, data conversion and migration, user training and documentation, and transition and system migration. Are these areas covered in the project charter and preliminary PMP?

Gate 5—Detailed project plan and functional specifications

This gate should confirm the completeness and feasibility of the detailed project plan and definition of requirements. Decisions to go ahead with major spending commitments and to make major procurement choices (such as issuing RFPs or awarding contracts) take place here. All major unknowns have been sufficiently researched to provide assurance that high-impact risks have been effectively mitigated. Estimates for the entire project should be ±15 per cent (see Section 3, "The Gating Framework," in The Independent Reviewer's Handbook). (Note that project contingency should never fall below 10 to 15 per cent.) This gate calls for a quick review or full review, depending on the size and complexity of the project.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	Revalidate lines of enquiry for previous gate(s). As well, ask: Has anything changed? Is the detailed project plan consistent with the business imperative? Will the solution address the business problem or opportunity originally presented? By way of a reality check, ask the question, "Should the project continue?"
	Vision and scope	Revalidate lines of enquiry for previous gate(s). As well, ask: Has anything changed? Are the project activities in the detailed business plan properly focussed and restricted to what is essential to realize the business solution?
	Expected business outcomes	Revalidate lines of enquiry for previous gate(s). As well, ask: Has anything changed? Are the expected project outcomes and deliverables clearly set out in the detailed business plan? Do project participants and stakeholders accept them?
	Alignment with policies, programs, and strategies	Revalidate lines of enquiry for previous gate(s). As well, ask: Is project realignment required if changes have occurred to the business imperative, the vision and scope of the project, or the expected business outcomes?
	Assumptions, constraints, and dependencies	Revalidate lines of enquiry for previous gate(s). As well, ask: Are the underlying project assumptions, constraints, and dependencies still valid?
2. Sponsorship, leadership, and governance	Ownership and commitment Leadership and management roles Stakeholder roles and responsibilities Governance and decision making	Revalidate lines of enquiry for previous gate(s). As well, ask: Do the meeting minutes and records of decisions of the governance bodies demonstrate that the overall process is functioning effectively? Are the appropriate issues being addressed? Are the right decision makers engaged? Are meetings and decisions timely so as to avoid costly delays? Are decisions being effectively communicated?
3. Concept and approach	Project classification and business change management	Revalidate lines of enquiry for previous gate(s). As well, ask: Is the detailed project plan consistent with the project classification? Does the extent and impact of the project on the business fully correspond to all activities addressed?
	Alternatives and options	Revalidate lines of enquiry for previous gate(s). As well, ask: Have all management choices and decisions been satisfactorily addressed to establish clear direction for the project going forward?
	Project sizing and packaging	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan adequately address project sizing and packaging?
	System transition strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan elaborate on the transition and implementation strategy to be deployed?
	Service delivery and management strategy	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan address all the activities necessary to implement the approved concept of operations?
	Gating and exit strategies	Revalidate lines of enquiry for previous gate(s).
4. Organizational readiness and capacity	Organizational capacity	Revalidate lines of enquiry for previous gate(s).
	Executive and management capacity	Revalidate lines of enquiry for previous gate(s).
	Business capacity	Revalidate lines of enquiry for previous gate(s).
	IM/IT capacity	Revalidate lines of enquiry for previous gate(s).
5. Risk management	Identification of risks	Revalidate lines of enquiry for previous gate(s).
	Impact assessment	Revalidate lines of enquiry for previous gate(s).
	Mitigation and contingency plan	Revalidate lines of enquiry for previous gate(s). As well, ask: Are there any remaining risks at this stage that cannot be mitigated or reduced within the overall project cost and time constraints?
6. Project structure and mechanics	PMO	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan cover all project management activities necessary for project execution? Have they been assigned to the PMO? Is the PMO now fully functional in the areas represented by the Gate 5 review points? Completeness and feasibility of the detailed project plan. Completeness and feasibility of the definition of requirements.
	Project organization	Revalidate lines of enquiry for previous gate(s). As well, ask: Are resource requirements fully defined? Does the detailed project plan fully define the project organization and reporting structure?
	Facilities and working environment	Revalidate lines of enquiry for previous gate(s).
	Project management methodology and discipline	Revalidate lines of enquiry for previous gate(s). As well, ask: Are PMO staff trained and prepared to use the project management methodology? Does the detailed project plan contain a complete project schedule, including the WBS and substantive cost estimates? Is there a complete list of deliverables? Does the plan link milestones and expenditures well enough to permit effective project tracking and reporting? Are the responsibilities and processes for monitoring and reporting project progress against the plan appropriate? Are they working? Have testing and QA strategies been finalized, and are these functions now resourced? Have a high-level acceptance plan and acceptance criteria been developed? Is IV&V included in the detailed project plan? To whom does this team report? Does the detailed project plan allow for effective problem and change management procedures and tools? Is there a requirements traceability matrix? Is the formal change management process in effect? Do change management records indicate that it is performing effectively? Have arrangements been considered for suppliers who may be required to complement in-house problem management?
	Estimating and related assumptions	Revalidate lines of enquiry for previous gate(s). As well, ask: As estimating techniques should now be fully refined, with enough detail and accuracy suitable for pre-execution, are the assumptions used in estimating fully documented? Have efforts been made to validate the reasonableness of the estimates, where feasible? Do the estimates make allowances for risks and assumptions, or are they planned backward to fit budgets and schedules? Have any cross-checking techniques been used to validate the reasonableness of estimates? Were adjustments made when discrepancies were observed? Is it reasonable for the project to commit to schedule and cost, considering the quality of the estimating up to this stage?
	Review and audit	Revalidate lines of enquiry for previous gate(s).
7. Business requirements	Business model and architecture	Revalidate lines of enquiry for previous gate(s). As well, ask: Has a comprehensive review of the business model and architecture been completed? Does the detailed project plan cover final decisions on the business model and architecture?
	Requirements definition	Revalidate lines of enquiry for previous gate(s). As well, ask: Have business requirements been approved in enough detail to support the detailed project plan elements? Do business managers clearly understand the compromises deemed necessary in the requirements? (An example would be to contain the project's scope or adapt to a COTS solution.) Have the requirements been converted into a preliminary high-level functional design? Has this been subjected to a design review? Has the design been explored sufficiently to ensure, in principle, a solution to complex areas? Have proofs of concept, prototypes, or other tools been used to confirm the requirements or design approach?
	Security and privacy	Revalidate lines of enquiry for previous gate(s). As well, ask: Are all issues that were uncovered in threat and risk assessments and privacy impact assessments now mitigated? Have the security certification strategy and plan been completed?
8. Technology	Architecture and standards	Revalidate lines of enquiry for previous gate(s). As well, ask: Has a comprehensive review of the technology architecture been completed? Is there a formal record ofdecisions taken? Does the detailed project plan confirm the technology solution and demonstrate how it will be delivered to meet the schedule? If the project group is proposing any deviations from either federal government or departmental standards for architecture or technology, has an appropriate rationale been prepared? If the deviations involve any risks, are there plans to mitigate them?
	Maturity	Revalidate lines of enquiry for previous gate(s). As well, ask: Have the requirements for contract resourcing and staff training for any new or unfamiliar technologies been included in the detailed project plan?
	Performance and availability	Revalidate lines of enquiry for previous gate(s). As well, ask: Do performance and availability specifications take into account the final decisions on business requirements covered in the detailed project plan?
9. Procurement	Procurement strategy and plan	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan confirm the procurement strategy and plans? Does it address all necessary procurement activities such as RFPs, bid evaluation, and contract award? Are activities synchronized with the project schedule? Have issues such as performance penalties and dispute resolution been anticipated and taken into account? Have procurement and contracting activities for necessary goods and services been confirmed in the PWGSC work plan? Have procurement officers been committed for these activities?
10. Human factors	Staffing	Revalidate lines of enquiry for previous gate(s). As well, ask: Is project staffing sufficiently complete, and does it represent an A Team, so to speak?
	Training	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan satisfactorily cover costs and scheduling for project training?
	Teamwork, morale, and communication	Is there evidence—for example, in the project plans—that management has considered measures to foster productive working relationships? Such measures could include team-building exercises; regular staff updates on project progress, achievements, and issues; team and individual recognition and rewards; and access to employee assistance programs.
11. Funding	Funding strategy and plan	Revalidate lines of enquiry for previous gate(s). As well, ask: Have the necessary departmental and Treasury Board funding approvals been satisfactorily addressed in project planning? Are submissions and approvals proceeding accordingly? Are departmental and TBS financial representatives in agreement with proposed funding arrangements? Are they providing the appropriate guidance and assistance?
12. Implementation and deployment	Organizational and business readiness	Revalidate lines of enquiry for previous gate(s). As well, ask: Are all critical success factors accounted for in planning?
12. Implementation and deployment	Deployment and support plans	Revalidate lines of enquiry for previous gate(s). As well, ask: Does the detailed project plan address business change management? Does it fully explain deployment plans, including data conversion and migration, problem and change management, training, and transition?

Gate 6—Construction complete and deployment readiness

This gate represents a major point of approval for business readiness. It should verify that the system under development is ready for implementation and that the project is fully prepared for a successful deployment. Because the construction and deployment phases of a large and complex project may occur over a period of several months and may require intermediate health checks, the department, based on the given situation and the degree of risk, should decide on the number, timing, and focus area for health checks during construction and deployment. For example, depending on the project structure, a gate could be established at the completion of a major development release or at the completion of a major rollout and deployment to a specified set of users. In the case of large iterative development methodologies, gates and review points would ideally be established to ensure that development is moving toward closure—that the iterations would not continue indefinitely or until the project runs out of time and money. This gate, then, is focussed on the sign-offs for what has been delivered and on the clear agreement for what remains to be done. Gate 6 calls for a quick or full review, depending on the size, risk, and complexity of the project.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative	Revalidate lines of enquiry for previous gate(s).
	Vision and scope	Revalidate lines of enquiry for previous gate(s).
	Expected business outcomes	Revalidate lines of enquiry for previous gate(s).
	Alignment with policies, programs, and strategies	Revalidate lines of enquiry for previous gate(s).
	Assumptions, constraints, and dependencies	Revalidate lines of enquiry for previous gate(s).
2. Sponsorship, leadership, and governance	Ownership and commitment	Revalidate lines of enquiry for previous gate(s).
	Leadership and management roles	Revalidate lines of enquiry for previous gate(s).
	Stakeholder roles and responsibilities	Revalidate lines of enquiry for previous gate(s).
	Governance and decision making	Revalidate lines of enquiry for previous gate(s).
3. Concept and approach	Project classification and business change management	Revalidate lines of enquiry for previous gate(s).
	Alternatives and options	Revalidate lines of enquiry for previous gate(s).
	Project sizing and packaging	Revalidate lines of enquiry for previous gate(s).
	System transition strategy	Revalidate lines of enquiry for previous gate(s).
	Service delivery and management strategy	Revalidate lines of enquiry for previous gate(s).
	Gating and exit strategies	Revalidate lines of enquiry for previous gate(s).
4. Organizational readiness and capacity	Organizational capacity	Revalidate lines of enquiry for previous gate(s).
	Executive and management capacity	Revalidate lines of enquiry for previous gate(s).
	Business capacity	Revalidate lines of enquiry for previous gate(s).
	IM/IT capacity	Revalidate lines of enquiry for previous gate(s).
5. Risk management	Identification of risks	Revalidate lines of enquiry for previous gate(s).
	Impact assessment	Revalidate lines of enquiry for previous gate(s).
	Mitigation and contingency plan	Revalidate lines of enquiry for previous gate(s).
6. Project structure and mechanics	PMO	Revalidate lines of enquiry for previous gate(s).
	Project organization	Revalidate lines of enquiry for previous gate(s).
	Facilities and working environment	Revalidate lines of enquiry for previous gate(s).
	Project management methodology and discipline	Revalidate lines of enquiry for previous gate(s).
	Review and audit	Revalidate lines of enquiry for previous gate(s).
7. Business requirements	Business model and architecture	Revalidate lines of enquiry for previous gate(s).
	Requirements definition	Revalidate lines of enquiry for previous gate(s).
	Security and privacy	Revalidate lines of enquiry for previous gate(s).
8. Technology	Architecture and standards	Revalidate lines of enquiry for previous gate(s).
	Maturity	Revalidate lines of enquiry for previous gate(s).
	Performance and availability	Revalidate lines of enquiry for previous gate(s).
9. Procurement	Procurement strategy and plan	Revalidate lines of enquiry for previous gate(s).
10. Human factors	Staffing	Revalidate lines of enquiry for previous gate(s).
	Training	Revalidate lines of enquiry for previous gate(s).
	Teamwork, morale, and communication	Revalidate lines of enquiry for previous gate(s).
11. Funding	Funding strategy and plan	Revalidate lines of enquiry for previous gate(s).
12. Implementation and deployment	Organizational and business readiness	Revalidate lines of enquiry for previous gate(s). As well, ask: Are supplier support and maintenance contracts in place? Are business line and user service level agreements approved? Are release management and support plans and procedures available? Are performance and availability measures established and tools in place? Have the users been informed of release and deployment plans and procedures, including those for problem reporting and change management? Is there a clear sense among project team members that everything is in place and ready to go, or are there substantive matters still to be addressed?
12. Implementation and deployment	Construction, deployment, and support	Revalidate lines of enquiry for previous gate(s). As well, ask: Is construction complete for all system components? This includes having unit systems and sub-systems tested, successful data conversion, system and user documentation available, training courses and materials ready, contracted deliverables accepted, security certification performed, etc. Have plans, facilities, and criteria been established for user acceptance testing? Do business lines and users have the capacity to handle system transition? Will parallel operations be required? Are plans to move into the production environment completed? Have acceptance criteria been established with the operations team? Is system documentation ready for the maintenance and support teams? Has the support plan been developed? Are support training materials and courses available? Is the technology deployment or rollout and software distribution plan complete? Has the business deployment plan, including business process re-engineering, been completed? Has the business deployment training plan been completed? Are training materials available? Have all the data migration and conversion activities necessary for system migration been completed? Are system migration plans complete? This includes reversion and rollback procedures, backup or disaster recovery, business resumption, etc.

Gate 7—Post-implementation review

This gate should confirm completion, assess the extent to which the project has achieved its goals, and provide an assessment of value for money. Gate 7 typically occurs approximately six months following project completion. A review at this point can also catalogue the lessons learned during the project—those identified by the project group and those captured by independent reviewers. This gate calls for a workshop or a full review, depending on the size and complexity of the project.

Topic	Issue	Suggested Lines of Enquiry
1. Business proposition	The business imperative Vision and scope Expected business outcomes	Can an assessment of business outcomes be made at this point? If not, is there a formal plan to do this? When will an assessment be made and by whom? Did the project satisfy the business imperative? Did it position the business as envisioned to realize the expected outcomes? Were the expected business outcomes realized within project costs and schedules? Did the business case justification for the project prove to be appropriate, reasonable, and realistic? If the project were starting over, what—if anything—would be done differently? Was the project a success?
2. Sponsorship, leadership, and governance	Ownership and commitment	Did the project sponsor or owner demonstrate clear and consistent responsibility and accountability throughout the project?
	Leadership and management roles	Were project leadership and management effective? If not, what were the shortcomings, and how could they have been remedied?
	Stakeholder roles and responsibilities	Was stakeholder engagement appropriate and effective? If not, what were the shortcomings, and how could they have been remedied?
	Governance and decision making	Were the governance framework and processes appropriate and effective? If not, what were the shortcomings, and how could they have been remedied?
3. Concept and approach	Project classification and business change management	Was the project class properly assigned at the outset? Did the project management approach suit the classification? Was the approach complex enough given the extent of business change?
3. Concept and approach	Project sizing and packaging	Was the project appropriately sized and phased?
4. Organizational readiness and capacity	Organizational capacity Executive and management capacity Business capacity IM/IT capacity	Did organizational capacity and readiness in the executive and management areas, business area, and IM/IT area measure up to the project undertaking? If not, what issues were encountered, and how were they addressed? How would the organization rate on the Software Engineering Institute capability maturity model?
5. Risk management	Identification of risks Impact assessment Mitigation and contingency plan	Was risk management effective? Were impacts reassessed as the project advanced, and were contingency provisions adjusted accordingly?
6. Project structure and mechanics	PMO Project organization	Were the project management structure and organization suited to the scale and complexity of the undertaking?
	Facilities and working environment	Were the project's facilities and working environment good for productivity and morale? Did these factors affect the project's success and, if so, to what extent?
	Project management methodology and discipline	Did the methodology employed provide the appropriate degree of control and discipline for the scale and complexity of the project? Have the following post-implementation activities been completed: Archiving of project information and deliverables? Knowledge transfer and transition to successful managed service? Capture of lessons learned? Project close-out or closing report completed, including assessment against original objectives?
	Review and audit	Did reviews and audits help identify critical issues and ensure their timely remediation?
7. Business requirements	Business model and architecture Requirements definition Security and privacy	In the final analysis, did the business model, architecture, and requirements map well to business outcomes? Are the users or clients satisfied that processes have improved? Was any significant rework necessary during implementation and deployment to compensate for deficiencies in the requirements? If so, how could this have been avoided?
8. Technology	Performance and availability	Have stated systems performance and availability requirements been met? Are users or clients satisfied? If not, are timely remedial measures possible? At reasonable costs?
9. Procurement	Procurement strategy and plan	Has the completion of contractual obligations been formally confirmed? What residual obligations remain, and how will they be managed? Were the appropriate contracted goods and services acquired as planned and with minimal impact on the project schedule? Were the services provided by PWGSC timely and appropriate? Has feedback been provided? Were supplier relationships conducive to the project's success, and do they remain favourable for future support relationships? Are appropriate supply and service arrangements in place for ongoing systems support and maintenance?
10. Human factors	Staffing Training Teamwork, morale, and communication	Do project team members on both the business and the IM/IT sides feel that they were part of a successful venture and that they have benefited from the experience? Are steps being taken to ensure that the employee skills, training, and experience acquired throughout the project are properly recognized and recorded so that they can be applied to future project undertakings?
11. Funding	Funding strategy and plan	Did the project submission and approvals process ensure timely and appropriate funding allocations tosupport planned budgetary needs? Was the project completed within the planned budget?
12. Implementation and deployment	Construction, deployment, and support	Is the project considered complete, and has it been signed off accordingly? Have all deliverables been met and accepted, all project activities concluded, and all resources redeployed? Are there any unresolved issues remaining at this point? How will they be dealt with? Are the lessons learned from the overall project experience being captured and analyzed to be used in future endeavours? Will the results be made available for continuous improvement program purposes within the department and throughout the federal government?

Appendix A—Abbreviations

COTS:: Commercial off-the-shelf
HR:: Human resources
IM:: Information management
IT:: Information technology
IV&V:: Independent Verification and Validation
PMO:: Project Management Office
PMP:: Project management plan
PWGSC:: Public Works and Government Services Canada
QA:: Quality assurance
RFP:: Request for proposal
TBS:: Treasury Board of Canada Secretariat
WBS:: Work breakdown structure

Appendix B—Attributes of Different Types of Reviews

	Workshop Review	Quick Review	Full Review	Health Check Review
When appropriate	For early project gates (such as Gates 1 and 2) For specific issues	For Gates 3 to 7 Assesses risk in project components without the expense of a full review	For Gates 3 to 6 (and occasionally Gate 7)	Instead of looking at a gate's expectations, it focusses on factors such as governance, budgeting,working relationships, etc.
Time required (elapsed)	About one week	Less than three weeks	Four to six weeks	Three to five weeks
Size of review team	Two to three members. There is 100% participation by members.	Three to five members. There is 80% participation by members.	Three to five members. There is 50 to 70% participation by members.	Two to three members. There is 50 to 70% participation by members.
Tasks	Reading preparation Workshop with review sponsor, project staff, and review team Possibly conducting a few targeted interviews Making conclusions/ recommendations Debriefing and reporting	Reading preparation Listening to presentations by project group Doing key interviews Consolidating/ validating observations Making conclusions/ recommendations Debriefing and reporting	Reading preparation Listening to presentations by project group Doing 15 to 20 interviews Consolidating/ validating observations Making conclusions/ recommendations Debriefing and reporting	Reading preparation Listening to presentations by project group Doing interviews Consolidating/ validating observations Making conclusions/ recommendations Debriefing and reporting
Deliverables	Report or review presentation	Review presentation	Review presentation Executive summary presentation Executive briefing note Report Executive summary of report	Review presentation Executive summary presentation Executive briefing note Report Executive summary of report

Appendix C—Related Policies and Publications

Treasury Board Policies and Policy Instruments

Management of Major Crown Projects
Policy on Access to Information
Policy on Government Security
Policy on Information Management
Policy on Investment Planning – Assets and Acquired Services
Policy on Management of Information Technology
Policy on Management, Resources and Results Structures
Policy on Privacy Protection
Policy on the Management of Projects
Framework for the Management of Risk
Standard for Organizational Project Management Capacity
Standard for Project Complexity and Risk

Other Resources of Interest

Office of the Privacy Commissioner of Canada
UK Office of Government Commerce

Page details

Date modified:: 2012-11-07

Language selection

Search

Review Topics for Enquiry: A summary of issues by project gate

Table of Contents

Foreword

1. Reviewer's Master Outline of Topics for Enquiry

1.1 Topics for enquiry tables: An overview

1.2 How to use this outline

1.3 Primary issues to be reviewed

Primary Issues Table: Topic 1—Business proposition

Primary Issues Table: Topic 2—Sponsorship, leadership, and governance

Primary Issues Table: Topic 3—Concept and approach

Primary Issues Table: Topic 4—Organizational readiness and capacity

Primary Issues Table: Topic 5—Risk management

Primary Issues Table: Topic 6—Project structure and mechanics

Primary Issues Table: Topic 7—Business requirements

Primary Issues Table: Topic 8—Technology

Primary Issues Table: Topic 9—Procurement

Primary Issues Table: Topic 10—Human factors

Primary Issues Table: Topic 11—Funding

Primary Issues Table: Topic 12—Implementation and deployment

1.4 Gate-by-gate review points and lines of enquiry

Gate 1—Strategic assessment and concept

Gate 2—Project approach

Gate 3—Business case and general readiness

Gate 4—Project charter / PMP

Gate 5—Detailed project plan and functional specifications

Gate 6—Construction complete and deployment readiness

Gate 7—Post-implementation review

Appendix A—Abbreviations

Appendix B—Attributes of Different Types of Reviews

Appendix C—Related Policies and Publications

Treasury Board Policies and Policy Instruments

Other Resources of Interest

Page details

Review Topics for Enquiry: A summary of issues by project gate

We have archived this page and will not be updating it.

We have archived this page and will not be updating it.

Table of Contents

Foreword

1. Reviewer's Master Outline of Topics for Enquiry

1.1 Topics for enquiry tables: An overview

1.2 How to use this outline

1.3 Primary issues to be reviewed

Primary Issues Table: Topic 1—Business proposition

Primary Issues Table: Topic 2—Sponsorship, leadership, and governance

Primary Issues Table: Topic 3—Concept and approach

Primary Issues Table: Topic 4—Organizational readiness and capacity

Primary Issues Table: Topic 5—Risk management

Primary Issues Table: Topic 6—Project structure and mechanics

Primary Issues Table: Topic 7—Business requirements

Primary Issues Table: Topic 8—Technology

Primary Issues Table: Topic 9—Procurement

Primary Issues Table: Topic 10—Human factors

Primary Issues Table: Topic 11—Funding

Primary Issues Table: Topic 12—Implementation and deployment

1.4 Gate-by-gate review points and lines of enquiry

Gate 1—Strategic assessment and concept

Gate 2—Project approach

Gate 3—Business case and general readiness

Gate 4—Project charter / PMP

Gate 5—Detailed project plan and functional specifications

Gate 6—Construction complete and deployment readiness

Gate 7—Post-implementation review

Appendix A—Abbreviations

Appendix B—Attributes of Different Types of Reviews

Appendix C—Related Policies and Publications

Treasury Board Policies and Policy Instruments

Other Resources of Interest

Page details