Audit of Time Reporting Final – Report

(Approved March 26, 2013)

Table of contents

Executive summary

Context

Throughout the Public Service Commission (PSC) branches, time reporting is used for various purposes, including cost recovery, accountability reporting and capitalization, as well as project planning and monitoring. Two key time-reporting systems are used. The Revenue Management System (RMS) is used both for client service request management and for client billing. EZ-Time is used for planning, allocation of resources and capitalization costing of in-house software development. In addition, some systems have been developed in-house to respond to specific needs.

Rationale

A recent internal audit of the Audit and Data Services Branch (ADSB) raised concerns with respect to the use and reliability of time reporting. No audit had been conducted on time reporting at the PSC overall.

The PSC’s Internal Audit Committee approved the Audit of Time Reporting in February 2012, as part of the 2012-2013 risk-based internal audit plan.

Objective

To assess whether PSC time reporting systems and processes are adequate and effective for producing reliable information in a cost-effective manner.

Overall conclusions

The auditors found that PSC time reporting systems and processes are only partially adequate and effective. The overall reliability of time reporting varied with the system and the work unit. Technology was an underlying factor that set limits on the usability of the data and on the process used to gather and verify them.

On one hand, RMS is performing its primary function of providing accurate and reliable data for billing quite well. This is largely due to the fact that the RMS process and underlying technology encourage diligence both in entering data, on the part of employees, and in providing feedback, on the part of supervisors. Nonetheless, RMS has a few process and technology alignment issues that make both co-ordination between units and the data entry approval process more cumbersome than they need be. On the other hand, EZ-Time has both process and technology deficiencies that negatively affect its capacity to meet the objectives of time reporting at the PSC. The many shortcomings of EZ-Time are indicative of a system that was developed to meet the needs of a small group, and that was subsequently adopted in an uncontrolled fashion by other units.

Stakeholder needs are partially being met. Both systems have been affected by governance deficiencies, with consequent shortcomings such as diminished cost-effectiveness and limitations in their capacity to link with other systems. An RMS governance framework is now in place to ensure that future changes are controlled and based on user feedback; however, no cost-benefit analysis has ever been undertaken to establish the input and operational requirements of RMS and to understand how to optimize the system to accommodate the data collection process. Information Technology Services Directorate management has referred to EZ-Time as an “unapproved system,” meaning that it has not gone through a software testing and approval cycle, and is expected to be accepted “as is” (aside from changes to the coding tags) by any unit that chooses to deploy it. As an unapproved system, EZ-Time system development has been very low-cost; however, its many shortcomings limit its effectiveness.

Management has provided robust action plans, which we believe will remedy the shortcomings noted.

1. Introduction

The PSC’s Internal Audit Committee approved the Audit of Time Reporting in February 2012, as part of the 2012-2013 risk-based internal audit plan. This is the first time that an internal audit of time reporting has been conducted for the Commission.

This audit focuses on RMS and EZ-Time, which are the two most important time reporting systems at the PSC.

This is a horizontal audit, involving more than one Office of Primary Interest (OPI); hence, the Internal Audit Directorate (IAD) established an advisory committee to co-ordinate and clear the key milestones for the audit. Each branch designated a representative who was responsible for helping plan and conduct audit work and for validating and clearing the results related to that branch.

Since the recent Internal Audit of Audit and Data Services Branch Management has already covered time reporting within this branch, IAD included ADSB only peripherally in this engagement. ADSB did elect to have a representative on the advisory committee.

2. Background

Time reporting is the process of recording and classifying employees’ time at work. Time data captured in the PSC’s systems are used for cost recovery, capitalization, project planning and monitoring, along with the related accountability reporting.

  • Within the Staffing and Assessment Services Branch (SASB), RMS time data are used primarily for billing of client services rendered. It is also used for project management of client service requests, and by senior management to suggest process improvements, evaluate business trends and determine the efficiency of full-time equivalent (FTE) usage in the various regions.
  • SASB also uses EZ-Time to facilitate the monitoring of expenditures and ensure effective management and allocation of resources related to the maintenance and development of the Public Service Resourcing System (PSRS).
  • Within the Corporate Management Branch (CMB), time reported in EZ-Time is used for planning and, in some cases, for costing Information Technology (IT) system development and maintenance.
  • In ADSB, time reporting is used as a project planning tool. (A recent audit raised awareness of the risks associated with the use and reliability of ADSB’s time reporting system and the accuracy of the system’s data; hence, ADSB was not a focus of the current audit.)

A number of smaller, ad-hoc time reporting systems, such as specialized Excel spreadsheets, are also in place throughout the PSC. These systems, which are not the focus of this audit, serve various purposes:

  • Investigations Branch and some teams within Policy Branch measure performance against service standards; and
  • Some smaller groups in CMB record time for planning and decision-making.

Data from time-reporting systems are used to calculate items that appear in the PSC’s financial statements:

  • EZ-time data are used to calculate capitalization of assets under development, the value of which for FY2011-2012 was $1,561,076; and
  • RMS data are used to calculate the revenues earned, the value of which for FY2011-2012 was $12,629,615 (with Employee Benefit Plan (EBP) included) and $10,974,781 (no EBP).

3. Risks

Based on risks identified in the annual risk-based audit plan and during the preliminary survey phase of the audit, IAD, with input from the advisory committee, developed audit criteria that covered these key risks:

  • Time reporting processes: The risk that organizational processes — including training, supervision and data entry procedures — are insufficient for the reliable recording and coding of time data and the production of timely reports that are accommodated to the needs of the organization. In the context of this audit, some examples of process risks include unclear codes, the capacity of supervisors to validate entries and billing delays caused by process inefficiencies.
  • Time reporting technology: The risk that the time reporting systems have not been adapted to, and do not adequately support, the PSC’s time reporting processes, resulting in data security issues and workarounds that cause inefficiencies. ITSD management has referred to EZ-Time as an “unapproved system,” meaning that it has not gone through a software testing and approval cycle, and is expected to be accepted “as is” (aside from changes to the coding tags) by any unit that chooses to deploy it. As an unapproved system, EZ-Time carries many technology risks.EZ-Time was developed by just one individual to meet the needs of a small group, and was subsequently adopted without a further review of its suitability to purpose by other units.
  • Cost-effectiveness: The risk that the trade-off between cost and effectiveness is inappropriately made. Cost-effectiveness can also be compromised if a system that is developed for one situation and purpose is adopted, as is, into a different situation to fulfill a different purpose.
  • Linkages with other databases and applications: The risk that failure to link reliably with other systems that contain relevant data will result in the need for manual intervention, with accompanying inefficiency and risk of error. Of note, EZ-Time data are stored in three separate databases, which means that some management reports can take considerable administrative time to assemble.
  • Governance: The risk that system and process development and maintenance will not be carried out in a fashion that is accountable to stakeholders, and thus fails to meet to the needs of the organization.

4. Objective

To assess whether PSC time reporting systems and processes are adequate and effective for producing reliable information in a cost-effective manner.

5. Scope

The audit includes time accounting and reporting processes across all branches. However, as IAD reviewed time reporting in ADSB during a recent audit, this branch will only be peripherally included. In addition, client invoicing processes are excluded even though they use RMS output. These processes may be the topic of a subsequent audit. The audit covered the time reporting systems in place between March and November 2012.

6. Statement of conformance

This internal audit engagement conforms with the Internal Auditing Standards of the Government of Canada as supported by the results of the quality assurance and improvement program.

Audit work was planned to provide a reasonable level of assurance for the conclusions drawn.

7. Methodology

The audit was accomplished in accordance with the PSC standard audit process, which includes three principal phases: planning, which was initiated in March 2012; detailed examination, which was completed on October 19, 2012; and reporting, which will be completed when a final report is approved.

Because this was a horizontal audit that involved several branches, IAD formed an Advisory Committee with representatives from all branches to help plan the work and validate results.

During the planning phase, a preliminary risk assessment was used to identify lines of enquiry. An audit program was developed with control objectives and audit criteria that would enable the audit team to conclude on the overall objective of the audit examination phase, as stated in section 4 above. The draft criteria were provided to the Advisory Committee for review and comments. The audit program included detailed criteria for each control objective, as well as a description of the method to be used in gathering evidence to support a conclusion on each criterion.

During the examination phase, audit methodologies included interviews with management and staff, as well as review and analysis of key processes and supporting documentation. At the end of this phase, preliminary observations were developed and presented to the Advisory Committee. These observations were then shared with all Vice-Presidents (VPs), plus affected Directors-General (DGs) in CMB and SASB.

The Advisory Committee validated the observations.

Project deliverables were reviewed and signed off by the Director of Internal Audit.

8. Observations and recommendations

The following section presents the key observations of this audit, as well as the impact and recommendations.

Control objective 1

Time reporting entries are entered, processed and reported in a reliable and usable manner.

This control objective focused on the processes and technology that support data entry by employees and compilation of the data into reports that serve the business needs of the PSC.

The internal auditors expected that employees would understand how to use the system, as well as the importance of entering time data diligently. We also expected to find that time data entry would be embedded in processes that captured data at the appropriate level of detail, accommodated supervisory control, met the expectations of senior management for timely and reliable reports and allowed for review and correction of process inefficiencies.

Revenue Management System observations

RMS was found to use up-to-date technology. It is a Web-based application, built on an Oracle database with JavaScript coding. Since it is Web-based, it can be accessed by employees in the PSC’s regions. As a limitation, RMS has not been tested with high system loads and may be approaching the maximum number of users it can serve with adequate response time.

In the Client Services Directorate (CSD), RMS serves as a tool both for billing and for project management of client service requests, which means that it is highly integrated into the day-to-day workflow. Employees record their time usually daily, either directly in the system or on paper, and add an explanatory comment when they code their time into the system. Because these codes are used in client service request management, supervisors in CSD verify time data against these codes weekly, sometimes daily, and use the comments to understand and challenge discrepancies. Many codes come with a time standard that can be used to check the reasonableness of the time entered. To ensure accuracy of coding, problem codes are discussed during team meetings; one office regularly prints laminated cards to assist employees in entering codes that have recently posed a problem. Managers of operations monitor the work of the supervisors with independent spot checks and challenges to their recommendations; at the directorate level, an operations management committee reviews common process problems.

Despite being well integrated into the workflow, RMS is not aligned with the three-tiered time-reporting process that is common at the PSC, with employees being supervised by a team leader who reviews time entries before they are approved by a manager. Unless system-wide managerial access rights are granted to the team leader (which creates a data security problem), a team leader cannot view the explanatory comments except by using Report 4, which, at least according to one supervisor, is not well suited to the purpose. In addition, for an RMS supervisor to indicate to a manager that a review is complete, creative workarounds are needed, such as a review completion sheet that is stored in the Records, Documents and Information Management System.

In the Personnel Psychology Centre (PPC), the RMS process is not integrated into the day-to-day workflow. Time data are submitted monthly; as a consequence, data verification presents more of a challenge. Separate processes are used for professional services and for assessments. Specifically, time data for professional services are recorded in Excel time sheets, which are approved by the Director of Assessment and Counselling Services and forwarded to an assistant at the Director General’s office for entry into RMS. The assistant verifies the reasonableness of the time data and often suggests changes. Conversely, time data for assessments, after approval by the Director, are sent each month directly to the Integrated Services Directorate, which inputs the data without any additional verification, leaving open the risk that some inaccuracies may be getting through.

A process alignment problem commonly mentioned among CSD operations managers and supervisors using RMS was the delay caused by the monthly cycle for submissions of time data by PPC. PPC service providers do not work directly in RMS; their time is entered by data processing persons at Headquarters in Ottawa who, at the beginning of every month, receive both time sheets and e-mails noting any projects that have been completed during the month. Data processing adds a further delay before an invoice is created in RMS. These delays allow PPC to meet its administrative needs, but when PPC is working on a project with the CSD regional offices, the time from PPC is often the last to be submitted. Consequently, client billing can be delayed for a month or more.

It was found that the RMS backup processes are adequate. RMS depends on a regular nightly system backup. As well, data are stored in a standard database format, so there is little risk of obsolescence of the software used to open and read the data files.

EZ-Time observations

EZ-Time was developed by just one individual for use by a small group, and was subsequently adopted in an uncontrolled fashion by other units.

The EZ-Time Access database software with Visual Basic coding represents previous-generation technology. The EZ-Time user interface is rudimentary, with quirks like drop-down menus that pop up instead of dropping down. Because it is not Web-based, employees working in regional offices cannot input their time directly but must forward their time by e-mail for re-entry. Time sheets are under the control of data administrators, who print them out monthly for review by supervisors.

Based on interview evidence and a comparison with PeopleSoft records, it appeared that the majority of EZ-time users were making an effort to enter their time diligently; however, a few employees clearly were not. For example, in a random sample of the PeopleSoft records of 22 EZ-Time users, it was found that 4 had EZ-Time data entry errors of more than 30 hours during the first quarter of 2012. The total error for these 4 users was over 200 hours. Moreover, even otherwise diligent users commonly failed to record leave or overtime.

It also seems that some employees who are totaling their monthly hours correctly are nonetheless not coding them diligently. One employee, who estimated a coding error as high as 50%, entered time in one long session at the end of the month. Another employee simply coded the monthly time at the beginning of the month, with a view to the tasks that might be performed.

Like RMS, EZ-Time is not adapted to the three-tiered process that is common at the PSC, with employees being supervised by a team leader who reviews time entries before they are approved by a manager. From within EZ-Time, a team leader cannot indicate that a review is complete; indeed, there is no supervisory level of access. This means that supervisors cannot correct erroneous data entries. In addition, there is a data security problem in that, although employees are told that there is a “soft lock” on their data after they have been submitted, in actuality, they can go into the system and change them at any time.

With EZ-Time, supervisors tend to wait until the end of the month before reviewing time data entries, which makes it difficult to do an accurate review. In fact, there is as much emphasis on getting the monthly hours total correct as on coding the time correctly. EZ-Time supervisory control is further weakened by the relegation of some oversight to the EZ-Time data administrators, who send reminder messages when delays or discrepancies threaten to delay reporting to senior management, who need the information for accurate planning and budgeting, or to the Finance and Administration Directorate (FAD), who needs it for the calculation of capitalizable assets in the financial statements.

As with RMS, there is no special backup procedure for EZ-Time. EZ-Time depends on a regular nightly system backup; the Access database uses a standard database format, so there is little risk of obsolescence of the software used to open and read the data files.

Summary of findings on process and technology

The auditors found that overall reliability of time processes varied. Technology was a underlying factor that set limits on the usability of the data and on the process used to gather and verify them. Employees appeared to have a good understanding of how to appropriately input data into both RMS and EZ-Time. However, in the case of EZ-Time, there was evident lack of diligence on the part of some employees, with significant impact on coding and timing accuracy. In the case of EZ-Time, and also PPC’s use of RMS, the monthly data entry and verification cycle reduces confidence in the reliability of the data collected. Timeliness is variably controlled. CSD users of RMS exercise strong practices, while PPC users of RMS and EZ-Time users do not exercise the rigorous practices necessary to ensuring timeliness. In contrast to the RMS process, which is subject to a reasonable level of review and correction, the auditors found no evidence that the EZ-Time process was subject to similar continuing improvement review. Backup processes for both EZ-Time and RMS are adequate.

Control objective 2

Ongoing stakeholder needs for the time reporting systems are considered by the organization.

This control objective focused on the processes that structure and maintain time reporting technology at the PSC so that the needs of the organization continue to be met.

We expected that, to make time reporting as cost-effective as possible, data manipulation would be automated so as to reduce effort and the chance of error, that systems would communicate electronically with one another to minimize data re-work and re-entry and that time reporting systems would be subject to an organizational governance that managed system change and obsolescence.

Observations on cost-effectiveness

As about 100 employees use each system, we note that both RMS and EZ-Time themselves take time, estimated at:

  • As much as 0.05 FTEs per employee for data entry in RMS, and as little as 0.01 FTEs for EZ-Time;
  • 30 hours/month per system for supervision and quality assurance (rough estimate);
  • 19 person-days per month for EZ-Time maintenance and management reporting, compared to 15 for RMS; and
  • Unknown amounts of time for ongoing system development.

Note: There is no process in place to measure or monitor the cost-effectiveness of time accounting and reporting activities. Hence, the auditors’ cost-effectiveness estimates were based largely on anecdotal information gathered through interviews.

Data entry, supervision and quality assurance efforts are directly affected by the number of task codes that are being managed and verified. EZ-Time has slightly more codes than RMS (approximately 130 for EZ-Time when totalled across all three databases, compared to 80 for RMS); however, ITSD has reduced the cost of data entry by rationalizing and reducing the EZ-Time task codes used in its database. In addition, data entry into RMS is more time-consuming because the process requires that each code be accompanied by an explanatory comment. The EZ-Time burden on supervisors is also lessened by relegating to administrative staff the responsibility of sending reminder messages when data entry deadlines are not met.

The auditors also noted some controversy about whether the number of RMS task codes was appropriate. The controversy arises because the task codes being used are derived from planning and estimation, which requires considerable detail, while the purpose of time data entry is client job management and invoicing, which may require less detail.

Both RMS and EZ-Time require the intervention of system administrators not only for system maintenance, but also to generate routine management reports. RMS reports are used by senior management for such purposes as trend analysis, process oversight and FTE efficiency monitoring. In addition to capitalization costing, EZ-Time reports are used for process efficiency monitoring and accountability costing (e.g. the ITSD dashboards). Manually generating these reports is time-consuming and inherently introduces the risk of errors. This suggests the need for an independent verification and error correction process. However, the volume of data and number of manipulations involved would make such a process onerous.

In addition to other maintenance activities, effort must regularly be made to reset EZ-Time’s Access database at the end of every fiscal year, as Access does not handle high volumes of data well.

Observations on data linkages

We expected to find that systems and applications would communicate electronically with one another to minimize data re-work and re-entry.

The two key systems have differing database structures. RMS data are stored in a single, central database, which means that there is one source of structured data for management reporting and billing purposes. In contrast, EZ-Time does not have a central database. Rather, data are stored in three separate databases — one for each of ITSD, SASB, and ADSB — with disparate coding structures. This creates a problem for oversight on projects that span more than one database, such as the PSRS. Several projects relating to PSRS involve both SASB and ITSD employees who report their time on separate databases; hence, the data must be integrated by hand in order to calculate the cost of the projects they are working on.

The auditors found that RMS, and the tools and systems it must interact with, do not always communicate effectively. For example, data from the Estimator tool, used to create estimates for client service requests, must be re-entered manually into RMS. RMS does have the capacity to interface with Second Language Evaluation (SLE) databases and the FreeBalance accounting system; however, the SLE data are not error-free, and manual intervention is required to correct it.

EZ-Time is completely standalone, which means that EZ-Time users must re-enter leave and overtime data. This is a common source of error that could be avoided, since the data are already entered with high reliability in PeopleSoft. In addition, to produce management reports, time data must be matched up with salary data either by means of an error-prone Visual Basic tool known as the “Integrator” or directly by hand from time data that the Integrator does not link to. The shortcomings of the Integrator are of particular concern because it is used to calculate capitalization of assets under development, the value of which for FY2011-2012 was $1,561,076, broken down as follows:

  • Calculated using the Integrator: $1,327,192.64
    • PSRS: $1,189219.22
    • EDW: $137,973.43; and
  • Calculated by hand directly from EZ-Time data: $233,883.

Observations on governance

The auditors expected to find that system change requests are addressed appropriately and that a governance process is established for time-reporting systems.

Governance is the exercise of ensuring that objectives are met and operations are carried out in an accountable manner. Hence, governance applies not only to how new systems and process changes are introduced, but also to their ongoing operation. In the IT system development context, governance includes an initial feasibility report and system and usability testing before system implementation.

The auditors found that RMS was subject to a reasonable governance structure, even though neither a feasibility study nor systematic usability testing were ever performed. RMS process development, software updates and system maintenance are managed coherently. Software is tested before release and is amended after release based on user feedback; change management is under the control of the Information Management / Information Technology (IM/IT) Committee.

EZ-Time is considered an unapproved system and, as such, does not fall under the PSC’s IT governance structure. It was developed very quickly by a single individual without allowing for testing or future maintenance. EZ-Time and related software have not been subject to any meaningful testing cycle. In fact, the ITSD staff actually detected a significant bug in the related Integrator code while explaining to the auditor how it worked. Furthermore, the operation and use of EZ-Time is not subject to any systematic error-tracking and feedback. As a consequence, problems are worked around rather than directly addressed. For example, rather than requesting a fix to the Integrator (which is used in calculating capitalization costs), an administrator was simply adjusting the data in an Excel spreadsheet.

Summary of findings on consideration of stakeholder needs

Neither system is optimally cost-effective, and both show the results of trade-offs. Opportunities exist to reduce costs and increase effectiveness by simplifying input requirements, enhancing integration between systems and automating production of output reports. Such solutions could lower both costs and errors. For example, whereas RMS time data are stored in a single database in such a manner that it facilitates automation and integration, some of the data stored in EZ-Time must be extracted manually from more than one database and then manually re-tagged to make them relevant to management work. In addition, the RMS governance process is formal and contributes to the satisfaction of stakeholder needs. By contrast, the EZ-Time governance process, insofar as it exists, does not function effectively.

Recommendations and action plans

To assist management in developing an integrated action plan to respond to the recommendations in this report, these recommendations can be summarized with respect to RMS and EZ-time as follows:

Recommendations on RMS

Recommendation 1

The VP of SASB should conduct a needs assessment of time reporting together with a cost-benefit analysis of areas where continued time reporting is needed, with a view to improving the RMS data entry and approval process by:

  1. More tightly aligning the time reporting cycles of CSD and PPC; and
  2. Aligning time verification and approval access in RMS with SASB's review practices.
Management response
  1. The VPs of SASB and CMB will conduct a needs assessment of time reporting and, as required, analyze the cost-benefit of enhancing RMS time reporting functionality for billing purposes.
    • Responsibility: VP-SASB and VP-CMB
    • Completion date: April, 2014
  2. Subject to Executive Management Committee approval of the IM/IT prioritization, mandatory components of the joint CMB and SASB IT request for RMS changes will be completed.
    • Responsibility: VP-SASB and VP-CMB
    • Completion date: April, 2014
Recommendation 2

The VP of SASB should also conduct a cost-benefit analysis of improvements to the functionality of RMS-based time reporting, giving consideration to:

  1. Identifying those key interfaces with other processes/systems where process improvement and automation of time data would reduce manual intervention or risk of data errors; and
  2. Rationalizing coding to match organizational needs.
Management response
  1. The VPs of SASB and CMB will also determine the cost-benefit of adapting RMS in order to gain further efficiencies in the data entry process and to minimize manual entries.
    • Responsibility: VP-SASB and VP-CMB
    • Completion date: April, 2014
  2. Subject to EMC approval of the IM/IT prioritization, further automation of time
    data from other systems into RMS will be implemented as indicated in the
    non-mandatory components of the RMS IT service request.
    • Responsibility: VP-SASB and VP-CMB
    • Completion date: April, 2014
  3. An RMS account management mailbox has been established to centralize account approvals. All account changes, approvals and deletions will require approval by the Business Management and Support Directorate of SASB before any change can occur.
    • Responsibility: Director, Business Management and Support (SASB)
    • Completion date: January 2013 and ongoing
  4. SASB’s Business Management and Support Directorate will verify time reporting for client billing and raise timing issues with the manager for follow up. Service request issues and/or delays are to be discussed at weekly staff meetings.
    • Responsibility: Director, Business Management and Support (SASB)
    • Completion date: ongoing
  5. While further automation of RMS is being considered, the VP of SASB will review data entry procedures and implement standardized manual data entry processes across its users.
    • Responsibility: VP-SASB
    • Completion date: December, 2013
  6. SLE scheduler system training will be given to data entry users to ensure proper tool utilization that would avoid/reduce file transfer errors into RMS.
    • Responsibility: Director, Business Management and Support (SASB)
    • Completion date: June, 2013

Recommendations on EZ-Time

Recommendation 1

The VPs of CMB, SASB and ADSB should each determine their time-based reporting needs currently being addressed by EZ-Time, and, if time reporting continues to be needed, they should:

  1. Identify acceptable alternatives to EZ-Time; and
  2. Implement migration to the new process.
Management response
  1. The VPs of CMB, SASB and ADSB will articulate their specific and sustained needs for time reporting outside RMS.
    • Responsibility: VP-SASB, VP-CMB and VP-ADSB
    • Completion dates:
      • CMB (ITSD) – Done
      • SASB – December, 2013
      • ADSB – April, 2014
  2. As required following this needs assessment, the Director General (DG) responsible for PSRS will develop a cost-benefit analysis of keeping EZ-Time or using alternative solutions to replace EZ-Time in collaboration with the DG of ITSD, and bring forward recommendations to the VP.
    • Responsibility: VP-SASB and DG-ITSD
    • Completion date: April, 2014
  3. EZ-Time will be phased out for ITSD and replaced with Clarity Time Management.
    • Responsibility: DG-ITSD
    • Completion date: December, 2013
  4. Based on the implementation of Clarity Time Management in ITSD, this new time reporting tool may be considered in the cost-benefit analysis of alternatives to EZ-time for reporting on other PSC activities and IM/IT.
    • Responsibility: DG-ITSD and DG responsible for PSRS
    • Completion date: April, 2014.
  5. For the Data Services and Analysis Directorate, EZ-time will be phased out at the end of 2012-2013, as other project management tools, such as Microsoft Project and Remedy, are already being used and have been identified and approved as acceptable alternatives to EZ-time.
    • Responsibility: VP-ADSB
    • Completion date: April, 2013
  6. ADSB has recently purchased TeamMate working paper software that includes a Time Reporting component for the Audit Directorate. Until full implementation of the Teammate tool, the Audit Directorate will keep using EZ-time, as it is indicative of time spent per audit phases as well as on administrative activities. A two-step response to this recommendation is proposed:

    * Firstly, TeamMate is currently in the testing phase and will be piloted with audits in June 2013. An evaluation of the usefulness of this tool to report on audit time will be undertaken.
    • Responsibility: VP-ADSB
    • Completion date: June, 2013
    * Secondly, if Teammate is found to be an appropriate tool for time reporting, a transition period that includes the development of software training sessions for staff followed by full roll-out will be required. We expect full implementation of TeamMate by June 2014.
    • Responsibility: VP-ADSB
    • Completion date: June, 2014
Recommendation 2

Until the use of EZ-Time is discontinued, the DGs responsible for ITSD and PSRS should develop and implement strategies for ensuring accuracy in EZ-Time data entry and supervisory review, with no more rigor than necessary to the intended use of the information.

Management response
  1. To address quality issues until alternatives to EZ-time are fully implemented, the DG responsible for PSRS and the DG of ITSD will implement a standardized and improved procedure for data entry and quality control of activities within their respective teams.
    • Responsibility: DG responsible for PSRS and DG-ITSD
    • Completion date: December, 2013
  2. The DG responsible for PSRS and the DG of ITSD will also establish the required verification process for the time reported against the determined activities within their respective teams.
    • Responsibility: DG responsible for PSRS and DG-ITSD
    • Completion date: December, 2013.

We believe that management’s action plan will remedy the shortcomings noted.

9. Overall conclusions

The auditors found that PSC time reporting systems and processes are only partially adequate and effective for producing reliable information in a cost-effective manner. The overall reliability of time reporting varied with the system and the work unit. Technology was an underlying factor that set limits on the usability of the data and on the process used to gather and verify them.

On one hand, RMS is performing its primary function of providing accurate and reliable data for billing quite well. This is largely due to the fact that the RMS process and underlying technology encourage diligence both in entering data on the part of employees and in providing feedback on the part of supervisors. Nonetheless, RMS has a few process and technology alignment issues that make both co-ordination between units and the data entry approval process more cumbersome than they need be. On the other hand, EZ-Time has both process and technology deficiencies that are negatively impacting its capacity to meet the objectives of time reporting at the PSC. The many shortcomings of EZ-Time are indicative of a system that was developed to meet the needs of a small group, and that was subsequently adopted in an uncontrolled fashion by other units.

Stakeholder needs are partially being met. Both systems have been affected by governance deficiencies with consequent shortcomings such as diminished cost-effectiveness and limitations in their capacity to link with other systems. An RMS governance framework is now in place to ensure that future changes are controlled and based on user feedback; however, no cost-benefit analysis has ever been undertaken to establish the input and operational requirements of RMS and to understand how to optimize the system to accommodate the data collection process. ITSD management has referred to EZ-Time as an “unapproved system,” meaning that it has not gone through a software testing and approval cycle, and is expected to be accepted “as is” (aside from changes to the coding tags) by any unit that chooses to deploy it. As an unapproved system, EZ-Time system development has been very low-cost; however, its many shortcomings limit its effectiveness.

Management has provided robust action plans, which we believe will remedy the shortcomings noted.

Appendix A – List of audit criteria

Control objectives Audit criteria
Control objective 1

Time reporting entries are entered, processed and reported in a reliable and usable manner.
Time reporting processes and time reporting technology

1.1 Employees know how to appropriately input time reporting entries.
1.2 Time reporting data are input and processed efficiently.
1.3 Time reporting input is timely.
1.4 Time reporting data are sufficiently accurate and complete.
1.5 Time reporting data are independently verified for completeness, accuracy, timeliness, appropriateness and consistency.
1.6 Time reporting outputs are independently verified for completeness, accuracy, timeliness, appropriateness and consistency.
1.7 Procedures for backup and restoration of time reporting data and information are in place and are followed.
Control objective 2

Ongoing stakeholder needs for the time reporting systems are considered by the organization.
Cost-effectiveness, linkages with other systems and applications, and governance

2.1 Time reporting systems are cost-effective.
2.2 Time reporting systems are electronically integrated with related PSC systems.
2.3 Time reporting outputs capture the information relevant to management work.
2.4 Ongoing requests for system changes are addressed appropriately.
2.5 A governance structure is established for time reporting systems.

Appendix B – Glossary

  • ADSB Audit and Data Services Branch
  • CMB Corporate Management Branch
  • CSD Client Services Directorate (part of SASB)
  • FAD Finance and Administration Directorate (part of CMB)
  • FTE Full-Time Employee
  • IAD Internal Audit Directorate
  • IM/ITC Information Management/Information Technology Committee
  • IT Information Technology
  • ITSD Information Technology Services Directorate (part of CMB)
  • OPI Office of Primary Interest
  • PPC Personnel Psychology Centre (part of SASB)
  • PSC Public Service Commission
  • PSRS Public Service Resourcing System
  • RMS Revenue Management System
  • SASB Staffing and Assessment Services Branch
  • SLE Second Language Evaluation

Page details

Date modified: