Audit of iCase File Compliance - May 2015

Recommended for Approval to the Director of Public Prosecutions by the Departmental Audit Committee on May 20, 2015

Approved by the Director of Public Prosecutions on May 27, 2015

Table of Contents

1.0 Executive Summary

1.1 Objectives and Scope

The overall objective of this audit was to provide assurance on the compliance of opening and closing of iCase files in terms of completeness, accuracy and timeliness.

The audit focused on the:

  1. Integrity and quality of iCase data; and,
  2. Functionality of the system and processes; specifically file opening & closing.

This audit was planned and conducted during August 2014 to February 2015.

1.2 Audit Conclusion

The Internal Audit Division (IAD) assessed the compliance of opening and closing of iCase files in terms of completeness, accuracy and timeliness against predetermined audit criteria based on Treasury Board policies and directives, Public Prosecution Service of Canada (PPSC) policies, directives, protocols, and procedures as well as general best practices. Overall, there are opportunities to improve the governance for ensuring compliance of opening, closing and updating files within the case management system.

1.3 Summary of Recommendations

This report includes the following recommendations:

  1. The Deputy Directors of Public Prosecutions (DDPP) in consultation with the Director General of Human Resources, should review the roles and responsibilities of iCase support staff.
  2. The Performance Measurement Committee (PMC) should revise its mandate to include national monitoring of the compliance with iCase protocols, directives and manuals;
  3. The DDPP, Regulatory & Economic Prosecutions and Management Branch (REPMB) should:
    • Ensure that case management protocols, directives and manuals are approved by the PMC;
    • Establish a plan to ensure that case management documentation and training meet PPSC needs; and
    • Develop a monitoring plan to ensure national compliance with file management and timekeeping.

1.4 Statement of Assurance

In my professional judgment as the PPSC’s Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report. The audit findings and conclusion are based on a comparison of the conditions, as they existed at the time of the audit, against pre-established and approved audit criteria that were agreed upon with the PPSC’s management. The findings and conclusion are applicable only to the entity examined. The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada.

I appreciate the cooperation and assistance provided to the audit team by PPSC staff in Headquarters (HQ) and in the regional offices visited.



Julie Betts
Chief Audit Executive

2.0. Introduction

2.1. Background

iCase is a web-based national application used by the PPSC to support the practice of law, and the management and delivery of prosecution services with the following functions: case management; timekeeping and operational reporting. The Corporate Service Provider (CSP) is the developer and owner of the application, and provides services to the PPSC through a Memorandum of Understanding (MOU) between the departments. PPSC along with several other government departments, including the CSP, have developed system requirements for a new case management system that would be housed on a Government of Canada server. The new system is expected in the fiscal year 2016–2017.

The Strategic Planning and Performance Management Division (SPPMD) is responsible for performance measurement and reporting based on iCase data, and also has a functional role regarding iCase support personnel in the regions by providing guidance, directives and information that are specific to PPSC. In addition, the PMC is responsible for the following activities specifically in relation to Case Management System OversightFootnote 1 as per its Terms of Reference (ToR):

The IAD conducted this Audit of iCase File Compliance in accordance with the PPSC’s 2014–2017 Risk-Based Audit Plan, which was approved by the Director of Public Prosecutions (DPP) on April 17, 2014.

2.2. Objectives, Scope and Methodology

The audit methodology included:

The scope was limited to reviewing in-house files/reports and excluded the Fine Recovery Inter-jurisdictional Set-Off interface for the National Fine Recovery Program.

The planning and conduct phases of the audit were carried out between August 2014 and February 2015.

3.0 Observations and Recommendations

3.1 Governance

The PMC provides oversight with respect to the iCase application; however improvement is required in terms of developing, updating and communicating directives and protocols, as well as monitoring compliance.

Governance is the combination of processes and structures implemented to inform, direct, manage and monitor the activities of the organization toward the achievement of its objectives. Oversight bodies are one of the structures that help ensure that management’s direction, plans and actions align with these objectives and are communicated effectively.

The audit team expected the iCase application to have appropriate governance that supports effective decision making and communication.

The PMC plays an oversight role with respect to the application and conducts the roles and responsibilities as per its ToR. However, the ToR do not make reference to monitoring activities in relation to case management to ensure national compliance with approved standards, directives and protocols. Without effective monitoring at the national level, there is a risk that regions are monitoring file and timekeeping compliance in an inconsistent and heterogeneous manner.

Interviews indicated that the PMC was not always involved in decisions related to iCase. For example, the decision to close all fine recovery files including those where fines were still outstanding was made with no involvement of the PMC. As well, an iCase Agents’ Fine Recovery Manual was developed and distributed to the regional offices without the involvement or approval of the PMC. When decisions are made or directives are distributed without the input of an oversight body, there is the risk that they are based on erroneous or incomplete information.

Interviews also indicated that while PPSC change requests are accommodated, they may take longer than expected since PPSC requirements may not be a priority for the CSP. The MOU does not specifically address service standards or escalation processes in relation to the provision of iCase application support to the PPSC. The absence of such service standards and a formalized process to resolve conflicts with the CSP leaves the PPSC in a subordinate position with respect to achieving its own objectives for the application.

A review of documentation revealed that with few exceptions (National Timekeeping Protocol (NTP), Complexity Reference Table, Default Complexity Level – Federal Statutes, iCase Guide to the Charge Laid Date and iCase Guide to Mandatory Minimum Penalties), PPSC users must rely on the CSP documentation that is sometimes outdated and/or does not meet PPSC requirements. This may result in inconsistent and/or incorrect use of the application.

Recommendations:

  1. The PMC should revise its mandate to include national monitoring of the compliance with case management protocols, directives and manuals.
  2. The DDPP, REPMB should ensure that case management protocols, directives and manuals are approved by the PMC.

3.2 Accountability

Organizational responsibilities for the timekeeping function of iCase are clear and communicated; however there is limited direction in connection with prosecution file management. The roles and responsibilities of iCase AdministratorsFootnote 2 vary from one region to another.

Accountability is the duty to report on the fulfillment of one’s responsibilities. As such, a key control of accountability is an appropriate organizational structure that clarifies authorities, responsibilities, and the duty to report.

The NTP provides a framework governing timekeeping practices in the PPSC and details the roles and responsibilities of Timekeepers, Team leaders (TL), Managers, iCase Administrators and iCase Coordinators with respect to timekeeping as follows:

Timekeepers

Timekeepers have the following responsibilities:

All timekeepers interviewed reported that they personally input their time in iCase as required, however there was variation in how this was accomplished. Some timekeepers enter their time throughout the day, some enter time at the end of the day and others recorded their time manually and entered time at the end of the week. All timekeepers also received a copy of the monthly timekeeping report “Time Summary by Lawyer and Paralegal” that they reviewed, signed and dated. In the Québec Regional Office (QRO), each timekeeper reconciles this report with PeopleSoft as requiredFootnote 3. Other regions visited did not necessarily follow this requirement.

Team Leaders

In addition to their timekeepers’ role, TLs have to ensure that timekeepers in their unit comply with the requirements of the NTP. The TLs interviewed monitor their staff’ reports, though not in a consistent manner. QRO TLs for example, review the reconciliation report, sign and date it. The TL reviews time coded to administrative functions and requests a correction if errors are spotted. The report is also discussed during the monthly bi-laterals with employees. TLs from another regional office indicated that they did not systematically review monthly timekeeping reports but would pay special attention to outliers, such as employees spending unreasonable amount of time on a specific file.

Managers

Managers are responsible for:

Interviews indicated that Chief Federal Prosecutors (CFPs) are mindful of timekeeping. Accuracy of data is thought to be important for reporting and resource allocation. The audit found that the reviews and monitoring activities are delegated to the iCase Administrators, and vary from one region to another.

iCase Administrators

iCase Administrators are responsible for:

Interviews with iCase Administrators indicated that they were responsible for producing timekeeping reports at month-end and distributing them to the various managers/team leaders. They also provide support to timekeepers and generated other iCase audit reports for managers. In addition to generating and distributing reports to timekeepers, TLs and the CFP, they perform review and monitoring of timekeeping, including leave and cost recovery billing on behalf of managers.

In addition to their iCase Administrators related duties; they perform other duties related to file management including opening, closing and updating files. As well, classifications and levels of iCase Administrators vary from one region to another.

Other than for timekeeping, the ODPP iCase Business Standards Manual 2007 (the Manual) makes minimal reference to roles and responsibilities with no reference made in other iCase documentation. The lack of direction on the roles and responsibilities with respect to prosecution file management may result in inconsistencies in how the iCase application is being used for this purpose. In addition, the lack of consistency and clarity in connection with the iCase Administrator role may compromise the level of regional support for the application.

Recommendation:

  1. The Deputy Directors of Public Prosecutions (DDPP) in consultation with the Director General of Human Resources, should review the roles and responsibilities of iCase support staff.

3.3 People

There are opportunities for improvement in providing training and up-to-date online tools to support users in fulfilling their duties.

For employees to effectively utilize any business application, they must be provided with the necessary training, tools, resources and information. The audit team expected to find that employees received the knowledge and training to meet their needs for using iCase.

Training

Two of the three iCase Administrators interviewed indicated that they received on the job training by the former iCase Administrators. The third iCase Administrator received training from the National Business Application Analyst; however it was not available in the language of her choice.

All of the iCase Administrators interviewed indicated that they provided training to users on ‘as requested’ basis and/or when there is a new employee. The content of training is not standardized and iCase Administrators develop their own list of training topics based upon users’ needs. The type of training received depended on the user and how they would be using iCase.

Interviews indicated that some users had never received iCase training. For those that had, the training was not standardized and inconsistent. Some users recall having received formal training when the application was introduced ten years ago while most received some type of on-the-job training when they started in their position. Users stated that they were not aware of all the functionality of the application, some felt that additional training might be beneficial, while most indicated that the training they had received was adequate for their needs. A few of the users who had recently received the training within the Ontario Regional Office (ORO) felt that it had been very beneficial in pointing out shortcuts. As well, classroom training is available in the National Capital Regional Office (NCRO) and includes an Introduction to iCase and Timekeeping and Introduction to iCase, Timekeeping and Administration Functions. In addition, a self-paced training – iCase Training Guides and Videos – is available online. In general, there were no specific training requirements identified by users.

The National Business Application Analyst felt that training is an issue and that a refresher session for file openers is needed as people don’t fully understand why they have to enter information, such as outcomes, and its impacts on cost recovery, fine recovery and corporate reporting. This type of refresher training was recently provided by the National Business Application Analyst to ORO file openers at the request of the regional iCase Administrator.

Tools

iCase users indicated that they had reliable support for any problems they encountered through their iCase Administrator and/or the Business Support Team. In addition, there was a high degree of awareness of email updates regarding iCase. Interviews also revealed that users made limited use of the online tools and reference material as iCase online documentation was outdated, not PPSC specific or not easy to locate. There has been some discussion regarding the establishment of an “iCase Management Corner” that would be the single point of reference for iCase; however it is still under development.

The audit found that the iCase online reference material can be found on iNet under the Information Management and Information Technology site and the Legal Resources page. As mentioned above, there is limited material that is PPSC specific and references are made to some CSP documents which were not developed for the PPSC and may be outdated. This may result in users referring to documents which do not meet PPSC requirements or, as observed, not making use of the reference material.

Recommendation:

  1. The DDPP, REPMB should establish a plan to ensure case management documentation and training meet PPSC needs.

3.4 Stewardship

Limited monitoring and inconsistent review of data integrity have resulted in non-compliance.

Stewardship objectives are enabled through controls. Control activities prescribe how activities should be performed. Controls include a range of activities as diverse as policies and procedures, transaction management practices and controls, information security practices as well as a host of activities designed to effectively manage third parties.

The audit expected:

File Information

The Manual describes the practices and procedures to be followed when using iCase. This Manual explains what information must be entered in iCase and why it is entered. For each file, the details, related activities and all documents must be recorded in iCase from the time the file is opened until it is closed with some exceptions. A new litigation file is opened for any matter that has been assigned a Court file number or when it is anticipated that there will be an action, judicial review, application or trial[…] All iCase files must first be opened in RIMS on receipt of initial documentation or requestFootnote 4.

The Manual and the iCase Agents’ Fine Recovery Manual list mandatory fields to be populated within iCase. All information contained in iCase and hard copy files should agree at all times, including the Prefix, File Number, File Security, File Type, File Status, File English Subject, File French Subject, File Opened Date, and File Closed Date.Footnote 5

Interviewees indicated that a file opening form is filled out and used as the underlying document to open the file in GCDOCS. Once the file is created in GCDOCS, the information is instantly transferred to iCase and the file is opened in iCase. Interviewees further indicated that prosecution files are updated throughout the time the file remains open.

The audit found that there are inconsistencies in file opening practices, in terms of mandatory fields when opening files from one region to another and sometimes within the same region. SPPMD started reviewing the practises nationwide and compiled a list of fields by region however a national standardized list has yet to be developed.

Files [...] will be closed only when the last judicial proceeding is complete and after the appeal period has elapsed […] The file must be closed within 30 calendar days after the receipt of the final outcome of the matter or after any applicable appeal periods have expired”Footnote 6.

Interviewees stated that for the file to be closed, a closing form must be completed and signed, and the physical file sent to the record room for closure within GCDOCS. When a file is closed, it is reviewed by a clerk to ensure that the file information contained within iCase is accurate and up-to-date.

The audit team conducted a detailed review of a sample of 67 prosecution files and 44 fine recovery files selected from three regional offices to assess whether mandatory information is entered in both iCase and the hard copy file. Files were 90% or more compliant for 15 prosecution file data fields and 5 data fields for fine recovery files, however they were less than 90% compliant for the remaining 9 data fields for prosecution files and 5 data fields for fine recovery files (see Appendix A for details).

In addition to the detailed file review, an analysis of national iCase data was also conducted and the following exceptions were observed:

These individual results are not significant however the aggregate demonstrates a non-compliance at the national level and there is a risk that the inaccurate and incomplete data may impact decision making and reporting.

Timekeeping

Counsel and paralegals must accurately record the total time spent each day on PPSC activities. The daily total should not be an estimate, nor should the total day be rounded up or down, as this practice can significantly distort the data” and To help ensure data quality, time should be recorded on a daily basis (or as soon as possible when iCase access is available to the timekeeper). In all circumstances, time entries must be completed by the end of each week”.Footnote 7

Timekeepers stated that at the end of each month (or a week before the end of the period) the iCase Administrator sends them a reminder to enter their time in iCase and at the beginning of the following month (or the following period) a report is generated and sent to the timekeeper for review and certification.

A sample of 210 timekeeping reports was reviewed (35 timekeepers from three regional offices) for compliance with the NTP for the period of April 1st to September 30th 2014. The audit found a 70% compliance with NTP requirements including 14 missing timekeeping reports. (See Appendix A for details.)

In addition to the detailed timekeeping report review, the audit team conducted an iCase national analysis using data from April 1st to September 30th, 2014 and found that:

The PPSC General Timekeeping – File Standardization indicates how to use a “General” file to record time spent in Committees, Internal Management/Administrations and Training. The audit reviewed time coded to “General” and “Litigation” files for compatibility of activity and tasks and found, as shown in Appendix A, that 8,200 hours of litigation were coded to “General” (1.7% of 492,000 representing the total litigation stages hours) and 10,920 corporate function hours were coded to “Litigation” (11.5% of 94,500 total corporate function hours).

In addition, 68 timekeepers out of 276 (25%) who used “Paid Time Off Activity” between April 1 2014 and December 31 2014 used it for administrative duties at least once. These administrative duties include the review of emails, team meeting, etc. The audit team was advised that this incompatibility issue would be solved with the iCase release expected in April 2015.

Corrections and Error Reports

The iCase Super-Administrator indicated that there are no procedures for timely follow-ups and corrections of errors within iCase. Currently, when errors are detected, the regional iCase Administrator where the errors originated is contacted and a review is conducted to address the issue. This may result in errors not being addressed in a timely manner.

As well, the iCase National Application Business Analyst indicated that the system was not designed to generate error messages and communicate them when information is missing; however there is a database from which they can retrieve this information.

The review of the application output is conducted by SPPMD which generates 13 cyclical reports for the purposes of data quality control. As well reviews were conducted for data completeness, accuracy and reasonableness when ad-hoc requests for data are made. The iCase Administrators generate monthly audit reports to CFPs such as timekeeping compliance reports, file opening and closing compliance reports. The PPSC iCase Super-Administrator generates an annual national timekeeping compliance report for the DPP and the DDPPs. A report on active files with no timekeeping activity is also generated three times a year and distributed to CFPs, with a request to review their respective regional files and to close files where appropriate. These reports are used to monitor timekeepers working fewer than normal working hours, as well as noting any missing information.

Data Integrity between Applications

In order to support the cost recovery process, details of iCase timekeeping and client file information –for in-house timekeepers – is transferred to the IFMS. All necessary information is shared on a nightly basis and ensures that the most recent time-slips and file details are provided.

The PPSC Cost Recovery Process Improvement (CRPI) Initiative representative from Finance and Acquisition Directorate reconciles the IFMS and iCase information on a monthly basis to ensure that the amount billed to client departments within iCase corresponds to the IFMS data and solves any issues related to data rejected by IFMS.

The representative indicated that at the beginning of the initiative implementation there were many transactions rejected but these exceptions have improved over time. The audit noted that the number of transactions rejected was one per month during the months of August and September 2014 and two transactions in October 2014, which are minor in terms of the overall number of transactions.

The CRPI representative indicated that they receive monthly error reports (iCase data rejected by the IFMS) from SPPMD but they do not have access to error reports generated by IFMS. The CSP is responsible for the cost recovery billing as per the MOU and produces the billing report, advising if there are any follow-ups/corrections required.

The audit reviewed cost recovery information exchanged between iCase and IFMS for 26 timekeepers from three regional offices during April 1st to September 30th 2014 and found that the number of billable hours entered in iCase was the same number of hours billed to client departments in IFMS.

Recommendation:

  1. The DDPP, REPMB should develop a monitoring plan to ensure national compliance with file management and timekeeping protocols.

4.0 Conclusion

The IAD assessed the compliance of opening and closing of iCase files in terms of completeness, accuracy and timeliness against predetermined audit criteria based on Treasury Board policies and directives, PPSC policies, directives, protocols, and procedures as well as general best practices. Overall, there are opportunities to improve the governance for ensuring compliance of opening, closing and updating of files within the case management system.

5.0 Management Action Plan

Recommendation Management Response and Action Plan Office of Primary Interest Target Date
1. The PMC should revise its mandate to include national monitoring of the compliance with case management protocols, directives and manuals.
(Risk: Medium)

Agreed. The PMC will revise its mandate accordingly. The PMC’s role will be limited to determining what monitoring should take place on a national basis.

PMC Chair

Q3, 2015-16

2. The DDPP, REPMB should ensure that case management protocols, directives and manuals are approved by the PMC.
(Risk: Medium)

Agreed. Management protocols, directives and manuals will be sent to the PMC for approval as they are prepared.

DDPP, REPMB

Ongoing. No target date would apply.

3. The DDPPs in consultation with the Director General of Human Resources should review the roles and responsibilities of iCase support.
(Risk: Medium)

Roles and responsibilities for case management support will be undertaken once a decision has been made about the next case management system.

DDPPs
DG-HR

Q4, 2016-17

4. The DDPP, REPMB should establish a plan to ensure that case management documentation and training meet PPSC needs.
(Risk: Medium)

Given the imminent replacement of iCase as a case management system, limited investments will be made in iCase. A plan to establish proper support documentation and training will be prepared in concert with the roll-out plan for the new case management system.

As the current training and documentation is prepared by Justice, there is little sense in investing effort in revising it at the current time.

In respect of training, a note will be prepared and sent to all regions, reminding them to provide basic, on-site training to new iCase users.

DDPP, REPMB

Not available as this will depend on timing for roll-out of new case management system which is still in the very early planning stages.

Note to regions will be prepared in Q2, 2015-16.

5. The DDPP, REPMB should develop a monitoring plan to ensure national compliance with file management and timekeeping protocols.
(Risk: Medium)

The plan will be developed, based on recommendations from the PMC (see recommendation and response 1).

DDPP, REPMB

Q2, 2016-17

Appendix A – Detailed examination

Prosecution Files

iCase File data yes no % yes
Complexity 67 0 100%
Initiative 50 9 75%
AFT Name 64 3 96%
AFT DOB 51 14 76%
AFT Address 40 14 60%
AFT gender, Fingerprints # 61 5 91%
Offence Date 65 2 97%
Offence Location 60 5 90%
Offence Date before Charge Date 57 3 85%
Offence Date before File Open Date 63 3 94%
File Open Date before Closed date 66 1 99%
Charge Description 63 4 94%
Count Number 63 4 94%
Charge Laid Date 56 5 84%
Statute 62 4 93%
Section 61 4 91%
Specifier 52 11 78%
Service Recipient 37 8 55%
Investigative Agency 64 3 96%
Charge Laid Date before Outcome Date 55 6 82%
Outcome date 64 3 96%
Outcome category (Conviction, Guilty Plea...) 64 3 96%
Participants on File 59 4 88%
Participants Role on file 60 4 90%

Discrepancies between iCase and hard copy

Fine Recovery Files

iCase File data yes no % yes
Charge Outcome 44 0 100%
Outcome Date 44 0 100%
Fine Amount 44 0 100%
Date Fine Ordered 44 0 100%
Amount Type (Fine, Surcharge) 44 0 100%
Lead (Fine Recovery Division) 23 21 52%
Due Date on File 18 26 41%
Court File Number 39 5 89%
Details on the AFT 30 14 68%
Outstanding amount 29 15 66%

Discrepancies between FR iCase and hard copy data:

Timekeeping Reports

Files data yes % yes
Hours worked greater or equal to normal hours 126 60%
Corporate functions in litigation type file 49 23%
General greater than 30% of hours worked 49 23%
Litigation+ Advisory greater than 95% of hours worked 27 13%
Litigation stages used in the general file 32 15%
Timekeeping Report Certified by Timekeeper 162 77%
Timekeeping Report Certified within the following month 92 44%
Timekeeping Report Reviewed by a supervisor 56 27%
Leave in iCase different than Leave in PeopleSoft 64 30%
Timekeeping report missing 14 7%

Litigation hours recorded to the general code

Activity Hours
Appeals 267
Post-Charge Hearings 319
Post-Charge Preparation and Follow-up 4,966
Pre-Charge Hearings 33
Pre-Charge Investigative Advice and Assistance 1,172
Pre-Charge Preparation 332

Appendix B – Audit Criteria

Lines of Enquiry Audit Criteria
1. Governance 1.1 The organization has the appropriate structures in place to support PPSC’s objectives in relation to iCase file compliance.
2. Accountability 2.1 Organizational responsibilities are clearly documented and communicated.
3. People 3.1 The organization provides the training and tools necessary to support staff in fulfilling their duties regarding iCase file compliance.
4. Stewardship 4.1 Data input is prepared by authorized and qualified staff following established procedures that provide for adequate segregation of duties.
4.2 Data input is performed in a timely, complete and accurate manner.
4.3 Errors and irregularities are detected so that they can be reported and corrected.
4.4 Transactions are checked for integrity of content before passing data between internal applications.

Appendix C – List of Acronyms

AFT
Accused, Fugitive or Target
ASU
Agent Supervision Unit
BCRO
British Columbia Regional Office
CFP
Chief Federal Prosecutor
CRPI
Cost Recovery Process Improvement
CSP
Corporate Services Provider
DDPP
Deputy Director of Public Prosecutions
DPP
Director of Public Prosecutions
GCDOCS
Government of Canada Electronic Document Record Management Solution
HQ
Headquarters
IAD
Internal Audit Division
IFMS
Integrated Financial and Material System
MOU
Memorandum of understanding
NCRO
National Capital Regional Office
NTP
National Timekeeping Protocol
ORO
Ontario Regional Office
PMC
Performance Measurement Committee
PPSC
Public Prosecution Service of Canada
QRO
Quebec Regional Office
SPPMD
Strategic Planning and Performance Measurement Division
TL
Team Leader

Appendix D – References

PPSC National Timekeeping Protocol

File Standardization - General Timekeeping, July 2014

File Standardization - Agent Affairs Program, July 2014

Complexity Reference Table, April 2013

Default Complexity Level - Federal Statutes, April 2013.

iCase Guide to the Charge Laid Date

iCase Guide to Mandatory Minimum Penalties

Memorandum iCase/PeopleSoft Reconciliation by the DPP to all PPSC employees, April 18, 2013

Date modified: