Previous Table of Contents Next


EVALUATION AND ACCEPTANCE PHASE CONSIDERATIONS

The IT auditor should pay particular attention to the adequacy of testing. If the testing is inadequate, or deficient, the organization runs the risk of serious problems or disruptions due to an unsafe system being implemented.

If the development methodology is deficient in the testing area, the IT Audit Professional may wish to suggest one of the previous references as a test strategy. The key aspects of testing for this phase are the development of an adequate test plan, the execution of the test plan, and the analysis and reporting of test results. The auditor should be particularly concerned with the test report. This report should not only indicate what works and does not work, but also include the test group’s opinion on the adequacy of the system.

Companies can use one of two test approaches. First, they can use an independent test team (a group of people independent of the project users that are professional testers). Second, system users can create their own test conditions and determine whether the system is acceptable to them for use in production.

DETAILED AUDIT TESTING

The project sponsor relies on the test analysis and security evaluation report to determine whether to accept the system. The sponsor, however, is usually not technically oriented and does not have the necessary background to challenge the information included in the report. The independent opinion of the adequacy of that report, provided by the auditor, can be important in determining whether the application will be accepted or, if it is accepted, whether any counter-strategy must be put into place to compensate for potential weaknesses.

Testing is a critical phase of the life cycle. Programs and applications should pass system and acceptance tests before being implemented. These tests cover two different areas of concern, yet they have the same goal.

The system test provides an internal assessment of the correctness, performance, and reliability of the operational system, whereas the acceptance test determines user reaction to the product, its performance, installation procedure, documentation, and reliability.

Once these tests have been performed, the project team reviews the results to ensure that the system meets user requirements and is acceptable to the user. In addition, the auditor must ensure that testing is adequately planned and performed in compliance with approved standards and that the test results are properly evaluated and included in system documentation.

Evaluation and Acceptance Phase Audit Tests. Once again, the IT auditor should develop an audit program for this phase of the audit. The audit program should include secondary objectives for the audit, suggested tests to accomplish those objectives, and tools for conducting those tests. This program should increase the auditor’s effectiveness in reviewing systems in the evaluation and acceptance phase.

AUDIT RESULTS AND REPORTING

If the IT Audit Professional identifies a problem or weakness in part of the application, management may desire to have assistance from the audit department in the form of recommendations about changes to make or courses of action to take.

The IT auditor should only propose recommendations that are appropriate based on the nature and extent of the problems. A minor variance may not warrant highlighting in an audit report or offering recommendations. Recommendations should be limited to those findings that have a significant impact on the company’s mission.

Deficiencies identified in this phase most often represent operational deficiencies. If they are not corrected before the application is placed into production, they can cause or lead to a system failure. At this point in the development cycle, there is no time to compensate for deficiencies in future phases. The following deficiencies are common to the evaluation and acceptance phase:

  Testing does not include all tests contained in the test plan, which results in untested functions being placed into production.
  A test report is not prepared or, if prepared, does not adequately indicate which areas have been validated to function correctly; this can result in applications being placed in production without the user knowing what does and does not work.
  User management is not involved in the decision of whether to put the system into production; therefore, systems are placed into production that may have defects.
  The test plan, test results, and test reports either are not complete or are not prepared in a format that can be used as ongoing maintenance documentation, and maintenance personnel must therefore spend more time and money reproducing test conditions and test results.
  A parallel test is not conducted; when current capabilities are in existence, this leaves the user uncertain as to whether the new system can produce the same results as the old system.
  The system is not field-tested at selected locations and thus it may not work properly in the operational environment.
  Systems development documentation is not updated to reflect the changes and activities that occurred during development. The application may end up being maintained based on inaccurate documentation, with the potential of either increasing the defect rate or maintenance costs.
  A written installation and conversion plan is not prepared and followed; this results in the potential for increased conversion costs and inaccurately or incompletely performed conversion tasks.
  System security is not certified, causing potential security vulnerabilities in the operational system.

EVALUATING AUDIT RESULTS AND PLANS

The IT Auditor should complete the audit program once the system becomes operational. The insight gained during the developmental process should be passed on to the audit team reviewing the operational system in order to properly focus and maximize the audit effort. The insight to be included in the program is described in the previous phase.

The auditor must select the final operational audit tools during this phase. These tools should be tested (when the system is tested) to ensure that they work. Thus, the auditor undergoes an evaluation and acceptance of audit tools while user management undergoes an evaluation and acceptance of the system.


Previous Table of Contents Next