Previous Table of Contents Next


Reviewing Evaluation and Acceptance Phase Plans

The final phase is frequently between the point at which the programs are complete and the date when those programs must be placed into production. If the production date is firm, the time allocated to this phase may be insufficient.

Therefore, the IT auditor should determine that someone has determined that any critical systems have been tested and are functioning as expected. It is unrealistic to expect exhaustive testing to occur, although it is desirable. There are always compromises between budget and schedule, and complete testing.

Frequently, there are no options regarding when the system is placed into production, particularly when it is mandated by legislation. It is important to optimize the test time available.

GATHERING AND VERIFYING INFORMATION ON THE PHASE STATUS

The IT Audit Professional will look for the status testing reports that should have been developed and retained during the development process on the status of testing. The criteria for testing should be included in the verification and testing plan and specifications.

These criteria indicate which functions are to be tested and what conditions are to be used to test those functions. In the hierarchy of testing, the units or programs should be tested first. When these have been validated as performing correctly, the integration or interfaces among the units or programs are tested. Once those interfaces have been validated, the acceptance test, which validates the interfaces between the users and the system, is performed.

The test status reports should indicate which functions have been tested, which functions work, which functions are in the process of being corrected, and when those functions should be retested. At any point, the IT Audit Professional should be able to determine how many functions have been validated and how many remain invalid.

If this status information is not available, the IT auditor should be concerned over whether the end-product of testing will adequately indicate systems performance before the system is put into production. Without this information, management cannot make a knowledgeable decision regarding the installation and operation of the system.

By the time this phase commences, all of the work necessary to develop the system should be complete and the company should have an executable system. The company must be assured that the executable system meets the system requirements and specifications.

The IT Audit Professional will find that almost all of the development work that is performed in the evaluation and acceptance phase is testing. Once the modules are successfully tested, they can be assembled into programs. Some of the assembled programs involve utility programs and other aspects of the computer’s operating software. These programs are then tested prior to assembling them into modules.

Finally, the modules are put together as a system. This new system is then tested and validated to determine that it works in the operating environment, when interfacing with other systems, and that it meets user requirements.

The IT auditor must become familiar with the flow of work during this phase. This includes becoming familiar with the various types of testing and the expectations from those tests. As in other aspects of system development, the exact flow of documents varies from methodology to methodology and among companies using the same methodology.

The validation and acceptance phase may produce one completely new deliverable, while all the other deliverables are updates of deliverables from other phases. The auditor should look at all of these documents, but should emphasize verifying that the test analysis and security evaluation report properly implements and accomplishes the test plan objective and that the test results are properly reflected in the evaluation report.

The auditor must verify that all appropriate responsible participants are involved in this phase, that they have been assigned the appropriate role, and that they have correctly fulfilled that role. The auditor usually interviews the participants to verify their needed contribution. The desirable participants and the questions that the auditor must ask them follow.

Senior Management/Project Steering Committee.

  Have the appropriate updates to the project plan been made?
  Has the test analysis and security evaluation report been supported and overseen?
  Has the system security been certified?
  Has the user manual been revised, based on test results?
  Has the operations and maintenance manual been updated, based on test results?
  Has the installation and conversion plan been updated, based on test results?

Project Sponsor/Project Team.

  Have the test results been reviewed?
  Has the test analysis and security evaluation report been reviewed?
  Have the security components of the installation and conversion plan been reviewed?
  Has the revised project plan been approved?
  Has the revised installation and conversion plan been approved?
  Have the necessary updates been made in the system decision paper?
  Has the necessary training been overseen?
  Has the system been accepted for operation?

Security Specialist.

  Have the test results been reviewed?
  Has the test analysis and security evaluation report been reviewed?
  Do the updates to the user manual, operations and maintenance manual, and installation and conversion plan reflect any impact on the security documentation?

Quality Assurance Specialist.

  Have the validation, verification, and testing results been reviewed?
  Have responsible participants been advised on the system achievement of the needs statement?

SETTING OBJECTIVES FOR THE AUDIT

The IT auditor will find that the audit objectives to be accomplished during this phase vary with management’s needs. If the project team does not have an adequate test plan, management may ask the auditor to play a more active role in testing. Sometimes, the auditor performs some of the testing that occurs during this phase. Although this is not recommended, it is sometimes a necessity because testing would not be performed otherwise.

The test program outlined here includes the more common audit objectives for this phase. It is these objectives that must be customized on the basis of the needs of management as well as the audit evaluation of previous phases. This phase is the auditor’s last opportunity to evaluate the system before its placement into production. The greater the risks associated with the system or the greater the concerns uncovered in previous phases, the greater the need for audit involvement during this phase.


Previous Table of Contents Next