Previous | Table of Contents | Next |
The IT Audit Professional will review this phase to ensure that the system has been evaluated and that the end users have formally accepted the system before it has gone live in a production environment. During this phase, unit testing is completed, and integration and system testing are undertaken.
The results of these tests provide user management with the information it needs to decide whether to accept, modify, or reject the system. The total system and data must be validated and must fully meet all user requirements.
The IT auditor should continue to emphasize internal control requirements as an area requiring specific audit attention. The fulfillment of this objective can be done in conjunction with the validation, verification, and testing plan and specificationsor independently of that plan.
In any event, the IT auditor should review the work of the validation, verification, and testing team and conduct additional tests as appropriate. The actual performance of the task usually is too time-consuming for the auditor to perform. It has been estimated that this phase of the developmental process can consume as much as 30 percent of the developmental effort.
If testing is properly performed, a test plan, test results, and a test report should be available. The test plan should indicate the systems functions and then cross-reference them to the tests designed to validate the correct operation of those functions. Test results should be specifically documented and retained. The test report should indicate the results of those tests and relate the results to the function, indicating whether it performs correctly.
Once the test results have been completed and prepared for the test report, the IT auditors role becomes much easier. In these instances, the auditor needs only to perform sufficient tests to ensure that the test results are correct. The auditor should then be able to draw the same conclusions from the test results as the test team drew.
The IT auditor should conduct this portion of the review in accordance with the revised validation, verification, and testing plan and specifications. Completed code is tested as described in the revised plan.
Three types of program testing are usually performed: unit, integration, and system. If performed properly, unit testing validates the functioning of the unit, integration testing validates the interfaces between the units and the operating environment, and system testing validates the interaction between the application system and the user area. Although it is often difficult to do, unit testing should be completed before integration testing commences, and integration testing before system testing commences.
Adequate time must be allocated to testing. Software testing is an underplanned and undermanaged facet of the developmental process because the previous phases are frequently completed late, although the installation date remains fixed.
After the review, analysis, and testing of the system, including execution of the programs on test data, the system should be field-tested in one or more representative operational sites. For particularly sensitive systems, disaster recovery and continuity of operations plans should be fully documented and operationally tested as well.
The IT auditor should also determine if the application or any of the data stored within it should be designated as sensitive. If this designation has been granted, then the security in the system should be comprehensively tested prior to implementation, and certified if the testing is successful.
Security evaluation should be part of the broader test results and test evaluation report. The accreditation statementthe last critical activity of the phaseis a statement from the responsible accrediting official (e.g., the sponsor/user or information resources management official) that the system is operating effectively and is ready to be installed. Any caveats or restrictions should be provided at this time.
Participants and Their Tasks
The IT Audit Professional should find that all or most of the participants responsible for the system play an active role in evaluation and acceptance. In the early phases, the responsible participants are frequently senior people. For example, the manager or assistant managers of the user area may be directly involved in the early developmental phases.
As the work becomes more technical, the responsibilities are frequently delegated downward to lower-level people in the operational areas. During the evaluation and acceptance phase, as critical decisions have to be made, the senior people should again be involved. The participants and their responsibilities in the evaluation and acceptance phase are:
Senior Management/Project Steering Committee: Approves the updated system decision paper to advance to the evaluation and acceptance phase, in consultation with the sponsor/user and the DP manager (this occurs between phases).
Project Sponsor: Approves the revised project plan and installation and conversion plan, updates system decision paper, oversees training, and accepts (accredits) system for operation.
Project Team: Updates the project plan, supports and oversees the test analysis and security evaluation report, and certifies system security. This person revises the user manual, operations and maintenance manual, and installation and conversion plan, based on test results.
Security Specialist: Reviews the test analysis and security evaluation report and security components of the installation and conversion plan.
Quality Assurance Specialist: Reviews the validation, verification, and testing results and advises responsible participants on system achievement of the needs statement.
Evaluation and Acceptance Phase Deliverables
The IT auditor evaluates the work performed in this phase by looking at the phase documentation. The phase produces one new document and updates of existing documents.
The new document is the test analysis and security evaluation report. This document details the test analysis results and findings; presents the demonstrated capabilities and deficiencies, including the security evaluation report needed for system certification; and provides a basis for preparing a statement of system and software readiness for implementation.
The IT auditors primary source of information for the procedures performed in this phase are the audit results and workpapers from previous phases. If the same individuals are involved in evaluation and acceptance as were involved in previous phases, background preparation work should be minimal.
The IT auditor, however, is still concerned with the flow of work, ensuring that the responsible participants have fulfilled their roles, and acquiring and reviewing the documentation produced during this phase.
Reviewing Programming Phase Output
At this point, the system is complete. The IT auditors objective of this phase is to identify and remove significant defects from the system, if have not already been addressed by someone else. This can be accomplished through the creation of a series of test conditions that, when processed against the executable code, produce the proper results by which the system is judged.
The IT auditor may wish to review some of the documents from the earlier phases because they indicate what the system is supposed to do. The programming phase documents are oriented toward what the system does to meet its objectives, and the user manual is unique to the company for which it was developed, explaining how the system should be operated by the end users.
The auditor must understand both the what and the how, in preparation for reviewing this final phase of the system development process. The auditor must also ensure that the test data and testing documentation is saved for use in validating subsequent changes to the system.
Previous | Table of Contents | Next |