Previous Table of Contents Next


COLLECT DATA

Most of the work performed during an evaluation (including the planning phase) serves the purpose of data collection. Often, the techniques used to collect data represent building blocks in the construction of evaluation methods. The exact nature of the data to be collected depends on the evaluation methods and tools selected. This task covers the following common data collection techniques:

  Provision by management
  Document review
  Interviews

Provision by Management With this method of data collection, systems management provides introductory and detailed briefings and tutorials on the application and its security safeguards. It also includes the provision of four critical documents. Ideally, these documents already exist; however, they usually must be developed.

Security Requirements. Security requirements are the fundamental baseline for security testing. If an acceptable requirements statement does not exist, it must be formulated during certification. This is best accomplished through a joint effort of audit and development personnel. Audit personnel are needed because systems developers do not usually have a thorough understanding of security, especially with respect to external policies. Systems developers, however, have a better understanding of the application, especially with respect to user needs.

Risk Analysis. The application risk analysis shows threats and assets. This is useful in validating the requirements and defining the underlying problem to be solved. Again, when this document does not exist, it is best prepared through a joint effort by audit and development personnel.

Application Flow Diagram. The application flow diagram shows input, processing steps, and output. Complete transaction flows must be included for important transaction types. This is critical to understanding the application.

List of Application Controls. Controls can be the most difficult part of the security picture to define. One common difficulty is the seemingly simple task of distinguishing controls (e.g., authorization mechanisms, sequence checking) from application activities subject to control (e.g., initiation, recording, transcription, calculation). A useful rule of thumb is that a control is any protective action, device, procedure, technique, or other measure that reduces exposure.

Provision of this information by systems personnel can have benefits beyond that of easing the burden of data collection. In particular, it can increase the security awareness of systems personnel and draw the attention of testing personnel to application areas that are not well understood and might warrant further analysis.

IT auditors should not accept documentation provided by systems management as absolutely accurate, because systems personnel might not be objective. Document reviews and interviews can help validate this information. Nevertheless, documentation provided by systems personnel often proves to be an excellent source of information and has the added advantage of making the certification process as a whole less expensive.

Document Review. Document review becomes increasingly important as evaluation attention focuses on more detailed issues. The potential set of documents to be reviewed varies substantially in each certification, depending on objectives and the availability and value of documentation.

Interviews. Interviews, although time-consuming, can sometimes produce information not available through other means.

Ensuring Accurate Information One purpose of a certification program is to provide checks and balances. This purpose is not served if IT auditors simply report on the opinions of developers and users. Some interviewees may not know the facts, and others may misrepresent them. In addition, IT auditors may misinterpret the answers. Using interviews, instead of simply requiring subjects to complete questionnaires, improves information quality—personal interaction helps in interpreting meanings behind words, counteracting biases, and following leads.

CONDUCT BASIC EVALUATION

The basic evaluation typically suffices for most aspects of an application under review, although most applications also require some detailed evaluation work in problem areas. The general distinction between basic and detailed evaluation is primarily concerned with the overall functional security posture, not with the specific quality of individual controls. For example, basic evaluation is concerned with whether access authorization at the file level is sufficient or whether it might be required at the record level. As another example, evaluation might be concerned with whether authorization subjects must include terminals or individuals and processes. Basic evaluation is also concerned with verifying that security functions actually exist and that the implementation method is of sufficient, reliable quality. Detailed evaluation, on the other hand, is concerned with whether security functions work properly, satisfy performance criteria, and acceptably resist penetration.

There are three subtasks in a basic evaluation:

  Security requirements evaluation. Are applicable security requirements acceptable?
  Control existence determination. Do the security functions exist?
  Methodology review. Does the implementation method provide assurance that security functions are acceptably implemented?

Security Requirements Evaluation. The major purpose of certification is to determine whether system safeguards satisfy security requirements. This process is meaningful only if the system has well-defined security requirements; unfortunately, most do not. For certification to be useful, then, security requirements must be critically examined to determine whether they are reasonable and comply with IT, company, and user requirements. These requirements are usually documented in the project request form. When these requirements are not documented, they must be formulated. Accurate, complete, and understandable security requirements are fundamental to certification. To formulate and evaluate security requirements for a system, the IT auditor must consider two classes of needs—policy and situational. Policy needs derive from principles and practices (e.g., laws, regulations, standards, and company policies) with which the system must comply. Situational needs derive from the system’s characteristics and environment. To determine situational needs, four primary areas are considered:

  Assets. What should be protected?
  Threats. What are assets being protected against?
  Exposures. What could happen to assets if a threat is realized?
  Controls. How effective are security safeguards in reducing exposures?


Previous Table of Contents Next