Previous Table of Contents Next


Security Safeguard Evaluation Versus IT Audit

With respect to the technical processes themselves, a security safeguard evaluation and an IT audit have many similarities. For example, both assess compliance with policies; both assess the adequacy of safeguards; and both include tests to verify the presence of controls. Because IT audits are generally broader in scope (e.g., part of a general internal review), however, they often address such issues as cost and efficiency in achieving mission objectives that are outside the purview of evaluations for certifications.

Beyond this difference, there are others of a more subtle nature. For example, IT audits in general place more emphasis on data reliability and validate the data processed by the application (i.e., substantive testing). In a security safeguard evaluation, file inconsistencies are of interest mainly to the extent that they reveal inadequacies in the safeguards. As another example, IT audits usually are concerned with threats anticipated by systems developers and therefore are tested for in the application and in audit journals. Security safeguard evaluations, although also concerned with anticipated threats, are often additionally concerned that safeguards counter threats in which the application is used in ways not anticipated or intended by its developers. Penetration of an application through a design flaw is an example of an unanticipated threat. Analyses of these two forms of threats require different skills.

As both security audits and security evaluations evolve, the differences are diminishing and more overlap is occurring. For example, the historical limitation of IT audits to financial concerns is diminishing, as is the historical limitation of security evaluations to violations associated with unauthorized disclosure. IT audits now consider the entire spectrum of computer applications that are being used to manage information resources, and security safeguard evaluations increasingly consider such exposures as public embarrassment or competitive disadvantage that were formerly primarily of concern to IT auditors.

Plan Security Tests

The planning process is, in itself, a mini basic evaluation, because it must anticipate problem areas, needs for specialized skills and support tools, and other issues that cannot be determined without insightful situation-specific analysis. Indeed, the planning process might even determine that further evaluation is not required. This might be the case, for example, if planning analysis were to reveal general controls to be so weak that further evaluation would be of little value. (In such instances, the application still requires a security evaluation report and an accreditation decision.) Planning thus requires expertise in and knowledge of the application and the testing process. The enlistment of external support might be required to assist in planning.

During this task, the IT auditor must determine the need for security certification, partition the application system, assess the importance of that partition for evaluation, determine how resources should be allocated, and develop a security certification plan.

Determine the Need for Certifying Security. The IT auditor must decide whether certification is necessary. In some cases, management dictates the need for certification; in others, the IT auditor must make that evaluation.

Determine the Need for Certifying Security. The IT auditor must establish boundaries for certification. The rule of thumb is that the boundaries of an application must be drawn to include all relevant facets of an application’s environment, including the administrative, physical, and technical areas. Without this, certification gives an incomplete and perhaps misleading picture of application security. For example, technical controls might be excellent, but they are worthless if administrative security is not properly defined or if physical security is inadequate.

As boundaries are formulated, it is important to document security assumptions that are made about areas outside the boundaries. For example, if the operating system is excluded from the security review, documentation should note that the operating system is assumed to provide a sufficiently secure base with respect to process isolation, authentication, authorization, monitoring and maintaining the integrity of security labels, and enforcing security decisions. Once boundaries have been established, the IT auditor must decide how to partition the work within the boundaries. Sometimes, one person has the skills and experience to perform the full evaluation; more often, a team is required.

External reviews often suffice in some of the areas. For example, reviews of physical and personnel security might have been performed for the organization as a whole. An internal control review might exist for administrative and accounting controls. The operating system and hardware might have been audited regarding the adequacy of their security.

In partitioning the work, the IT auditor examines several characteristics of the application to estimate required number of personnel, skill levels of security evaluators, and evaluation time and activities. The following major characteristics are examined:

  Size. This is a critical planning factor. The larger the application or partition, the greater the required time and number of people.
  Complexity. This is based on such factors as the nature of the functions being performed, the extent to which operating system specifics need to be examined, and the clarity and level of abstraction of the languages used. Size and complexity are assessed not just for the application as a whole, but for each of its component parts.
  Documentation quality. This is an important consideration in planning the evaluation. There are several questions to ask in this area, including:
—Does an application flow diagram exist?
—Is a listing of controls available or must this information be gathered from application documentation?
—Does documentation distinguish security controls from other functions?
—Do functional requirements documents, system specifications, test documentation, procedure manuals, and other documents exist?
—Are they up to date, accurate and complete, understandable, and (especially for requirements documents) agreed on?

There may be other characteristics of the application that can affect the evaluation. Examples include a distribution of functions over physically separate sites and anticipated resistance from applications personnel.


Previous Table of Contents Next