Welcome » IT Booklets » Information Security » Security Monitoring » Condition Monitoring » Independent Tests
Independent tests include penetration tests, audits, and assessments. Independence provides credibility to the test results. To be considered independent, testing personnel should not be responsible for the design, installation, maintenance, and operation of the tested system, or the policies and procedures that guide its operation. The reports generated from the tests should be prepared by individuals who also are independent of the design, installation, maintenance, and operation of the tested system.
Penetration tests, audits, and assessments can use the same set of tools in their methodologies. The nature of the tests, however, is decidedly different. Additionally, the definitions of penetration test and assessment, in particular, are not universally held and have changed over time.
Penetration Tests. A penetration test subjects a system to the real-world attacks selected and conducted by the testing personnel. The benefit of a penetration test is that it identifies the extent to which a system can be compromised before the attack is identified and assesses the response mechanism's effectiveness. Because a penetration test seldom is a comprehensive test of the system's security, it should be combined with other monitoring to validate the effectiveness of the security process.
Audits. Auditing compares current practices against a set of standards. Industry groups or institution management may create those standards. Institution management is responsible for demonstrating that the standards it adopts are appropriate for the institution.
Assessments. An assessment is a study to locate security vulnerabilities and identify corrective actions. An assessment differs from an audit by not having a set of standards to test against. It differs from a penetration test by providing the tester with full access to the systems being tested. Assessments may be focused on the security process or the information system. They may also focus on different aspects of the information system, such as one or more hosts or networks.
Management is responsible for considering the following key factors in developing and implementing independent tests:
Personnel. Technical testing is frequently only as good as the personnel performing and supervising the test. Management is responsible for reviewing the qualifications of the testing personnel to satisfy itself that the capabilities of the testing personnel are adequate to support the test objectives.
Scope. The tests and methods utilized should be sufficient to validate the effectiveness of the security process in identifying and appropriately controlling security risks.
Notifications. Management is responsible for considering whom to inform within the institution about the timing and nature of the tests. The need for protection of institution systems and the potential for disruptive false alarms must be balanced against the need to test personnel reactions to unexpected activities.
Data Integrity, Confidentiality, and Availability. Management is responsible for carefully controlling information security tests to limit the risks to data integrity, confidentiality, and system availability. Because testing may uncover nonpublic customer information, appropriate safeguards to protect the information must be in place. Contracts with third parties to provide testing services should require that the third parties implement appropriate measures to meet the objectives of the 501(b) guidelines. Management is responsible for ensuring that employee and contract personnel who perform the tests or have access to the test results have passed appropriate background checks, and that contract personnel are appropriately bonded. Because certain tests may pose more risk to system availability than other tests, management is responsible for considering whether to require the personnel performing those tests to maintain logs of their testing actions. Those logs can be helpful should the systems react in an unexpected manner.
Confidentiality of Test Plans and Data. Since knowledge of test planning and results may facilitate a security breach, institutions should carefully limit the distribution of their testing information. Management is responsible for clearly identifying the individuals responsible for protecting the data and providing guidance for that protection, while making the results available in a useable form to those who are responsible for following up on the tests. Management also should consider requiring contractors to sign nondisclosure agreements and to return to the institution information they obtained in their testing.
Frequency. The frequency of testing should be determined by the institution's risk assessment. High-risk systems should be subject to an independent test at least once a year. Additionally, firewall policies and other policies addressing access control between the financial institution's network and other networks should be audited and verified at least quarterly.The quarterly auditing and verification need not be by an independent source. See NIST Special Publication 800-41. Factors that may increase the frequency of testing include the extent of changes to network configuration, significant changes in potential attacker profiles and techniques, and the results of other testing.
Proxy Testing. Independent testing of a proxy system is generally not effective in validating the effectiveness of a security process. Proxy testing, by its nature, does not test the operational system's policies and procedures, or its integration with other systems. It also does not test the reaction of personnel to unusual events. Proxy testing may be the best choice, however, when management is unable to test the operational system without creating excessive risk.