Security Testing at a Glance
Black box testing is generally used when the tester has limited knowledge of the system under test or when access to source code is not available. Within the security test arena, black box testing is normally associated with activities that occur during the pre-deployment test phase (system test) or on a periodic basis after the system has been deployed.
Black box security tests are conducted to identify and resolve potential security vulnerabilities before deployment or to periodically identify and resolve security issues within deployed systems. They can also be used as a “badness-ometer” to give an organization some idea of how bad the security of their system is. From a business perspective, organizations conduct black box security tests to conform to regulatory requirements, protect confidential and proprietary information, and protect the organization’s brand and reputation
Application Security Test Tools. The CSI/FBI 2005 Computer Crime and Security Survey showed a significant increase in web-based attacks, with a full 95% of survey respondents advising that they had experienced more than ten web site incidents during the last year. This was an exponential increase in web site incidents over the previous year, when only 5% of respondents experienced more than ten incidents.
Fortunately, a significant number of black box test tools focus on application security related issues. These tools concentrate on security related issues including but not limited to
- input checking and validation
- SQL insertion attacks
- injection flaws
- session management issues
- cross-site scripting attacks
- buffer overflow vulnerabilities
The tools are developed and distributed by a collection of open source communities and for-profit businesses such as the Open Web Application Security Project (OWASP), Cenzic, SPI Dynamics, NT Objectives, Sanctum, and others. As the trend for web application security incidents increases, the need for these tools to test Internet-enabled applications increases as well. 360logica is committed to follow tools described above.Pre-Deployment: During the Development Life Cycle. The relative cost of repairing software defects increases the longer it takes to identify the software bug . For example, we estimates that it can cost twenty times more to fix a coding problem that is discovered after the product has been released than it would have cost if discovered during the system test phase, when black box test tools are normally used.This figure should not be surprising considering the personnel and processes required to address a security issue after deployment. Help desk personnel are required to take trouble calls from the customer; support engineers are required to confirm and diagnose the problem; developers are needed to implement code fixes; QA personnel are called to perform system regression tests; and managers oversee the entire process. There are additional expenses to consider, such as those associated with patch distribution and the maintenance of multiple concurrently deployed versions. Additionally, a serious post-deployment software vulnerability may result in potential business issues, such as damage to brand or company reputation, and potential legal liability issues.Accordingly, black box security test tools can be used during the system test phase to identify and address these issues, reduce system development costs, and reduce business risks associated with company reputation and liabilityPost-Deployment: An Evolving Challenge. Unfortunately, the specific instance of security vulnerabilities is constantly changing. For example, each month catalogues approximately 300 new security vulnerabilities. CERT cataloged 3,780 vulnerabilities in 2004. These figures highlight the challenges all organizations face when using software in the conduct of their business.
Many of these failures are a direct result of improper security designs or implementation errors that are introduced during development. Organizations that purchase software do not have control over these issues. Fortunately, many black box security test tools are periodically updated to test for these newly discovered vulnerabilities, allowing businesses to conduct periodic security tests of their systems.
Benefits and Limitations of Black Box Testing. As previously discussed, black box tests are generally conducted when the tester has limited knowledge of the system under test or when access to source code is not available. On its own, black box testing is not a suitable alternative for security activities throughout the software development life cycle. These activities include the development of security-based requirements, risk assessments, security-based architectures, white box security tests, and code reviews. However, when used to complement these activities or to test third-party applications or security-specific subsystems, black box test activities can provide a development staff crucial and significant insight regarding the system’s design and implementation.
- Black box tests can help development and security personnel.
- Identify implementation errors that were not discovered during code reviews, unit tests, or security white box tests.
- Discover potential security issues resulting from boundary conditions that were difficult to identify and understand during the design and implementation phases.
- Uncover security issues resulting from incorrect product builds (e.g., old or missing modules/files).
- Detect security issues that arise as a result of interaction with underlying environment (e.g., improper configuration files, unhardened OS and applications)
Accordingly, black box security test efforts complement the critical security activities throughout the SDLC. The tools help developers and security personnel verify that the system security components are operating properly and also identify potential security vulnerabilities resulting from implementation errors. Additionally, black box security tests can help security practitioners test third-party components that may be considered for integration into the overall system and for which source code is not available. These tests may help the development staff uncover potential security vulnerabilities and make intelligent decisions about the use of certain products within their overall system.
Although these tests should not be considered a substitute for techniques that help developers build security into the product during the design and implementation stages, without these tests, developers may overlook implementation issues not discovered in earlier phases. Despite the best efforts of the development staff, mistakes do occur—coding errors, incorrect components in the latest software build, unexpected interaction with the deployed environment, and boundary conditions, to name a few. Black box security tests provide a method to validate the security of the system before it is deployed.
Black box testing tools provide various types of automated support for testers. They help testers work more efficiently by automating whatever tasks can be automated, and they also help testers avoid making mistakes in a number of tasks where careful bookkeeping is needed. Their main roles include
Test automation: providing automated support for the actual process of executing tests, especially tests that have already been run in the past but are being repeated
Test scaffolding: providing the infrastructure needed in order to test efficiently
Test management: various measurements and scheduling and tracking activities that are needed for efficient testing even though they are not directly involved in the execution of test cases.