The Evolution of Security Testing – Security Boulevard

A benchmarking study by the NSA Center for Assured Software found that the average SAST tool covers only 8 out of 13 weakness classes and finds only 22 percent of flaws in each weakness class. Based on these numbers, the average SAST tool is likely to find only 14 percent of the vulnerabilities in an applications code.

Security needs to be part of the development experience. This has given rise to the application security space. And, like the internet itself, it needs to evolve. AppSec 1.0 was built with waterfall methodologies in mind. These include static analysis software testing and penetration testing and it assumes that security is binary. You are either secure or insecure, there is no grey area. Yet it is in the grey areas, the zero days, where the attackers lurk. It then becomes a question of code coverage is your application security solution providing protect your organization?

Verification and validation testing is conducted in two different ways:

invalid set of inputs. The purpose of negative testing is to ensure the application remains stable in unexpected use cases. While this type of testing is typically conducted by security teams, modern DevOps shops may collaborate closely with QA or development teams.

Positive testing is easier to conduct. There is a finite number of features and flows introduced per release. Therefore, there is a finite combination of valid inputs to test. Thus, there is a clear definition of what done means. Security is, however, a game of cat and mouse. Organizations are always trying to stay one step ahead of the attacker. And when you are trying to stay ahead of something, speed (and agility) is everything. So negative testing is necessary.

Instead of looking at code as static, as either secure or insecure, AppSec 2.0 security testing understands the developer and attacker mindsets by identifying coding

mistakes early and often. Not only does this reduce costs but it results in faster time to market. Previously, with waterfall, developers were only given information as they needed it, when they needed it. The goal was to keep them focused and to only do what they do best. Dont pull them away if you dont have to. And, certainly, never come to them at the 11th hour market is demanding a change in the product or that theres a problem with something they weeks ago. Yet that is what happens in the modern world. Instead, organizations, in order to stay ahead, be innovative, yet they still need to think like an attacker. They need to know offense in order to implement defense. This is where fuzz testing plays a vital role.

Fuzz testing, or fuzzing, is a dynamic application security testing (DAST) technique for negative testing. Fuzzing aims to detect known, unknown, and zero-day vulnerabilities. Fuzzers send malformed inputs to targets. Their objective is to trigger bad behaviors, such as crashes, infinite loops, and/or memory leaks. These anomalous behaviors are vulnerabilities. Fuzzing helps organizations verify that the application works as expected, even in unexpected situations. This is key as ecosystems get complex. Its not just about people mistreating applications, its also about how an application will react if an integrated app misbehaves. In other words, if a system connected to your app acts up, can the app still function? Or will it crash. And if it crashes, does that allow for malicious code to run instead?

At a high level, fuzzing provides predictability. If testing is done continuously during the development cycle, this decreases time to market and should reduce the costs associated with the application over its lifetime. There will be a lower number of in-field issues when properly tested first. Fuzzing also contributes to productivity. Its a security test solution that protects developer productivity with zero false positives. Zero. The crashes that are reported are indeed reproducible vulnerabilities, allowing developers to address them quickly.

As the issues are shared and fixed, regression testing remembers the previous test crashes and verifies the remediations.

Fuzz testing is a heavy-weight yet versatile DAST solution that is able to conduct multiple types of testing across the SDLC. It runs quietly, continuously, and synchronously in the background as a part of the build process. Its also proven technology. Google, for example, identifies 80% of bugs with fuzz testing while the other remaining 20% is found through other means (SCA) or in production

There are different types of fuzzers available. Random fuzzers are just that: random. Its hard to tell how much code coverage is tested. Template takes known good input and mutates it. Again, its hard to tell how much code coverage there is. Guided fuzzing provides a roadmap, its more targeted. Here, the code coverage increases. to autonomously map out the usable parts of the

application for testing. Symbolic execution takes binaries and mathematically reasons through various logic and functions, so it can break into new areas of the program for further testing. Advanced fuzz testing is particularly effective for continuous testing because it aims to protect developer productivity at all costs. The key is autonomous test case management. Once Advanced Fuzz Testing identifies how, it breaks through functions, using guided fuzzing to craft test cases to test each branch. It uses the applications behavioral feedback to inform what test cases to generate next. If it identifies minor anomalous behavior, it will continue to probe by generating alternative variations.

See how continuous testing enables security teams to keep pace with development and operations teams in modern development, and to deliver deep integration and automation of security tooling.

Download the Whitepaper More Resources

This means better and more thorough testing and coverage. This is a significant breakthrough from conducting coverage-guided fuzzing alone. Guided fuzzers can be powerful with guidance from a technical security expert to help inform the fuzzer how to traverse through the code. However, without manual intervention, it will continue blindly, guessing inputs to break through

functions, which can result in breaking the code or the fuzzer being stuck for a while. It also deduplicates test cases to give developers the minimum set of test cases to thoroughly test the code.

For individual developers, theres component testing. As AFT identifies issues, developers are notified that there are issues. AFT has a zero false-positive rate and to prove it, it shares a test case also known as a witness to reproduce the issue. Theyre also provided system level

information that will help developers understand the consequence (from a technical standpoint) to the application. AFT runs deduplication. There may be a single defect. running all of them. AFT reduces them to the one test case. The point of this is to reduce testing efficiency.

The Value of AFT to the Development Team For development teams, theres regression testing. each owning a component of the larger application. Ideally, when the components are assembled, theyll work together seamlessly. This is hardly the reality. Google cites that 45% of bugs that they identify are through regression testing. AFT compiles the test case from each component for regression testing. This is where the dedup effort at the component level pays off in dividends because the regression testing time will be reduced through efficiency.

Time is of the essence in release cycles, especially in agile software development. Regression testing picks up where it left off, meaning it doesnt rerun regression tests reduces regression test cycles and ensures you continue to get the most out of your limited

The Value of Advanced Fuzz Testing

Fuzzing provides a proactive approach to security testing. It is the negative or non-functional testing. It shows whether or not an application can withstand unexpected situations, and it helps uncover zero days. One way to think about (and justify) Advanced Fuzz Testing is that it is penetration testing in a machine. Like pen testing, Advanced Fuzz Testing thinks box. However, there are benefits to Advanced Fuzz Testing not found with pen testing. Unlike pen testing, Advanced Fuzz Testing is continuous, not just a point in time. It can be done at human) speed. It can be performed at machine scale, and with machine accuracy. This coverage than what a human is capable of doing.

Learn more by downloading our Guide To Automated Continuous Security Testing

*** This is a Security Bloggers Network syndicated blog from Latest blog posts authored by Tamulyn Takakura. Read the original post at: https://forallsecure.com/blog/the-evolution-of-security-testing

More:

The Evolution of Security Testing - Security Boulevard