What is Fuzz Testing?

Fuzzing (or fuzz testing) is an automated testing technique that automatically and repeatedly executes tests and generates new test cases at a very high frequency to detect faulty behavior of the system under test. This method stresses applications, exposing unexpected behaviors, crashes, and potential security flaws that traditional testing methods may overlook.

Fuzz testing is widely associated with cybersecurity. It has proven to be an effective mechanism for finding corner-case vulnerabilities that traditional human-driven verification mechanisms, such as unit and integration testing, can miss. Such vulnerabilities can often lead to malicious exploitation.

Paul Butcher, Head of Dynamic Analysis at AdaCore, explains fuzzing by comparing it to traditional testing methods.

“One effective way to explain fuzz testing is by contrasting it with more traditional testing methods, such as unit testing. In unit testing, we design specific inputs to exercise different aspects of the code and verify expected outputs against requirements through test assertions. These tests are finite, often aiming for a predefined level of code coverage.

Fuzz testing, on the other hand, shifts the focus towards bug detection and vulnerability discovery. Tools like GNATfuzz may still check functional correctness using mechanisms such as design by contract, but the core idea is to automate test case generation through rapid mutations of a starting corpus. Unlike unit tests, which typically execute hundreds or thousands of cases, fuzz testing generates and executes millions—or even billions—of inputs, aiming for a more exhaustive exploration.

By bombarding an application with numerous inputs and monitoring for anomalies, fuzz testing helps uncover issues like memory safety vulnerabilities. This exhaustive approach enhances confidence in the robustness of the code, complementing the finite nature of traditional verification methods."

Does fuzzing replace or complement other test methodologies?

At AdaCore, we would describe it as a complementary form of testing.

It's traditionally used as a security testing capability because it's very good at finding memory issues. For example, buffer overflows where, if you've adopted a more traditional form of unit testing, you'll tend to do things like a boundary value analysis where you'll take your input data and look at the smallest values, the biggest values, maybe add one and take one away from them and maybe a mid-range value.

You can gain some confidence in the assurance of whatever requirement you're trying to implement with that approach, but there are clearly huge amounts of permutations of data that you're missing. So fuzz testing tries to plug that gap by trying to find the weird corner-case scenarios in your application that can actually result in one of these memory safety issues.

Fuzz Testing in Safe and Secure Software Environments

Fuzz testing is increasingly crucial in high-assurance software environments, such as aerospace, automotive, and defense. These industries demand rigorous security and reliability standards to ensure software integrity, as failures or vulnerabilities can have severe, even life-threatening consequences. Fuzz testing can play a key role in multiple phases of high-integrity software development lifecycles, from early identification of hard-to-find software bugs to gathering evidence that a system is free of security vulnerabilities. More specifically, for automotive systems, ISO 26262 provides guidelines around the identification of robustness failures that may cause security vulnerabilities via automated fuzzing. Aerospace standards compliance has also evolved to include security considerations, particularly where a security vulnerability could lead to a safety hazard. For example, aviation systems needing FAA or EASA certification or military compliance now have multiple levels of requirements, including guarding against unauthorized electronic access.

Streamline Security Assurance with GNATfuzz

GNATfuzz leverages compiler-assisted fuzzing to detect software bugs and exploitable vulnerabilities and is available as part of the GNAT Dynamic Analysis Suite.

Fuzz-testing campaigns are complex and time-consuming to construct, execute, and monitor. GNATfuzz simplifies the process by analyzing an Ada code base and identifying subprograms that can act as fuzz-test entry points. GNATfuzz then automates the creation of test harnesses suitable for fuzzing. In addition, GNATfuzz will automate the building, executing, and analyzing of fuzz-testing campaigns.

GNATfuzz is a valuable tool for all Ada software across the entire spectrum of application domains, read more about GNATfuzz here. It is also essential in systems requiring high levels of security assurance and security-related safety assurance, since it simplifies the effort needed to implement fuzz-testing campaigns that meet the objectives of certification standards such as ED-202A and DO-326A.

Want to learn more? Read this interview with Electronic Design

https://www.electronicdesign.c...