OWASP Benchmark Project¶
The OWASP Benchmark for Security Automation (OWASP Benchmark) is a free and open test suite designed to evaluate the speed, coverage, and accuracy of automated software vulnerability detection tools and services (henceforth simply referred to as 'tools'). Without the ability to measure these tools, it is difficult to understand their strengths and weaknesses, and compare them to each other. Each version of the OWASP Benchmark contains thousands of test cases that are fully runnable and exploitable, each of which maps to the appropriate CWE number for that vulnerability.
You can use the OWASP Benchmark with Static Application Security Testing (SAST) tools, Dynamic Application Security Testing (DAST) tools like OWASP ZAP and Interactive Application Security Testing (IAST) tools. The current version of the Benchmark is implemented in Java. Future versions may expand to include other languages.
The OWASP Benchmark and Hdiv¶
Hdiv Detection (IAST) scored a 100%, which comes from a 100% true positive rate minus a 0% false positive rate.
OWASP Benchmark Scorecard¶
How to run the analysis of the OWASP Benchmark¶
If you wish to regenerate our Benchmark results, please proceed as follows.
Java, Maven and Git have to be installed in your environment.
Download the OWASP Benchmark Project from Github:
$ git clone https://github.com/OWASP/Benchmark.git $ cd Benchmark
pom.xmlfile and add the following lines to the
<properties> .. <cargo.jvmargs> -Dhdiv.config.dir=/Path-to-Hdiv-license-folder/ -Dhdiv.agent.debug=true -Dhdiv.xss.advanced=true -javaagent:/Path-to-Hdiv-Agent-folder/hdiv-ee-agent.jar -Dhdiv.file.level=FINE -Dhdiv.workingMode=FULL_DETECTION </cargo.jvmargs> <cargo.servlet.port>8443</cargo.servlet.port> <cargo.protocol>https</cargo.protocol> ... <properties>
- Save the file.
- Launch the Benchmark application and wait until it starts.
- In another terminal, run the Crawler and wait until it completes.
- An Hdiv report file will be generated:
- Move the report to
$ mv /Path-to-Hdiv-license-folder/hdivAgentLog.hlg ./results/
- Create scorecards. The following command will compute a Benchmark scorecard for all the results files in the
/resultsdirectory. The generated scorecard is put into the
- Check out the results.