Penetration-Testing
Comparison of penetration testing tools for web applications Frank van der Loo
Supervisor: Erik Poll
Research number: 653
Student number: 0314005
August 15, 2011
Executive summary
Testing the security of web applications with automated penetration testing tools produces relatively quick and easy results. However there are a lot of such tools, both commercial and free. In this thesis a selection of such tools are tested against a number of dierent test cases to compare the tools and nd out the quality of such tools. For each test case the number of reported vulnerabilities by the tools is recorded per type of vulnerability. For each type of vulnerability the reported vulnerabilities are manually checked for false positives and false negatives. The tools leave much to be desired. The tools appear to have problems with web applications that use techniques that are a bit more advanced than average pages, such as cookies for logging in or session ids. Further, the tools produce quite a lot of false positives and duplicate results. Also, all tools had false negatives. These false positives, duplicates and false negatives would have to be checked manually. This can take hours, especially for big web applications.
Some of the tools also have problems with crawling a web application when techniques such as includes are used. Another problem is that the tools are mainly good in nding SQL injection and XSS, while other vulnerabilities are not always detected by every tool. Other problems of the tools are that they depend on the server for some vulnerabilities (mainly SQL injection) and fail detection of this vulnerability for certain servers. Some of the tools have their own speci c problems that causes the tools to miss certain vulnerabilities.
Ultimately, it is impossible to name a tool that is the best. The usefulness of the tools depends on the web application that is going to be tested and the vulnerabilities that it is going to
Supervisor: Erik Poll
Research number: 653
Student number: 0314005
August 15, 2011
Executive summary
Testing the security of web applications with automated penetration testing tools produces relatively quick and easy results. However there are a lot of such tools, both commercial and free. In this thesis a selection of such tools are tested against a number of dierent test cases to compare the tools and nd out the quality of such tools. For each test case the number of reported vulnerabilities by the tools is recorded per type of vulnerability. For each type of vulnerability the reported vulnerabilities are manually checked for false positives and false negatives. The tools leave much to be desired. The tools appear to have problems with web applications that use techniques that are a bit more advanced than average pages, such as cookies for logging in or session ids. Further, the tools produce quite a lot of false positives and duplicate results. Also, all tools had false negatives. These false positives, duplicates and false negatives would have to be checked manually. This can take hours, especially for big web applications.
Some of the tools also have problems with crawling a web application when techniques such as includes are used. Another problem is that the tools are mainly good in nding SQL injection and XSS, while other vulnerabilities are not always detected by every tool. Other problems of the tools are that they depend on the server for some vulnerabilities (mainly SQL injection) and fail detection of this vulnerability for certain servers. Some of the tools have their own speci c problems that causes the tools to miss certain vulnerabilities.
Ultimately, it is impossible to name a tool that is the best. The usefulness of the tools depends on the web application that is going to be tested and the vulnerabilities that it is going to