Last month I got a chance to evaluate the two popular vulnerability assessment tools Webinspect and Appscan and I wanted to share my findings with others. As you will notice, currently I have published only few technical comparison I will add more to it sooner. This comparison is strictly between Appscan 6.5 and Webinspect 6.2. Both the companies have since come with a beta release of a new version of their products and this comparison may not be valid for their new versions.
The evaluation was done on a dynamic web application with approx 1800 web pages.
- Scan duration: On a website of approx. 1800 pages, Appscan took around 6 hours whereas webinspect took more then 12 hours (i had to stop it there as it would have taken a lot more time). Then i removed post-data injection from the list of attacks and the entire scan was completed in approx 4 hours.
- Ability to pause a scan: Both the tools give you an ability to pause a scan and restart later.
- Save a partial scan and restart later: Both the tools allow you to save a partial scan and restart it later.
- What if webapp stops responding during scan: If your webapp stops responding during the scan, Webinspect displays an error and pauses the scan, so you can fix the problem and resume the scan later. Though appscan does the same but since the pause button is not on the toolbar and is as a dropdown it get a little confusing.
- Change the network in the middle of a scan session: While a scan is in progress, if you want to pause it and resume from your home over a vpn connection, Appscan lets you resume the scan without any problems whereas Webinspect gave me an error. Though I just tried it once, I am not too sure if this is a problem with Webinspect. You can check it on your own if you're going to need this feature.
- Maximum thread that can run simultaneously: Appscan has maximum of 10 threads whereas Webinspect has 75. You can customize the number of threads you want to run simultaneously. However, if you use more then 15 threads in Webinspect, it becomes very resource intensive.
- Pick and choose attacks for a scan session: Both Appscan and Webinspect lets you do that.
- Skip an attack while the scan is in progress: Webinspect lets you skip an attack while the scan is in progress whereas Appscan doesn't.
- View the actual attack during a scan session: Webinspect displays the actual attack string on the status bar during the scan and also if a vulnerability was found whereas in Appscan you can only view if there was a vulnerability found.
- Customize the order of attack for a scan session: Neither of the product lets you customize the order of scan. So, if you want to run Cross site scripting and SQL injection first then you will have to create a new scan session and just opt for these two attacks and remove all others from the list.
- Custom attack scripts: Both Appscan and Webinspect lets you create a custom attack script as a macro. Webinspect however lets you create a custom attack agent using VB script.
- Vulnerability Database updated: Both the product update their vulnerabilities regularly.
- Vulnerabilities: Appscan found more vulnerabilities with more variants in a scan as compared to Webinspect. One other difference between the two products in terms of vulnerabilities is if there are 200 pages with same vulnerabilities, Appscan will display 1 vulnerability but can drill down to all the 200 pages whereas Webinspect will display as 1. Having said that, Appscan still detects more types of vulnerabilities then Webinspect.
- Login module at the start of scan:Appscan lets you record a login sequence as a macro whereas Webinspect gives you more options. It lets you record a login sequence as a macro or enter the authentication credentials in the tool itself. The problem however with entering the authentication credentials inside the tool is that if they are wrong, Webinspect does not detects them as wrong and goes ahead with the scan and hence may not scan the entire site. In the other scenario, if you are using macro to record the login sequence, then Webinspect records the complete login and logout sequence whereas Appscan only records the login sequence.
- Bookmark of follow up flag on a specific attack during a scan session: None of the tools currently have this feature.
- External tools: Both the products have additional tools bundled with them. Watchfire's power tools are not integrated with Appscan and can be downloaded and used separately. Appscan, however, provides a framework where external tools can be invoked from within Appscan. Webinspect on the other hand, have more tools and all of them are tightly integrated within the product. Webinspect does not allows you to invoke an external tool from within the application.
- Appscan scans for certain infrastructure vulnerabilities (like Apache, IIS, etc) whereas Webinspect does not.
This is by no means a complete report. I will add more points to it sooner so please check back again. Also, if there is something additional you would like to see in this report or if you would like to add to this report or even make changes to this report, please email me at anurag.agarwal "at" yahoo.com. I also got a chance to look at the beta release of Appscan 7.0 and Webinspect phoenix release and if you are looking to purchase one of these two products then I would strongly recommend looking at their beta releases of new versions.
