Saturday, March 14, 2009

Testing Tool

Software Testing Information


One Stop Testing has been developed to give all the information on Software Testing in one place. This site contains the information on the following:


This section covers - Static Testing, Static material Testing, Static Testing Machine, Static Materials Testing, Software Testing Static Analysis, Static Control Testing, Static Testing Techniques, Static Pressure Testing.


Dynamic Testing:
This section covers - Dynamic Tribology Oscillation Testing, Dynamic Testing, Dynamic Testing Machine, Dynamic Material & Tear, Wall, Load Testing.


Blackbox Testing:
This section covers - What is blackbox testing, difference between blackbox testing and whitebox testing, Blackbox Testing plans, unbiased blackbox testing.


Whitebox Testing:
This section has notes on Whitebox Testing (QA).


Unit Testing:
This section covers - Software Unit Testing, tools, research topics, toolkits, extreme programming unit testing.


Requirements Testing:
This section covers - (half face) Respirator fit testing requirements, GFCI, NCLEX testing requirements, CNA testing in Georgia Requirements.


Regression Testing:
Software Regression Testing, Network Regression Testing (Software), Automated Regression Testing, Dod regression standard testing, Regression Testing Tools, Web Regression Testing.










Web Site Test Tools and Site Management Tools
More than 420 tools listed in 12 categories


Organization of Web Test Tools Listing - this tools listing has been loosely organized into the following categories:
Load and Performance Test

Tools




Validators and Link CheckersPERL and C Programs for

Validating and Checking
Web Functional/Regression




Note: Categories are not well-defined and some tools could have been listed in several categories; the 'Web Site Management Tools' category includes products that contain: site version control tools, combined utilities/tools, server management and optimization tools, and authoring/publishing/deployment tools that include significant site management or testing capabilities. Suggestions for category improvement are welcome; see bottom of this page to send suggestions.


Check listed tool/vendor sites for latest product capabilities, supported platforms/servers/clients, etc; new listings are periodically added to the top of each category section; date of latest update is shown at bottom of this page.


Also see How can World Wide Web sites be tested? in the FAQ Part 2 for a discussion of web site testing considerations; also see What's the best way to choose a test automation tool? in the LFAQ section; there are also articles about web site testing and management in the 'Resources' section.

Load and Performance Test Tools


BrowserMob - On-demand, self-service, low-cost, pay-as-you-go service enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.


Load Impact - Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes - server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.


Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.


AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.


fwptt - Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms


JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.


vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows


Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.


RealityLoad XF On-Demand Load Testing - An on-demand load testing service (no licenses) from Gomez.com. Leverages Gomez' peer panel, which consists of over 15,000 end-user desktop testing locations distributed across the world, to provide distributed load tests that accurately reproduce the network and latency characteristics encountered by real users in a live environment.


OpNet LoadScaler - Load test tool from OpNet Technologies Inc. Create tests without programming; generate loads against web applications, and other services including Web Services, FTP, and Email. Record end-user browser activity in the OPNET TestCreatorTM authoring environment to automatically generate test scripts in industry-standard JavaScript. Modify, extend and debug tests with the included JavaScript editor. Alternatively, drag and drop icons onto the test script tree. No knowledge of a scripting language is required to customize test scripts.


StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.


The Grinder - A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.


Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.


Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.


Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.


Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.


Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.


LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.


QEngine Web Performance Testing - Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.


NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.


Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.


WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.


QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.


Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.


SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.


httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.


NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.


WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.


Visual Studio Team System 2008 Test Edition - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio.


OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.


Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.


TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.


SiteStress - Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.


Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.


JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.


WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.


Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.


OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.


PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.


ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.


Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.


WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.


eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.


WebPerformance Load Tester - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.


WebSuite - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.


FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.


http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.


QALoad - Compuware's tool for load/stress testing of web, database, and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.


Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.


IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux an

d z/OS as distributed controller agents; provides high-level and detailed views of tests.


SilkPerformer - Enterprise-class load-testing tool from Borland (formerly Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.


Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Basic version available as open source, add-ons available for purchase. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.


Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.
Return to top of web tools listing

Java Test Tools
VisualVM - A free visual tool from Sun to monitor and troubleshoot Java applications. Runs on Sun JDK 6, but is able to monitor applications running on JDK 1.4 and higher. Utilizes various available technologies like jvmstat, JMX, the Serviceability Agent (SA), and the Attach API to get data and uses minimal overhead on monitored applications. Capabilities include: automatically detects and lists locally and remotely running Java applications; monitor application performance and memory consumption; profile application performance or analyze memory allocation; is able to save application configuration and runtime environment together with all taken thread dumps, heap dumps and profiler snaphots into a single application snapshot which can be later processed offline.


Cobertura - Free Java tool to identify which parts of a Java program are lacking test coverage and calculate % coverage; based on jcoverage. Instruments already-compiled Java bytecode; execute from ant or from the command line; generate reports in HTML or XML; shows % of lines and branches covered for each class, each package, and for the overall project. Shows McCabe cyclomatic code complexity of each class, and average cyclomatic code complexity for each package, and for the overall product. Can sort HTML results by class name, percent of lines covered, percent of branches covered, etc. and sort in ascending or decending order.


QCare - Static code analysis tool from ATX Software SA, supports Java, as well as COBOL, C#, C++, VB, SQL, PL/SQL, JavaScript and JSP.


JProfiler - Java profiling tool from ej-Technologies GmbH. Check for performance bottlenecks, memory leaks and threading issues.


Parallel-junit - Open source small library extensions for JUnit. Extends the junit.framework.TestSuite class by running tests in parallel, allowing more efficient test execution. Because TestResult and TestListener aren't designed to run tests in parallel, this implementation coordinates the worker threads and reorder event callbacks so that the TestResult object receives them in an orderly manner. In addition, output to System.out and System.err are also serialized to avoid screen clutter.


EMMA - Open-source toolkit, written in pure Java, for measuring and reporting Java code coverage. Targets support for large-scale enterprise software development while keeping individual developer's work fast and iterative. Can instrument classes for coverage either offline or on the fly (using an instrumenting application classloader); supported coverage types: class, method, line, basic block; can detect when a single source code line is covered only partially; coverage stats are aggregated at method, class, package, and "all classes" levels. Reports support drill-down, to user-controlled detail depth; HTML reports support source code linking. Does not require access to the source code; can instrument individial .class files or entire .jars (in place, if desired). Runtime overhead of added instrumentation is small (5-20%); memory overhead is a few hundred bytes per Java class.


PMD - Open source static analyzer scans java source for problems. Capabilities include scanning for: Empty try/catch/finally/switch statements; Dead code - unused local variables, parameters and private methods; Suboptimal code - wasteful string/stringBuffer usage; Overcomplicated expressions - unnecessary if statements, for loops that could be while loops; Duplicate code - copied/pasted code - could indicate copied/pasted bugs.


Hammurapi - Code review tool for Java (and other languages with latest version). Utilizes a rules engine to infer violations in source code. Doesn't fail on source files with errors, or if some inspectors throw exceptions. Parts of tool can be independently extended or replaced. Can review sources in multiple programming languages, perform cross-language inspections, and generate a consolidated report. Eclipse plugin.


TestNG - A testing framework inspired from JUnit and NUnit; supports JDK 5 Annotations, data-driven testing (with @DataProvider), parameters, distribution of tests on slave machines, plug-ins (Eclipse, IDEA, Maven, etc); embeds BeanShell for further flexibility; default JDK functions for runtime and logging (no dependencies).


Concordian - An open source testing framework for Java developed by David Peterson. Utilizes requirements in plain English using paragraphs, tables and proper punctuation in HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow test scenarios to be checked against the system to be tested. The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification. The fixture is also a JUnit test case. Results are exported with the usual green and red indicating successes and failures. Site includes info re similarities and diffs from Fitnesse.


DBUnit - Open source JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts a database into a known state between test runs. Enables avoidance of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. Has the ability to export and import database data to and from XML datasets. Can work with very large datasets when used in streaming mode, and can help verify that database data matches expected sets of values.


StrutsTestCase - Open source Unit extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework, including validation methods. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing of Struts code with or without a running servlet engine. Uses the ActionServlet controller to test code, enabling testing of the implementation of Action objects, as well as mappings, form beans, and forwards declarations.


DDSteps - A JUnit extension for building data driven test cases. Enables user to parameterize test cases, and run them more than once using different data. Uses external test data in Excel which is injected into test cases using standard JavaBeans properties. Test cases run once for each row of data, so adding new tests is just a matter of adding a row of data in Excel.


JKool - A light weight performance measurement and monitoring tool from Nastel Inc. for live J2EE, Web and Web service-based applications. It provides timing information for web sessions, including JSP/servlets, JDBC, JMS and Java method calls, to measure performance, detect bottlenecks and failures. Probes include a Web probe (JSP, Servlets), a Java probe (Byte Code Instrumentation), a JMS probe, and a JDBC probe.


StrutsTestCase for JUnit - Open source extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing Struts code with or without a running servlet engine. Because it uses the ActionServlet controller to test code, can test not only the implementation of Action objects, but also mappings, form beans, and forwards declarations. Since it already provides validation methods, it's quick and easy to write unit test cases.


JavaNCSS - A free Source Measurement Suite for Java by Clemens Lee. A simple command line utility which collects various source code metrics for Java. The metrics are collected globally, for each class and/or for each function.


Open Source Profilers for Java - Listing of about 25 open source code profilers for Java from 2006 from the Manageability.org web site.


SofCheck Inspector - Tool from SofCheck Inc. for analysis of Java for logic flaws and vulnerabilities. Exlpores all possible paths in byte code and detects flaws and vulnerabilities in areas such as: array index out of bounds, buffer overflows, race conditions, null pointer dereference, dead code, etc. Provides 100% path coverage and can report on values required for 100% unit test coverage. Patented precondition, postcondition and presumption reporting can help detect Malware code insertion.


CodePro - Suite of Java tools from Instantiations Inc. CodePro AnalytixX is an Eclipse-based Java software testing tool and includes features like code audit, metrics, automated unit tests, and more. CodePro Profiler, an Eclipse-based Java profiling tool enables inspection of a running application for performance bottlenecks, detect memory leaks and solve thread concurrency problems. EclipsePro Test automatically generates JUnit tests and includes an editor and analysis tool, provides test cases/results in tabular layout; mouse over failing case and Editor shows the failure message. WindowTester Pro for Swing or SWT UI's enables recording of GUI tests; watches actions and generates test cases automatically; customize the generated Java tests as needed. Provides a rich GUI Test Library, hiding complexities and threading issues of GUI test execution; test cases are based on the JUnit standard


Squish for Java - Automated Java GUI testing tool for Java Swing, AWT, SWT and RCP/Eclipse applications. Record or create/modify scripts using Tcl, Python, JavaScript. Automatic identification of GUI objects of the AUT; inspect AUT's objects, properties and methods on run-time using the Squish Spy. Can be run via a GUI front-end or via command line tools. Can execute tests in a debugger allowing setting breakpoints and stepping through test scripts.


Klocwork K7 - Static analysis technology for Java, C, C++, analyzes defects & security vulnerabilities, architecture & header file anomalies, metrics. Developers can run Klocwork in Eclipse or various other IDE's. Users can select scope of reporting as needed by selecting software component, defect type, and defect state/status.


Coverity Prevent - Tool from Coverity Inc. for analysis of Java source code for security issues. Explores all possible paths in source code and detects security vulnerabilities and defects in multiple areas: memory leaks, memory corruption, and illegal pointer accesses, buffer overruns, format string errors and SQL injections vulnerabilities, multi-threaded programming

concurrency errors, etc.


GUIDancer - Eclipse-based tool from Bredex GmbH for automated testing of Java/Swing GUI's, Tests are specified, not programmed - no code or script is produced. Test specification is initially separate from the AUT, allowing test creation before the software is fully functional or available. Specification occurs interactively; components and actions are selected from menus, or by working with the AUT in an advanced "observation mode". Test results and errors viewable in a results view, can be saved as html or xml file.


CMTJava - Complexity measurement tool from Verifysoft GmbH. Includes McCabe cyclomatic complexity, lines-of-code metrics, Halstead metrics, maintainability index.


JavaCov - A J2SE/J2EE Coverage testing tool from Alvicom; specializes in testing to MC/DC (Modified Condition/Decision Coverage) depth. Capabilities include: Eclipse plugin; report generation into HTML and XML; Apache Ant integration and support for test automation.


Jameleon - Open source automated testing harness for acceptance-level and integration testing, written in Java. Separates applications into features and allows those features to be tied together independently, in XML, creating self-documenting automated test cases. These test-cases can then be data-driven and executed against different environments. Easily extensible via plug-ins; includes support for web applications and database testing.


Agitator - Automated java unit testing tool from Agitar Software. Creates instances of classes being exercised, calling each method with selected, dynamically created sets of input data, and analyzing results. Stores all information in XML files; works with Eclipse and a variety of IDEs. Also available are: automated JUnit generation, code-rule enforcement, and more.

etc. - works with a variety of configurable and modifiable rulesets. Integrates with a wide variety of IDE's.


JLint - Open source static analysis tool will check Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.


Lint4j - A static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis. Eclipse, Ant and Maven plugins available.
FindBugs - Open source static analysis tool to inspect Java bytecode for occurrences of bug patterns, such as difficult language features, misunderstood API methods, misunderstood invariants when code is modified during maintenance, garden variety mistakes such as typos, use of the wrong boolean, etc. Can report false warnings, generally less than 50%.


CheckStyle - Open source tool for checking code layout issues, class design problems, duplicate code, bug patterns, and much more.
Java Development Tools - Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.
AppPerfect Test Studio - Suite of testing, tuning, and monitoring products for java development from AppPerfect Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.
GJTester - Java unit, regression, and contract (black box) test tool from TreborSoft. Enables test case and test script development without programming. Test private and protected functions, and server application's modules, without implementing test clients, regression testing for JAVA VM upgrades. Useful for testing CORBA, RMI, and other server technologies as well. GUI interface emphasizing ease of use.
QFTest - A cross-platform system and load testing tool from Quality First Software with support for for Java GUI test automation (Swing, Eclipse/SWT, Webstart, Applets, ULC). Includes small-scale test management capabilities, capture/replay mechanism, intuitive user interface and extensive documentation, reliable component recognition and can handle complex and custom GUI objects, integrated test debugger and customizable reporting.
Cactus - A simple open-source test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files making up an application: source code but also meta-data files (such as deployment descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically use within your IDE, or from the command line, using Ant. From Apache Software Foundation.
JUnitPerf - Allows performance testing to be dynamically added to existing JUnit tests. Enables quick composition of a performance test suite, which can then be run automatically and independent of other JUnit tests. Intended for use where there are performance/scalability requirements that need re-checking while refactoring code. By Mike Clark/Clarkware Consulting, licensed under the BSD License.
Koalog Code Coverage - Code coverage analyzer for Java applications from Koalog SARL. Includes: in-process or remote coverage computation, capability of working directly on Java method binaries (no recompilation), predefined (XML, HTML, LaTex, CSV, TEXT) or custom report generation, and session merging to allow compilation of overall results for distinct executions. Integrates with Ant and JUnit.
Abbot Java GUI Test Framework - Testing framework by Timothy Wall provides automated event generation and validation of Java GUI components, improving upon the very basic functions provided by the java.awt.Robot class. (Abbot = "A Better 'Bot'). The framework may be invoked directly from Java code or accessed without programming through the use of scripts via 'Costello', a script editor/recorder. Suitable for use both by developers for unit tests and QA for functional testing. Free - available under the GNU Lesser General Public License
JUnit - Framework to write repeatable java unit tests - a regression testing framework written by Erich Gamma and Kent Beck. For use by developers implementing unit tests in Java. Free Open Source Software released under the IBM Public License and hosted on SourceForge. Site includes a large collection of extensions and documentation.
jfcUnit - Framework for developing automated testing of Java Swing-based applications at the UI layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free Open Source Software from SourceForge site.
JBench - Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for speed. Available as binary distribution (including documentation), source distribution, or jar file.
Clover - Code coverage tool for Java from Atlassian. Fully integrated plugin for Eclipse, IntelliJ IDEA and projects using Apache ANT and Maven. View coverage data in XML, HTML, PDF, or via a Swing GUI. Tracks cyclomatic complexity. TestOptimization automatically prioritises just the tests needed to cover the particular changes made.
JCover - Java code test coverage analysis tool from Codework Limited. Works with source or compiled files. Gathers coverage measures of branches, statements, methods, classes, file, package and produces reports in multiple formats. Coverage difference comparison between runs. Coverage API provided.
Structure101 - Java source code visualization tool from Headway Software. Lets user understand, measure, and control architecture, design, composition, and dependencies of code base. Analyzes byte code and shows all dependencies, at all levels and between all levels; method, class, package, application. Measures code complexity using a measurement framework called XS. For Windows, Linux and Mac OS X.
Java Tool Suite from Man Machine Systems - Includes JStyle, a Java source analyzer to generate code comments and metrics such as inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java code according to specified options; JCover test coverage analyzer; JVerify Java class/API testing tool uses an invasive testing model allowing access to internals of Java objects from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool and technique for writing reliable software; JEvolve, an intelligent Java code evolution analyzer that automatically analyzes multiple versions of a Java program and shows how various classes have evolved across versions; can 'reason' about selective need for regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool that automatically builds a Java-based test data generator.
JProbe Suite - Collection of Java debugging tools from Quest Software; includes JProbe Profiler and JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and race conditions. JProfiler freeware version available.
Krakatau Professional for Java - Software metrics tool from Power Software includes more than 70 OO, procedural, complexity, and size metrics related to reusability, maintainability, testability, and clarity. Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality improvement.
Jtest - ParaSoft's Jtest is an integrated, automatic unit testing and standards compliance tool for Java. It automatically generates and executes JUnit tests and checks whether code follows 400 coding standards and can automatically correct for many.
DevPartner Java Edition - Compuware's (formerly NuMega) debugging/productivity tool to detect and diagnose Java bugs and memory and performance problems; thread and event analysis, coverage analysis. Integrates with several Java IDE's.
VTune - Intel's performance tuning tool for applications running on Intel processors; includes Java support. Includes suggestions for optimization techniques.
TCAT for Java - Part of Software Research's TestWorks suite of test tools; code coverage analyzer and code analysis for Java; written in Java.
Open Source code analyzers listing - A listing of open source Java code analysis tools written in Java.
Open Source code coverage tools listing - A listing of open source Java code coverage tools written in Java.
Open Source Java test tools listing - A listing of open source tools and frameworks for Java testing, written in Java.
Open Source web test tools listing - A listing of open source web test tools and frameworks written in Java.
(Note: some other tools in these listings also handle testing, management, or load testing of java applets, servlets, and applications, or are planning to add such capabilities. Check listed web sites for current information.)
Return to top of web tools listing

Link Checking Tools
LinkTiger - Hosted link checker; free and $pro versions. Capabilities include e-mail alerts, dashboard, reporting; canned reports or create rich custom reports. Scans PDF, CSS, Flash and MS Office files, flash-animation.
SiteAnalysis - Hosted service from Webmetrics, used to test and validate critical website components, such as internal and external links, domain names, DNS servers and SSL certificates. Runs as often as every hour, or as infrequent as once a week. Ideal for dynamic sites requiring frequent link checking.
HiSoftware Link Validation Utility - Link validation tool; available as part of the AccVerify Product Line.
ChangeAgent - Link checking and repair tool from Expandable Language. Identifies orphan files and broken links when browsing files; employs a simple, familiar interface for managing files; previews files when fixing broken links and before orphan removal; updates links to moved and renamed files; fixes broken links with an easy, 3-click process; provides multiple-level undo/redo for all operations; replaces links but does not reformat or restructure HTML code. For Windows.
Link Checker Pro - Link check tool from KyoSoft; can also produce a graphical site map of entire web site. Handles HTTP, HTTPS, and FTP protocols; several report formats available. For Windows platforms.
Web Link Validator - Link checker from REL Software checks links for accuracy and availability, finds broken links or paths and links with syntactic errors. Export to text, HTML, CSV, RTF, Excel. Freeware 'REL Link Checker Lite' version available for small sites. For Windows.
Site Audit - Low-cost on-the-web link-checking service from Blossom Software.
Xenu's Link Sleuth - Freeware link checker by Tilman Hausherr; supports SSL websites; partial testing of ftp and gopher sites; detects and reports redirected URL; Site Map; for Windows.
Linkalarm - Low cost on-the-web link checker from Link Alarm Inc.; free trial period available. Automatically-scheduled reporting by e-mail.
Alert Linkrunner - Link check tool from Viable Software Alternatives; evaluation version available. For Windows.
InfoLink - Link checker program from BiggByte Software; can be automatically scheduled; includes FTP link checking; multiple page list and site list capabilities; customizable reports; changed-link checking; results can be exported to database. For Windows. Discontinued, but old versions still available as freeware.
LinkScan - Electronic Software Publishing Co.'s link checker/site mapping tool; capabilities include automated retesting of problem links, randomized order checking; can check for bad links due to specified problems such as server-not-found, unauthorized-access, doc-not-found, relocations, timeouts. Includes capabilities for central management of large multiple intranet/internet sites. Results stored in database, allowing for customizable queries and reports. Validates hyperlinks for all major protocols; HTML syntax error checking. For all UNIX flavors, Windows, Mac.
CyberSpyder Link Test - Shareware link checker by Aman Software; capabilities include specified URL exclusions, ID/Password entries, test resumption at interruption point, page size analysis, 'what's new' reporting. For Windows.
Return to top of web tools listing

HTML Validators
RealValidator - Shareware HTML validator based on SGML parser by Liam Quinn. Unicode-enabled, supports documents in virtually any language; supports XHTML 1.0, HTML 4.01, HTML 4.0, HTML 3.2, HTML 3.0, and HTML 2.0 ; extensible - add proprietary HTML DTDs or change the existing ones; fetches external DTDs by HTTP and caches them for faster validation; HTML 3.2 and HTML 4.0 references included as HTML Help. For Windows.
HTML Validator - Firefox add-on, open source by Marc Gueury. The validation is done on your local machine inside Firefox and Mozilla. Error count of an HTML page is seen as an icon in the status bar when browsing. Can validate the HTML sent by the server or the HTML in the memory (after Ajax execution). Error details available when viewing the HTML source of the page. Based on Tidy and OpenSP (SGML Parser). Available in 17 languages and for Windows and other platforms.
CSE 3310 HTML Validator - HTML syntax checker for Windows from AI Internet Solutions. Supports wide variety of standards; accessibility (508) checking; uppercase/lowercase converter. Free 'lite' version. For Windows.
(Note: Many of the products listed in the Web Site Management Tools section include HTML validation capabilities.)
Return to top of web tools listing

Free On-the-Web HTML Validators and Link Checkers
Site Check - Type in one URL and automatically run HTML and stylesheet validators, accessibility assessment, link check, load time check, and more. Organizes access to a collection of free online web test tools. Site of UITest.com/Jens Meiert. Also lists a wide variety of free online web analysis/development/test tools.
Link Valet - Online link checker, includes capability fot hilight links modified since a specified date.
Dead-Links.com - Free link-checker limited to 25 pages per domain and 150 external documents. Higher limits if site has a link to Dead-Links.com.
WDG HTML Validator - Web Design Group's validator - latest HTML version support, flexible input methods, user-friendly error messages.
Web Page 'Purifier' - Free on-the-web HTML checker by DJ Delorie allows viewing a page 'purified' to HTML 2.0, HTML 3.2, HTML 4.0, or WebTV 1.1. standards.
W3C HTML Validation Service - HTML validation site run by the WWW Consortium (the folks who set web standards); handles one URL at a time; Can choose from among 30 character encoding types, and multiple HTML and XHTML document types/versions.
W3C CSS Validation Service - CSS validation site run by the WWW Consortium (the folks who set web standards); handles one URI at a time; or upload file or validate by direct input.
W3C Link Checker - Link checking service run by the WWW Consortium (the folks who set web standards); configurable. Handles one URL at a time. PERL source also available for download.
Weblint Gateway - Site with online HTML validator; somewhat configurable. Site provided by San Francisco State University.
Web Page Backward Compatibility Viewer - On-the-web HTML checker by DJ Delorie; will serve a web page to you with various selectable tags switched on or off; very large selection of browser types; to check how various browsers or versions might see a page.
Return to top of web tools listing

PERL and C Programs for Validating and Checking
W3C Link Checker - Link checker PERL source code, via the WWW Consortium (the folks who set web standards); configurable. Handles one URL at a time.
HTML TIDY - Free utility available from SourceForget.net; originally by Dave Raggett. For automatic fixing of HTML errors, formatting disorganized editing, and finding problem HTML areas. Available as source code or binaries.
Big Brother - Freeware command-line link checker for Unix, Windows, by Francois Pottier. Available as source code; binary avaialable for Linux.
LinkLint - Open source Perl program checks local/remote HTML links. Includes cross referenced and hyperlinked output reports, ability to check password-protected areas, support for all standard server-side image maps, reports of orphan files and files with mismatching case, reports URLs changed since last checked, support of proxy servers for remote URL checking. Distributed under Gnu General Public License. Has not been updated in recent years.
MOMspider - Multi-Owner Maintenance Spider; link checker. PERL script for a web spider for web site maintenance; for UNIX and PERL. Utilizes the HTTP 'HEAD' request instead of the 'GET' request so that it does not require retreival of the entire html page. This site contains an interesting discussion on the use of META tags. Not updated in recent years.
HTMLchek for awk or perl - Old but still useful HTML 2.0 or 3.0 validator programs for AWK or PERL by H. Churchyard; site has much documentation and related info. Not updated in recent years.
Return to top of web tools listing

Web Functional/Regression Test Tools
SafariWatir - Ruby gem that adds Watir support for Safari on the Mac (the originial Watir is for Windows); open source by Dave Hoover and others. For OS X running Safari.
Celerity - An open source JRuby wrapper around HtmlUnit. Runs as a headless Java browser - speeding up web testing; Java threads enablle running tests in parallel; can run in background. JavaScript support. Provides simple API for programmatic navigation thu web apps. Intended to be API compatible with Watir. For any platform.
Webrat - Ruby-based utility to enable quick development of web app acceptance tests. Open source by Bryan Helmkamp. Leverages the DOM to run tests similarly to in-browser test tools like Watir or Selenium without the associated performance hit and browser dependency. Best for web apps that do NOT utilize Javascript; apps using Javascript in-browser tools may be more appropriate.
CubicTest - An open source graphical Eclipse plug-in for writing functional web tests in Selenium and Watir. Makes web tests faster and easier to write, and provides abstractions to make tests more robust and reusable. Tests are stored in XML, directly mapped from the CubicTest domain model to XML via XStream. Tests can at any time be exported to Selenium Core tables (a popular test format) or Watir test cases. Supports recording; maven.
Selenium Grid - An open source web functional testing tool that can transparently distribute your tests on multiple machines to enable running tests in parallel, cutting down the time required for running in-browser test suites. This enables speed-up of in-browser web testing. Selenium tests interact with a 'Selenium Hub' instead of Selenium Remote Control. The Hub allocates Selenium Remote Controls to each test. The Hub is also in charge of routing the Selenium requests from the tests to the appropriate Remote Control as well as keeping track of testing sessions. Requires Java 5+ JDK, Ant 1.7.x
Mechanize - Open source; Ruby library for automating interaction with websites; automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Also keeps track of the sites visited. It is a Ruby version of Andy Lester's Perl 'Mechanize' Note: does not handle javascript.
Automation Anywhere - Tool from Tethys Solutions using 'SMART' Automation Technology offers over 180+ powerful actions for web automation. Works with any website, including complex websites using java, javascript, AJAX, Flash or iFrames. Agent-less remote deployment allows automated task to be run over various machines on the network. An advanced Web Recorder ensures accurate re-runs taking into account website changes. Also includes an editor with Point & Click wizards to automate tasks in minutes. includes link checking.
StoryTestIQ - StoryTestIQ (STIQ) is a test framework used to create Automated Acceptance Tests. It's a mashup of Selenium and FitNesse: its "wiki-ized" Selenium with widgets and features that make it easier to write and organize Selenium tests.
Web2Test - Automated test tool from itCampus Software for testing of web-based applications and portals. Runs under Windows and Linux and supports Firefox, Internet Explorer, Mozilla and Seamonkey. Provides a scripting interface in Jython and Groovy. Test scripts are browser and platform independent; supports data driven and distributed testing.
Watij - Web Application Testing in Java, an open source pure Java API. Based on the simplicity of the Watir open source web test framework and enhanced by the capabilities of Java; automates functional testing of web apps through a real browser. Provides a BeanShell Desktop console; For MS IE on Windows.
TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for functional testing as well as performance and scalability testing. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.
AutoMate - Automation platform from Network Automation, includes capability to simulates GUI activity via the browser.Inc with robust automated testing capabilities. Capabilities include support for HTTPS; Microsoft Excel Integration; a test run Event Database, native Terminal Emulation support. Tasks can be developed via drag-and-drop without writing code. Runs on Windows platforms.
Automation Anywhere - Functional test automation tool from Tethys Solutions, LLC, includes web test automation capabilities - includes a web recorder that understands web controls; web page data extraction capabilities. For Win platforms
Gomez RealityCheck XF - Provides functional testing for Web 2.0 applications; a solution for cross-browser, functional QA testing of traditional, Ajax-enabled, and other Rich Internet Applications. Enables easy creation and loading of scripts of business transactions recorded using Selenium.
iMacros for Firefox - Free Firefox add-on to record and automate web interactions. Can use variables inside the macros, and import data from CSV files. Includes user agent switcher, PDF download and Flash, ad and image blocking functions. The recorded macros can be combined and controlled with Javascript, so complex tasks can be scripted. The EXTRACT command enables reading of data from a website and exporting it to CSV files. Full Unicode support and works with all languages including multi-byte languages such as Chinese. STOPWATCH command enables capturing of web page response times
Avignon Acceptance Testing System - Open source acceptance test system that allows writing of executable tests in a language that the user can define. It uses XML to define the syntax of the language but, if the user chooses to extend the language, allows the semantics of the tests to be user-defined. Includes modules for testing web applications through either IE or FireFox, and modules for testing Swing and .NET WinForm applications also..
InCisif.Net - Web test tool from InCisif Software for client side functional testing of web apps under MSIE, using C# or VB.NET. Use InCisif Assistant to record user interactions with web application. Write, edit, execute and debug using MS Visual Studio or Visual Basic and C# Express.
Sahi - Free open-source web test tool, written in java and javascript, by Narayan Raman; capabilities include an Accessor Viewer for identifying html elements for scripting, editable scripts (javascript), simple APIs, ant support for playback of suites of tests, multi threaded playback, HTTP and HTTPS support, AJAX support, logging/reports, suites can run in multiple threads thus reducing the test execution time.
Solstice Integra Suite - Automation tool from Solstice Software. Contains Solstice Integra Test Automation, which offers a set of out-of-the-box test automation and management features for testing within integration and SOA environments. Solstice Integra Test Automation can be used alone or in conjunction with one or more of pluggable Platform Libraries that are tailored to optimize validation of today's leading ESB and SOA platforms. Solstice Integra Platform Libraries include TIBCO, WebSphere, webMethods, J2EE and BEA. Includes unit testing, record-and-replay, message tracing, and simulation capabilities.
Vermont HighTest Plus - Regression-testing tool from Vermont Creative Software for browser-based applications; also tests stand-alone; direct integration into Internet Explorer. Integrated Debugger allows stepping through tests one line at a time to examine the value of variables in real-time.
WebFT - Web-centric functional testing solution from Radview, supports both established and emerging web technologies. Provides a visual environment for creating Agendas (scripts) that include test recording, editing, debugging, verification and reporting features.
Floyd - A Java library for automated testing of web applications; provides full control of standard web browsers such as Firefox and MSIE. Interaction with the browser and any loaded web pages is achieved via calls to Floyd's Java API. Has two main components: a normal browser embedded into the web application and controlled via its public interface, and an embedded servlet container/web server. Can be used with any unit test library,
Imprimatur - Free web functional testing tool by Tony Locke, written in Java as a command-line application. Tests are described in a simple XML file; along with standard GET, POST and DELETE methods, handles HTTP authentication and file uploads. Responses can be validated using regular expressions.
WET - Open source web testing tool that drives MSIE directly; from Qantom Software Pvt. Ltd. Has many features like multiple parameter based object identification for more reliable object recognition, support for XML Based Object Repository and more. Scripting in Ruby; written in Ruby.
SOASTA Concerto - A suite of visual tools for automated web functional and load testing from SOASTA, Inc. Available as services on the web. Drag and drop visual interface that also allows access to underlying message complexity. Task-specific visual editors support creation of targets, messages, test cases, and test compositions. Works with Firefox and MSIE, Win and OSX.
Regression Tester - Web test tool from Info-Pack.com allows testing of functionality of any page or form Reports are fully customizable.
Yawet - Visual web test tool from InforMatrix GmbH enables graphical creation of web app tests. Create, run and debug functional and regression tests for web applications. Can verify HTML, XML, and PDF' ability to do report generation, reusable step libraires and parameterization. Freeware; download jar file and start by double-click or with command javaw -jar yawet.jar
vTest - Web functional and regression test tool from Verisium Inc. Includes record and playback capabilities and scripting utilizing JavaScript. For Windows platforms.
SWExplorerAutomation - Low cost web tool from Webius creates an automation API for any Web application which uses HTML and DHTML and works with MSIE. The Web application becomes programmatically accessible from any .NET language. The SWExplorerAutomation API provides access to Web application controls and content. The API is generated using SWExplorerAutomation Visual Designer, which helps create programmable objects from Web page content. Features include script recording and VB/C# code generation.
LISA for Web Apps - Automated web application testing tool from iTKO, Inc. Browser-based test record and playback. Point-and-click capture and reuse of a test case against any web application using any browser type. No test coding or scripting. Supports active sessions, SSL, authentication and magic strings.
Squish for Web - Cross platform automated testing framework from Froglogic GmbH for HTML-based Web and Web 2.0/Ajax applications running in any of several browsers. Record or create/modify scripts using Tcl, Python, JavaScript. Automatic identification of GUI objects of the AUT; inspect AUT's objects, properties and methods on run-time using the Squish Spy. Can be run via a GUI front-end or via command line tools. Can execute tests in a debugger allowing setting breakpoints and stepping through test scripts.
Funkload - Web functional testing and load testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF. Functional tests are pure Python scripts using the pyUnit framework.
WebCorder - Free GUI web testing tool from Crimson Solutions, developed in VB. Designed for end users who are doing web based software testing, as a simple tool to record test scenarios, and play them back and generate log files. The user may also check for text or images on the screen or save screenshots.
Watir - 'Web Application Testing in Ruby', a free open-source tool, drives MSIE browser and checks results. Uses Ruby, a full featured object-oriented scripting language. Does not work with ActiveX plugin components, Java Applets, Macromedia Flash, or other plugin applications. Note: additional tools are available to extend some capabilities - see the Alternative Tools for Web Testing' page at the OpenQA site for more info.
FireWatir - Open source web testing tool has a similar API to Watir, though accesses the DOM by invoking JavaScript by using the JSSh XPI to telnet into the browser. While Watir works with MSIE, FireWatir is compatible with Firefox 1.5 and above. FireWatir allows Watir scripts written for IE to work with Firefox as well, usually requiring either no change or very small changes to existing scripts. It is planned for FireWatir and Watir to be merged. The wiki includes info on compatibility issues between Watir and Firewatir.
WatiN - 'Web Application Testing in .Net', a free open-source tool, drives MSIE browser and checks results. Uses C#. Automates all major HTML elements, find elements by multiple attributes, supports AJAX website testing, supports frames (cross domain) and iframes, supports popup dialogs like alert, confirm, login etc.,supports HTML dialogs (modal and modeless), and has a basic (extensible) logging mechanism Also available is a WatiN Test Recorder
Selenium - Free open-source tool, originially from Thoughtworks. Records web apps on Firefox; scripts recorded in 'Selenese' or any of 6 languages. Run against Internet Explorer, Mozilla and Firefox on Windows, Linux and Mac. For browser compatability testing and system functional testing.
PesterCat - Low cost web functional testing tool from PesterCat LLC. Features include recording and playback of HTTP web requests, XML format for saved scripts, HTTP response validations, perform backend database validations or call procedures, use variables and variable setters to make scripts dynamic, automate test scripts with Ant tasks to run scripts and generate reports. Requires Java JRE; for Linux, Mac OSX, and Windows.
IeUnit - IeUnit is an open-source simple framework to test logical behaviors of web pages, released under IBM's Common Public License. It helps users to create, organize and execute functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript for the Windows XP platform with Internet Explorer.
QEngine Web Test Studio - Automated testing tool from AdventNet for functional testing of web applications and web services. For Linux anx Windows. Records and plays in IE, Mozilla, and Firefox browsers.
AppPerfect DevSuite - Suite of testing, tuning, and monitoring products from AppPerfect Corp. that includes a web functional testing module. Records browser interaction by element instead of screen co-ordinates. Supports handling dynamic content created by JavaScript; supports ASP, JSP, HTML, cookies, SSL. For Windows and MSIE; integrates with a variety of IDE's.
JStudio SiteWalker - Test tool from Jarsch Software Studio allows capture/replay recording; fail definitions can be specified for each step of the automated workflow via JavaScript. JavaScript's Document Object Model enables full access to all document elements. Test data from any database or Excel spreadsheet can be mapped to enter values automatically into HTML form controls. HTML-based test result reports can be generated. Shareware for Windows/MSIE.
Test Complete Enterprise - Automated test tool from AutomatedQA Corp. for testing of web applicatons as well as Windows, .NET, and Java applications. Includes capabilities for automated functional, unit, regression, manual, data-driven, object-driven, distributed and HTTP load, stress and scalability testing. Requires Windows and MSIE.
actiWate - Java-based Web application testing environment from Actimind Inc. Advanced framework for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit etc. but with extended API), and Test Writing Assistant - Web browser plug-in module to assist the test writing process. Freeware.
WebInject - Open source tool in PERL for automated testing of web applications and services. Can be used to unit test any individual component with an HTTP interface (JSP, ASP, CGI, PHP, servlets, HTML forms, etc.) or it can be used to create a suite of HTTP level functional or regression tests.
jWebUnit - Open source Java framework that facilitates creation of acceptance tests for web applications. Provides a high-level API for navigating a web application combined with a set of assertions to verify the application's correctness including navigation via links, form entry and submission, validation of table contents, and other typical business web application features. Utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit.
SimpleTest - Open source unit testing framework which aims to be a complete PHP developer test solution. Includes all of the typical functions that would be expected from JUnit and the PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This includes web page navigation, cookie testing and form submission.
WinTask - Macro recorder from TaskWare, automates repetitive tasks for Web site testing (and standard Windows applications), with its HTML objects recognition. Includes capability to expand scope of macros by editing and adding loops, branching statements, etc. (300+ commands); ensure robustness of scripts with Synchronization commands. Includes a WinTask Scheduler.
Canoo WebTest - Free Java Open Source tool for automatic functional testing of web applications. XML-based test script code is editable with user's preferred XML editor; until recording capabilities are added, scripts have to be developed manually. Can group tests into a testsuite that again can be part of a bigger testsuite. Test results are reported in either plain text or XML format for later presentation via XSLT. Standard reporting XSLT stylesheets included, and can be adapted to any reporting style or requirements.
TestSmith - Functional/Regression test tool from Quality Forge. Includes an Intelligent, HTML/DOM-Aware and Object Mode Recording Engine, and a Data-Driven, Adaptable and Multi-Threaded Playback Engine. Handles Applets, Flash, Active-X controls, animated bitmaps, etc. Controls are recorded as individual objects independent of screen positions or resolution; playback window/size can be different than in capture. Special validation points, such as bitmap or text matching, can be inserted during a recording, but all recorded items are validated and logged 'on the fly'. Fuzzy matching capabilities. Editable scripts can be recorded in SmithSript language or in Java, C++ or C++/MFC.
TestAgent - Capture/playback tool for user acceptance testing from Strenuus, LLC. Key features besides capture/playback include automatically detecting and capturing standard and custom content errors. Reports information needed to troubleshoot problems. Enables 'Persistent Acceptance Testing' that activates tests each time a web application is used.
MITS.GUI - Unique test automation tool from Omsphere LLC; has an intelligent state machine engine that makes real-time decisions for navigating through the GUI portion of an application. It can test thousands of test scenarios without use of any scripts. Allows creation of completely new test scenarios without ever having performed that test before, all without changing tool, testware architecture (object names, screen names, etc), or logic associated with the engine. Testers enter test data into a spreadsheet used to populate objects that appear for the particular test scenario defined.
Badboy - Tool from Bradley Software to aid in building and testing dynamic web based applications. Combines sophisticated capture/replay ability with performance testing and regression features. Free for most uses; source code avalable.
SAMIE - Free tool designed for QA engineers - 'Simple Automated Module For Internet Explorer'. Perl module that allows a user to automate use of IE via Perl scripts; Written in ActivePerl, allowing inheritance of all Perl functionality including regular expressions, Perl dbi database access, many Perl cpan library functions. Uses IE's built in COM object which provides a reference to the DOM for each browser window or frame. Easy development and maintenance - no need to keep track of GUI maps for each window. For Windows.
PAMIE - Free open-source 'Python Automated Module For Internet Explorer' Allows control of an instance of MSIE and access to it's methods though OLE automation . Utilizes Collections, Methods, Events and Properties exposed by the DHTML Object Model.
PureTest - Free tool from Minq Software AB, includes an HTTP Recorder and Web Crawler. Create scenarios using the point and click interface. Includes a scenario debugger including single step, break points and response introspection. Supports HTTPS/SSL, dynamic Web applications, data driven scenarios, and parsing of response codes or parsing page content for expected or unexpected strings. Includes a Task API for building custom test tasks. The Web Crawler is useful for verifying consistency of a static web structure, reporting various metrics, broken links and the structure of the crawled web. Multi-platform - written in Java.
Solex - Web application testing tool built as a plug-in for the Eclipse IDE (an open, extensible IDE). Records HTTP messages by acting as a Web proxy; recorded sessions can be saved as XML and reopened later. HTTP requests and responses are fully displayed in order to inspect and customize their content. Allows the attachment of extraction or replacement rules to any HTTP message content, and assertions to responses in order to validate a scenario during its playback.
QA Wizard - Automated functional test tool for web and windows applications from Seapine Software. Advanced object binding reduces script changes when Web-based apps change. Next-generation scripting language eliminates problems created by syntax or other language errors. Includes capability for automated scripting, allowing creation of more scripts in less time. Supports unlimited set of ODBC-compatible data sources as well as MS Excel, tab/comma delimited file formats, and more. Free Demo and Test Script available. For Windows platforms.
HttpUnit - Open source Java program for accessing web sites without a browser, from SourceForge.net/Open Source Development Network, designed and implemented by Russell Gold. Ideally suited for automated unit testing of web sites when combined with a Java unit test framework such as JUnit. Emulates the relevant portions of browser behavior, including form submission, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages as text, an XML DOM, or containers of forms, tables, and links. Includes ServletUnit to test servlets without a servlet container.
iOpus Internet Macros - Macro recorder utility from iOpus Inc. automates repetitious aspects of web site testing. Records any combination of browsing, form filling, clicking, script testing and information gathering; assists user during the recording with visual feedback. Power users can manually edit a recorded macro. A command line interface allows for easy integration with other test software. Works by remote controlling the browser, thus automatically supports advanced features such as SSL, HTTP-Redirects and cookies. Can handle data input from text files, databases, or XML. Can extract web data and save as CSV file or process the data via a script. For Windows and MSIE.
MaxQ - Free open-source web functional testing tool from Tigris.org, written in Java. Works as a proxy server; includes an HTTP proxy recorder to automate test script generation, and a mechanism for playing tests back from the GUI and command line. Jython is used as the scripting language, and JUnit is used as the testing library.
TestDrive-Gold - Test tool from Original Software Group Ltd. utilizes a new approach to recording/playback of web browser scripts. It analyses the underlying intentions of the script and executes it by direct communication with web page elements. IntelliScripting logic removes the reliance on specific browser window sizes, component location and mouse movements for accurate replay, and for easier script maintenance; supports hyperlinks targeted at new instances of browser. Playback can run in background while other tasks are performed on the same machine.
Compuware TestPartner - Automated software testing tool from Compuware designed specifically to validate Windows, Java, and web-based applications. The 'TestPartner Visual Navigator' can create visual-based tests, or MS VBA can be used for customized scripting.
WebKing - Web site functional, load, and static analysis test suite from ParaSoft. Maps and tests all possible paths through a dynamic site; can enforce over 200 HTML, CSS, JavaScript, 508 compliance, WML and XHTML coding standards or customized standards. Allows creation of rules for automatic monitoring of dynamic page content. Can run load tests based on the tool's analysis of web server log files. For Windows, Linux, Solaris.
eValid - Web functional test tool from Software Research Inc. Browser-centric view simplifies test recording and editing, and replays user activity with accuracy by combining browser-internal data, timers, event counters, and direct DOM access. Can be used for AJAX-based web development methodologies. The built-in test suite management system eV.Manager controls test suite structure, runs tests automatically, records detailed logs and pass/fail statistics, and can handle hundreds of thousands of tests.
Rational Functional Tester - IBM's (formerly Rational's) automated tool for testing of Java, .NET, and web-based applications. Enables data-driven testing, choice of scripting languages and editors. For Windows and Linux.
QuickTest Pro - Functional/regression test tool from HP (formerly Mercury); keyword-driven; includes support for testing Web, Java, ERP, etc.
QA Center Test Partner - Functional/regression tool from Compuware for testing of web, Java, and other applications. Handles ActiveX, HTML, DHTML, XML, Java beans, and more.
SilkTest - Functional test tool from Borland (formerly Segue) for Web, Java or traditional client/server-based applications. Features include: test creation and customization, test planning and management, direct database access and validation, recovery system for unattended testing, and IDE for developing, editing, compiling, running, and debugging scripts, test plans, etc.
Return to top of web tools listing

Web Site Security Test Tools
Fortify 360 - Security product from Fortify Software Inc. includes vulnerability detection. Integrates static source code analysis, dynamic runtime analysis, and real-time monitoring to identify and accurately prioritize the greatest number of critical security vulnerabilities. Capabilities include the Program Trace Analyzer (PTA) that finds vulnerabilities that become apparent only while an application is running - integrate into a QA test to find vulnerabilities while a functional test is being conducted on an application.
OWASP Security Testing Tools - Variety of free and open source web security testing tools via the OWASP (Open Web Application Security Project) site. SQLiX is an SQL injection vulnerability test tool that uses multiple techniques - conditional errors injection; blind injection based on integers, strings or statements, MS-SQL verbose error messages ("taggy" method); can identify database version and gather info for MS-Access, MS-SQL, MySQL, Oracle and PostgreSQL. Other security testing tools available include WebScarab, Tiger, LAPSE, Pantera, etc.
Retina Web Security Scanner - Vulnerability scanning tool from eEye Inc. for large, complex web sites and web applications. Identifies application vulnerabilities as well as site exposure risk, ranks threat priority, produces graphical, intuitive HTML reports, and indicates site security posture by vulnerabilities and threat level. Also performs an advanced site analysis on site structure, content and configuration to identify inherent exposure to future or emerging threats.
Hailstorm - Automated web security testing tool from Cenzic Inc.; customize and configure tests based on requirements, or use pre-sets for quick assessments. Capabilities include: prioritize vulnerabilities with a quantitative score called HARM; easy-to-use wizard-based interface; 'SmartAttacks' library, updated frequently; comprehensive reports with detailed remediation information and export capabilities; administrator control over user roles, tasks and privileges. Enterprise, Pro, Core, and Starter versions.
GamaSec - Automated online website vulnerability assessment delivers proactive tests to Web Servers, Web-interfaced Systems, and Web-based Applications. Configurable scan intervals/frequency. Supports a wide variety of HTTP Authentication schemes, common HTTP protocol, BASIC, NTLM with abilities to analyze the broadest web technologies; PHP, ASP.NET, ASP, etc.
Wikto - Web server security assessment tool for windows servers, open source, from SensePost. It's three main sections are its Back-End miner, Nikto-like functionality, and Googler to obtain additional directories for use by the other two. Includes ability to export results to CSV file
Nikto Scanner - Open source web server scanner from CIRT.net which performs comprehensive tests against web servers for multiple items, including over 3300 potentially dangerous files/CGIs, versions on over 625 servers, and version specific problems on over 230 servers. Scan items and plugins are frequently updated and can be automatically updated.
HP WebInspect - WebInspect automated security assessment tool for web applications and services, from HP (Formely SPI Dynamics). Identifies known and unknown vulnerabilities, includes checks that validate proper web server configuration. Capabilities includes discovery of all XML input parameters and parameter manipulation on each XML field looking for vulnerabilities within the service itself. Requires Windows and MSIE.
AppScan - Tool suite from Rational/IBM (formerly Watchfire) automates web application security testing, produces defect analyses, and offers recommendations for fixing detected security flaws. Assessment module can be used by auditors and compliance officers to conduct comprehensive audits, and to validate compliance with security requirements.
Acunetix Web Vulnerability Scanner - Web site security testing tool from Acunetix first identifies web servers from a particular IP or IP range. It then crawls entire site, gathering information about every file it finds, and displaying website structure. After this discovery stage, it performs an automatic audit for common security issues. Applications utilizing CGI, PHP, ASP, ASP.NET can all be tested for vulnerabilities such as cross site scripting, SQL injection, CRLF injection, code execution, directory traversal and more. Requires Windows and MSIE.
Defensics Core Internet Test Suite - Security testing tool from Codenomicon Onc. searches and pre-emptively eliminates security-related flaws from the implementations that create the backbone of the modern Internet and communication between the networked devices. This includes, but is not limited to, routers, switches, firewalls, desktop and server systems, laptops, PDAs, cell phones and other mobile systems, as well as a large number of various embedded systems. Because several protocols from this category are often tightly coupled with the underlying operating system, serious flaws in handling them may easily result in total system compromises.
Perimeter Check - SecurityMetrics 'Perimeter Check' service analyzes external network devices like servers, websites, firewalls, routers, and more for security vulnerabilities which may lead to interrupted service, data theft or system destruction. Includes instructions to help immediately remedy security problems. Can automatically schedule vulnerability assessment of designated IP addresses during low traffic times.
Core Impact Pro - Security testing tool from Core Security Technologies for web apps and other systems. Uses penetration testing techniques to safely identify exposures to critical, emerging threats and trace complex attack paths
C5 Compliance Platform - Security testing apliance from SecureElements Inc. for determining security and compliance status across heterogeneous systems. Identifies security vulnerabilities, finds compliance exposures, evaluates and matches exposures with fixes, provides ready to deploy remediations and enforcement actions, and summarized or detailed views of monitored assets, information security exposures, and compliance risks.
Snort - Open source network intrusion prevention and detection system from Sourcefire Inc.; uses a rule-driven language, which combines the benefits of signature, protocol and anomaly based inspection methods. Can perform protocol analysis, content searching/matching and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more.
SecurityMetrics Appliance - Integrated software and hardware device includes Intrusion Detection and Prevention Systems and Vulnerability Assessment. Operates as a Layer 2 Bridge - no network configuration needed. Automatically downloads latest IDS attack signatures, vulnerability assessment scripts and program enhancements nightly.
Nessus - Vulnerability scanner from Tenable Network Security with high speed discovery, configuration auditing, asset profiling, sensitive data discovery and vulnerability analysis of security posture. Nessus scanners can be distributed throughout an entire enterprise, inside DMZs, and across physically separate networks. Free to download and subscriptions for vulnerability updates are free for home users; annual fee for Professional license. Updated continuously. Includes scripting language for writing custom plugins.
Security Center - Security management tool from Tenable Network Security for asset discovery, vulnerability detection, event management and compliance reporting for small and large enterprises. Includes management of vulnerability, compliance, intrusion and log data. Company also provides the Nessus Vulnerability Scanner, and Passive Vulnerability Scanner.
SARA - 'Security Auditor's Research Assistant' Unix-based security analysis tool from Advanced Research Corp. Supports the FBI/SANS Top 20 Consensus; remote self scan and API facilities; plug-in facility for third party apps; SANS/ISTS certified, updated bi-monthly; CVE standards support; based on the SATAN model. Freeware. Also available is 'Tiger Analytical Research Assistant' (TARA), an upgrade to the TAMU 'tiger' program - a set of scripts that scan a Unix system for security problems.
Qualys Free Security Scans - Several free security scan services from Qualys, Inc. including SANS/FBI Top 20 Vulnerabilities Scan, network security scan, and browser checkup tool.
GFiLANguard - Network vulnerability and port scanner, patch management and network auditing tool from GFI Software. Scans using vulnerability check databases based on OVAL and SANS Top 20, providing thousands of vulnerability assessments.
Qualys Guard - Online service that does remote network security assessments; provides proactive 'Managed Vulnerability Assessment', inside and outside the firewall,
PatchLink Scan - Stand-alone network-based scanning solution from Lumension Security that performs a comprehensive external scan of all of the devices on your network, including servers, desktop computers, laptops, routers, printers, switches and more; risk-based prioritization of identified threats; continuously updated vulnerability database for orderly remediation; comprehensive reports of scan results
Secure-Me - Automated security test scanning service from Broadbandreports.com for individual machines. Port scans, denial-of-service checks, 45 common web server vulnerability checks, web server requests-per-second benchmark, and a wide variety of other tests. Limited free or full licensed versions available.
SAINT - Security Administrator's Integrated Network Tool - Security testing tool from SAINT Corporation. An updated and enhanced version of the SATAN network security testing tool. Updated regularly; CVE compatible. Includes DoS testing, reports specify severity levels of problems. Single machine or full network scans. Also available is 'WebSAINT' self-guided scanning service, and SAINTbox scanner appliance. Runs on many UNIX flavors.
NMap Network Mapper - Free open source utility for network exploration or security auditing; designed to rapidly scan large networks or single hosts. Uses raw IP packets in novel ways to determine what hosts are available on the network, what services (ports) they are offering, what operating system (and OS version) they are running, what type of packet filters/firewalls are in use, and many other characteristics. Runs on most flavors of UNIX as well as Windows.
NetIQ Security Analyzer - Multi-platform vulnerability scanning and assessment product. Systems are analyzed on demand or at scheduled intervals. Automatic update service allows updating with latest security tests. Includes a Software Developer's Kit to allow custom security test additions. For Windows/Solaris/Linux

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)