IEEE TRANSACTIONS ON SERVICES COMPUTING,
NO. 1, JANUARY-MARCH 2012
Collaborative Testing of Web Services
Hong Zhu, Member, IEEE Computer Society, and Yufeng Zhang
Abstract—Software testers are confronted with great challenges in testing Web Services (WS) especially when integrating to services owned by other vendors. They must deal with the diversity of implementation techniques used by the other services and to meet a wide range of test requirements. However, they are in lack of software artifacts, the means of control over test executions and observation on the internal behavior of the other services. An automated testing technique must be developed to be capable of testing on-the-fly nonintrusively and nondisruptively. Addressing these problems, this paper proposes a framework of collaborative testing in which test tasks are completed through the collaboration of various test services that are registered, discovered, and invoked at runtime using the ontology of software testing STOWS. The composition of test services is realized by using test brokers, which are also test services but specialized in the coordination of other test services. The ontology can be extended and updated through an ontology management service so that it can support a wide open range of test activities, methods, techniques, and types of software artifacts. The paper presents a prototype implementation of the framework in semantic WS and demonstrates the feasibility of the framework by running examples of building a testing tool as a test service, developing a service for test executions of a WS, and composing existing test services for more complicated testing tasks. Experimental evaluation of the framework has also demonstrated its scalability. Index Terms—Software engineering, software testing, distributed/internet based software engineering tools and techniques, testing tools, ontology, web services, semantic web services, service composition.
past few years have seen a rapid growth in the development of Web Services (WS) techniques and applications to an increasingly wide range of domains. However, software testers are confronted with great challenges in testing Web Services especially when integrating to services owned by other vendors. This paper is concerned with the integration testing of WS applications.
1.1 Current State of the Art The research on testing Web Services (WS) has been growing rapidly in recent years , , , . Most research efforts fall into the following classes: . Generation of test cases. Techniques have been developed to generate test cases from syntax definitions of WS in WSDL , , , , , , , , , , , , , business process and behavioral models in BPEL , , , , , , , , , ontology-based descriptions of semantics in OWL-S , , , and other formal models of WS such as finite state machines and labeled transition systems , , , grammar graphs , , and first order logic , etc. These techniques have addressed various WS specific issues, such as the robustness in dealing with invalid .
inputs and errors in invocation sequences, fault tolerance to the failures of other services that it depends on and broken communication connections, and security in the environment that is vulnerable to malicious attacks, and so on. Generation of testbed. A service often relies on other services to perform its function. However, in service unit testing and also in progressive service integration testing, the service under test needs to be separated from other services that it depends on. Techniques have been developed to generate service stubs  or mock services  to replace the other services for testing. Checking the correctness of test outputs. Research work has been reported in the literature to check the correctness of service output against...