Before it can work well, a Web services must work, period. One analyst firm warns that unless testing plays a prominent role in development cycles, many Web services may be cast aside before they achieve their full potential and after they've already cost companies lots of cash and man hours.
A new report, "Web Services: A Nightmare For Testers," from London-based analyst firm Ovum states that unless designers build Web services with testing in mind, hard-to-find bugs may plague the reliability and quality of end products.
Gareth Johnson, an Ovum analyst and author of the report, said a worst-case scenario likely wouldn't bring down a company's entire Web infrastructure. Still, the strain of heavy use could result in unreliable performance or even render a Web service unusable, negating a company's investment.
Worse yet, when companies eventually rely on Web services to convey critical data, such as in medical emergencies, the failure of a Web service could have dire results.
"You don't really need to be an extremely technical person to understand that when software is developed by different companies in different locations, you can't assume anything about it when it's being implemented," Johnson said.
Johnson said the inherently complex nature of distributed software development makes Web services difficult to test.
"In traditional systems, with the way you integrate each component one at a time or a few at a time, you have much more control over the testing process," Johnson said. "If your system talks to other systems, it's a lot harder to do testing end-to-end."
He said a company should perform a risk analysis before it embarks on a Web services project, so that best-case and worst-case scenarios are identified long before a project reaches the testing stage.
"The process is basically telling the testing tool where the software components are, and they usually derive information from these component interfaces. Then the tools can generate tests automatically to test those components," Johnson said.
However, some vendors claim their products can adequately test Web services, when that may or may not be true.
"Web services is quite a general term," Johnson said, "and people use it in different ways. One definition is that they're programmatic interfaces and another is used to define the service being offered. So it's easy for vendors to say they can test Web services without being specific."
Johnson suggested that customers convince vendors to let them try demo versions of their products or develop a testing scenario for vendors to address. He said the best testing tool is one that can most accurately simulate a Web service's projected load.