WSJ Integration

We've seen a dramatic rise in the use of SOA and integration to provide better business process visibility and agility to organizations. The ease and low cost of assembling new systems together makes SOA an efficient and valuable business asset.

But the challenge remains: With so many moving parts in the SOA environment, and a multitude of heterogeneous systems to expose as services, how can businesses ensure all those crucial parts are working properly?

End-to-end SOA and integration testing provides the necessary insight to see what's happening inside the SOA environment and validate that services, messages, interfaces, and business processes are performing properly. End-to-end testing identifies elusive defects and corrects them - before the defect causes serious workflow interruptions.

To understand the full scope of end-to-end testing, businesses should be aware of the challenges of an SOA environment, why only end-to-end testing will work in this space, and the return on investment in end-to-end testing.

SOA Challenges
In SOA environments, systems and applications depend on one another to complete a business process or service (see Figure 1). But the numerous points of exchange create numerous opportunities for defects. To find and correct these defects, SOA and integration testing must address five core challenges of the SOA space:

• Defects in the SOA space are extremely difficult to diagnosis because the data in messages is buried in transport protocols that are inaccessible to the typical tester and system administrator. As a result, these defects usually aren't seen until the full system can be tested at the very end of the project, after potential problems have multiplied throughout the system and are more costly to fix.
• XML content and messages that are transmitted back and forth in an SOA environment often encapsulate the actual substance of the message, plus the function that it's supposed to perform. These messages are often written in SOAP (Simple Object Access Protocol), and may contain many customized formats and fields. An error can occur anywhere in the large number of fields, creating an enormous set of permutations and error points and making it extremely hard to have an effective testing solution around just the Web Services.
• Web Services and SOA demand very explicit and predictable inputs and outputs. This largely hinges on the accuracy of the application programming. For example, in Microsoft Excel each cell must be formatted to accept the proper data. Cells for dollar figures must accept dollar signs; cells for decimals must accept decimal points, etc. Unless this is explicitly expressed at the API level, errors in message communication can abound.
• The SOA testing model isn't just about unit testing. When businesses build an SOA or integration initiative, they're pulling and modifying data from dozens of different systems. Many systems provide only a confirmation message that a process, such as an update, has occurred. There's no guarantee that the data that came from System x, that's now in System y, is accurate or has been put in the right place. Again this opens the door for error if the update is affecting multiple systems and isn't executing correctly.
• Increased business process agility is the overwhelming advantage of moving to SOA. As SOA becomes more prevalent, SOA governance standards will become more prevalent too. However, because SOAs change frequently, it will be difficult to develop and adhere to governance standards and provide high-quality services without adopting a test-driven SOA approach.

Traditional versus SOA Testing
To overcome these challenges, most developers and quality assurance professionals have resorted to using traditional GUI testing tools. However, GUI testing is screen-based and doesn't address all of the deeply embedded moving parts that SOA and integration testing do. There are three key differences between the testing styles that make GUI-based testing ineffective for SOAs:

• Traditional testing is focused on the front-end. It involves checking aspects of the application that can be readily seen such as proper performance levels and the correct working order of all connections and interfaces. SOA and integration testing isn't focused on the GUI. It's focused behind the screen on the middle tier including application servers, enterprise services buses, legacy assets, and connection points between systems. SOA and integration testing is a headless, almost stateless, environment with very low human visibility or interaction.
• Traditional testing is more code-centric and application-specific. That means the developer writes code for a specific application and tests it.

  SOA and integration testing, on the other hand, is more assembly-centric and workflow-centric. Developers assemble components from existing applications and turn them into services, or they access components through a service repository and string them together. Developers need not only to test all the individual components, they need to test while the application is in the assembly phase and check that the workflows and business processes are properly connected.

• In traditional testing environments, the development team can test for defects in the system at the desktop level. Since GUI-based systems aren't overly dependant on one another, defects can be found more easily, contained, isolated, and repaired.

In SOA and integration testing environments, systems are distributed, highly dependent on one another, deployed on heterogeneous platforms, and often not even available. Defects in one application or message can cause a cascade of failures system-wide.

The End-to-End Answer
Traditional testing's front-end application-centric approach makes it ineffective in the SOA environment. SOA requires end-to-end testing to see deep inside the SOA, check for defects, and correct them before they interrupt critical business activities. The end-to-end SOA and integration testing market delivers four critical components that do this:

• Increased coverage. SOA and integration environments are made up of many components: Web Services, enterprise service buses, legacy assets, databases, files, and numerous transport protocols that move messages and orchestrate services.

  What's needed is a testing tool to address these components and the layers of complexity because they contain a tremendous amount of logic that falls outside the domain of traditional application testing, such as the dollar sign and decimal point example mentioned earlier. That example illustrates how deeply embedded logic can cripple a composite application that has been built from SOA or integration components. A traditional application testing tool wouldn't find these defects, but SOA and integration testing tools will.

• Test automation. SOA's assembly-based approach strongly favors the use of automation and test-driven development techniques. That means developers can automatically test at each stage of an application's development, or with every sprint release.

  This way the application is "evolved" as opposed to built and tested. The only way to constantly test is through an automated testing environment that supports agile practices and tools. Traditional testing tools don't have this automated capability.

• Process visibility. SOA testing helps QA teams and developers inspect Web Services, business process, and messages across transport protocols at many different levels. Developers can look at an actual message that moves from system A to system B, and correlate that message to a larger business process, such as making a deposit at the bank, and make sure that the larger process ultimately commits and executes as planned.

  Unlike traditional testing, SOA testing lets developers see each step of a process and the end result. This way SOA testing makes it easy to design test scenarios for complex business processes.

• Reuse. As SOA and integration environments grow larger and larger, and test case volume grows, communications between development, operations, and QA personnel can get skewed. To administer such large integration efforts and validate quality, businesses need a tool that lets developers, system administrators, and QA teams share test cases between them.

  SOA testing tools can do this by storing each test case and libraries of test suites, and allowing different application development personnel to access them. For instance, if a QA employee finds an error, he can quickly reference the specific test case and consult the developer to address the issue.

End-to-End ROI
End-to-end testing streamlines the entire process of discovery, diagnosis, and validation for SOA and integration environments in several quantifiable ways:

• Increased test coverage through rapid root cause identification. End-to-end testing provides the ability to identify and measure problems quickly, and generate specific reports and metrics related to SOA and integration quality.
• Cost savings. The loosely coupled yet highly connected applications and services in SOAs mean the impact of a single defect will be magnified throughout an organization. Finding those defects beforehand saves the cost of going back and fixing the problem after production, which can be as high as 10 times the original cost of production.
• Rework reduction. For each patch release put out, there's an initial cost of at least $65,000 to $70,000 for that patch. Multiply that by the hundreds of defects out there and the hundreds of patches that are needed and the cost adds up quickly.
• Reduced staff involvement and increased turnaround time. Automated end-to-end testing allows testing to happen at the click of a mouse, which can positively impact release frequency and quality as well as testing turnaround time. This increase in work efficiency can also decrease staffing costs.

Summary
The SOA and integration testing world is a world of "behind the screens" testing where defects in the systems are some of the most elusive to find - and most expensive to correct. One defect can cause a domino effect of poor quality across business processes and supporting systems.

The most effective way to validate SOA business processes is through a continuous end-to-end testing process that inspects each layer of the SOA and integration infrastructure as it's developed, run, and maintained. Further, the ability to see deep inside the SOA and integration environments provides an ROI that's vital to a streamlined and cost-effective business.

References
For more information about end-to-end integration and SOA testing, visit:

• www.SolsticeSoftware.com
• www.SOALink.com
• www.IntegrationConsortium.org

The Solstice Integra Suite

Solstice Integra Suite is a prime example of an end-to-end integration and SOA testing suite that shows businesses what's really happening deep inside their integration and SOA infrastructure. It provides end-to-end testing and validation by providing businesses with visibility across multiple protocols and multiple vendor architectures.

Integra works by automating the testing and troubleshooting of businesses' integration and SOA deployments before they go live, saving the time and cost of after-the-fact debugging.

By supporting industry-standard, vendor, and file database protocols, and integrating with Mercury, JUnit, and ANT, Integra limits risk and quality issues in large integration projects. Everyone involved in the project, including developers, testers, project managers, QA professionals, and offshore coordinators, uses Integra to:

• Gain visibility into message structure and content, and compare the content and path of messages
• Simulate unavailable systems
• Automate unit, functional, and regression testing
• Customize test scenarios such as parameterized input, validation of updates and replies, and multiple transport options
• Create a central shared repository of test cases

More information on Solstice Integra Suite can be found at www.SolsticeSoftware.com/Products.aspx