BPM

As Web services technology begins to impact business process management (BPM) and application-to-application integration, a question arises: "What business and technical challenges will we face applying Web services technology to BPM and application integration?" This article presents the seven dominant principles of good Web services design. The principles are based on 15 years of software technology evolution, combined with practical experience from today's deployed Web services applications.

The technical motivation for Web services is shown in Figure 1. In the '90s, the Internet enabled a network of browsers to access a flow of content. Today, the Internet enables a network of applications to access a flow of business transactions and processes. Web services standards promise to enable that shift.

The business imperative is also clear: yesterday's technological approaches to integration have failed - they are too expensive, too difficult to implement, and too vendor-specific.

So let's address these technical and business challenges proactively by looking at a series of principles that can guide your enterprise toward a better approach to integration.

Principle 1: Plan for a Virtual Enterprise
Industry pundits bifurcate the integration space into internal (enterprise application integration, or EAI) and external (business-to-business, or B2B) application integration. In practice, this distinction is useless. The operations of corporations are becoming virtual - within an enterprise, functions like customer relationship management, sales force automation, and billing are outsourced to third parties. New insight into the operations of partners makes you more knowledgeable and efficient. The notion of a purely internal application - one that doesn't traverse firewalls or public networks - is disappearing.

The distinction between internal and external applications is hazy from a technology point of view as well. Both require process-flow management, packaged application adaptors, messaging, and security.

So, forget the pundits and think in terms of business problems, not market classifications.

Principle 2: Be Aware of the Hidden Integration Foil: Heterogeneity
The hidden foil to integration is heterogeneity, which rears its ugly head during implementation. Why? In any integration, the applications involved typically:

• Have different representations of the same data
• Have different notions of similar business processes
• Are written in multiple programming languages
• Execute on more than one operating system
• Use more than one communication medium
• Were developed by teams with varying degrees of skill
• Use proprietary integration technologies

Individually, these problems present challenging technical hurdles; combine them, and you get prematurely aged project managers.

Some might say the answer lies in the emerging platforms for e-business: .NET and J2EE. Unfortunately, as we can see from the points of heterogeneity above, these environments don't come close to addressing our technology diversity. We don't really expect Microsoft software to run on IBM's OS/390 systems, Visual Basic developers to hack the Linux kernel, or COBOL applications to be rewritten in Java. We have no choice but to embrace our diversity.

Simple awareness of the depth and breadth of heterogeneity issues is what Principle 2 is all about. The remaining principles work together to mitigate the heterogeneity problem.

Principle 3: Describe Business Processes Succinctly
When compared to most applications, which affect subsets of an enterprise or pieces of a supply chain, the impact of BPM is pervasive. It spans organizations within an enterprise, impacts scores of trading partners and consumers, and employs a vast array of technologies.

Figure 2 describes a way to think about business processes as a combination of real-world and technology models:

• The process model describes the steps that occur in the real world (e.g., the trucks that deliver goods from point A to point B, the schedules and locations of the drivers).
• Workflow models describe the technology interactions that support, interact with, or implement the real-world process model (e.g., system X sends request for purchase order to system Y).

More than any other application, the requirements for BPM should be expressed in plain English and reviewed by participants from each function involved. Process models should be described in the briefest of terms; they should be edited with the same scrutiny an editor applies to an important piece of writing. For example, do all participants in a business process agree that a feature request document should go to a customer support rep first, or should they go to a product manager? What are the exceptions? What are the expected time frames for each point in the process flow? Can steps be eliminated? Can steps be combined?

The workflow models themselves should also be expressed succinctly, although English may not be the best language for expression. Industry-standard languages and modeling tools are often better choices to express these models, but it's still critical for everyone to understand and agree that they're accurate.

Effective organizations follow Principle 3 and begin with clear, well-understood business process models.

Principle 4: Leverage Standards
The software industry's solution to the heterogeneity problem is middleware. Vendors sell middleware as a "software bus" that isolates applications from the complexity of a heterogeneous environment (see Figure 3, Vendor view). Simply develop a single interface to the middleware, and other applications can plug in to the bus, like an appliance plugs in to an electrical outlet for power.

Unfortunately, traditional middleware has failed to achieve that promise (see Figure 3, Actual view). While EAI technology is useful for system-to-system integration, it requires homogeneous technology on each end of the connection and tends to create proprietary islands of integration. Proprietary tools and custom applications add more moving parts to the ugly reality of the enterprise technology infrastructure.

But the software industry has a history of neutralizing proprietary architectures once they become unwieldy with standards. From relational databases (SQL) to networks (TCP/IP) to operating systems (Linux), standardization - industry-driven or de facto - reduces our reliance on proprietary techniques and simplifies architecture.

Web services standards (see Figure 3, Web services standards) define "middleware for middleware." While proprietary products will not go away, proprietary interfaces between them will. Standardization cleans up the heterogeneity mess not by eliminating it, but by defining standard interfaces and services. Proprietary systems that support these standards can participate in the universal bus architecture that liberates software assets.

Despite the fact that Web services standards are still evolving, they provide real value today. Most popular tools support Web services already, so developers can use the tools they already know. Since the tools use the same standards, they require less proprietary training - think of the impact SQL had on the accessibility of databases. Standards are simpler to learn, which means developers of all levels can fully participate in the Web services party.

Principle 4 is about embracing standards early and incrementally - the sooner you incorporate Web services integration methodologies into your architecture, the sooner you'll liberate assets from proprietary tools. Furthermore, you'll begin to bridge the islands of integration that have formed and mitigate the heterogeneity challenge (Principle 2: Be aware of the hidden integration foil: heterogeneity).

Principle 5: Anticipate Change with Adaptive Infrastructure
When designing the interfaces for an application integration, think a year or two ahead. Envision a system that can withstand changes in the business, in the application, and in the underlying technology infrastructure. Systems that can adapt best to change are best suited for application integration.

Why is it important to develop adaptive architectures, from a business perspective?

1. The best way to optimize integration projects is to avoid them.
2. If your interfaces adapt, subsequent applications can be integrated more easily.
3. The longer your interfaces last, the better job you have done defining your requirements (Principle 3: Describe business processes succinctly).
4. Stable interfaces imply you have accomplished a high level of granularity (Principle 7: Focus on Granularity, below), which is a natural use for BPM technology.

A recently deployed telecommunications application provides a good illustration of adaptive design. The application is an internal Operation Support System (OSS) that required integration with the Internet (via a J2EE application server) and a WAP interface. Phase one enabled Web-based access to the mainframe system using J2EE. Rather than designing an interface for the first application, the team considered the requirements of the next two integration targets as well. Phase two of the system would expose the same information to cellular customers via WAP. Phase three required the same integration, but with Microsoft's .NET. Since each system placed different demands on integration, the team decided to build an integration architecture that was self-describing (i.e., another application could query the system and learn about the interface programmatically) and extensible (i.e., via XML, new functionality could be added to the document-oriented architecture that would not break previous integrations).

While the requirements phase of the first project was increased by four weeks, the time to integrate the three systems was reduced because:

• Each development team utilized native standards-based interfaces in the tools they were already using - no training was required.
• Each project phase capitalized on Web services provided by systems on different software and hardware (heterogeneity was mitigated).
• Although new requirements were added for Phase 2, the Phase 1 system didn't break because the XML interface was extensible.
• Web services standards provided the architecture to support self-describing, extensible architectures.

By choosing techniques that allow interfaces to adapt over time, Principle 5 helps reduce costs, improves quality and ensures more efficient integration projects later.

Principle 6: Think Integration Platform, Not Integration Applications
An application is installed and used as a black box - without regard for its inner workings. A platform, by contrast, provides services to other applications so they can operate more effectively and can scale as more applications are added, load is placed on the system, and more or less resources are required.

Enterprises with hundreds of applications have unique platform needs. For example:

• The ability to distribute and balance the load across many instances of the same application
• The ability to manage many applications, each running in its own environment, as you would manage a single homogeneous system
• Security management, where each system has its own way of managing security (yet another heterogeneity dimension)
• The ability to allow small-scale integration projects to interoperate with these complex environments as well.

The general classes of platform services include security, transaction management, messaging, and the orchestration of business process flow, as depicted in Figure 4. Each class of service has emerging standards that apply to them, and a platform needs to take these issues into account.

Principle 6 orients you to think of the Web services "universal bus" as a platform that provides support for these classes of service.

Principle 7: Focus on Granularity: Think Faxes, Not Phone Calls
Good workflow design utilizes large-grain interfaces to exchange complex information. Let's say you want to buy running shoes. You could call a shoe salesperson and have a conversation, or send for a catalog and place an order. The differences in these two approaches illustrate the final principle of good business process management: proper selection of granularity.

The fine-grain approach is like a phone call between two people. Consider the following conversation, or business process flow:

• "Do you have Saucony racing flats?"
• "No."
• "How about Nikes?"
• "Yes."
• "Do you have Nike Zoom Rival D?"
• "Yes, but only in sizes 10 and 9."
• "Great, I'm a 10; do you have three pairs in stock?"
• "No, just two."
• "OK, I'll take two pairs then."
• "That'll be $149.96."
• "What kind of credit cards do you accept?"
• ... and so on

This conversation can be seen conceptually in Figure 5. A few observations about this fine-grain conversation:

• Simple data is exchanged: "shoe type," "quantity," "size," and "price."
• Lots of data is exchanged, in small chunks.
• The conversation is tightly coupled: Replies are meaningless without the context of the question. The phrase "sizes 10 and 9" tells you nothing useful on its own.
• You have to understand to whom you're talking: You have different conversations with a shoe salesperson than a waiter.
• Synchronous communication: Because the conversation is tightly coupled, you must be able to associate replies with requests.

Now for the large grain approach: I send a letter to a mail-order shoe company requesting their catalog. A few days later, you receive it by mail. Reading the catalogue tells what shoes are available, their prices, and sizes. You fill out an order form and fax it to the company with your credit card number. You assume if a shoe isn't in stock, they'll order more from a warehouse.

Things to note about the large-grain approach:

• Complex data is exchanged: structured, self-describing data is exchanged. You got an entire shoe catalog. You knew it was a catalog by reading it, not by knowing that it was a response to my request.
• Large chunks of data are exchanged infrequently.
• Loose coupling: You didn't need to directly associate the catalog and your previous request because the catalog is sufficiently self-describing to anyone who knows how to read a catalog (understands the schema).
• Asynchronous communication: Enabled by the self-describing data and loose coupling. You could not know directly that the envelope containing the catalog was a response to your request.

This is the approach of large-grain systems, the ones best suited for effective business process management. It's also the approach that matches existing manual processes that are fax- or document-based. Somewhere in an enterprise, right now, a faxed order is being entered into a billing and shipping system. Proper choice of software interface granularity can leverage these existing document based systems.

It may seem obvious that large-grain system interfaces would tend to be more stable, easier to describe, and easier to integrate - so why would any developer develop fine-grained interfaces? First, specifying high-level, abstract interfaces is not easy. Developers tend to start exposing systems by mapping existing, low-level interfaces directly to Web services, since it's the most straightforward approach. Another factor conspiring to lead us astray is today's tools, which tend to encourage this simplistic approach to creating Web services. For example, most Web services "wizards" read existing service descriptions, objects, or component models and simply map them one-to-one to fine-grain Web services.

With such a confluence of factors steering developers toward a less desirable approach, it's easy to see why integration projects tend to be expensive, late, and error-prone. However, by observing Principle 7, we can avoid these issues by proactively specifying course grain architectures that ensure more loosely coupled, highly cohesive systems.

Pulling It All Together
To pull the pieces together, let's quickly describe an effective Web services architecture for business process management:

• It addresses the needs of the virtual enterprise - internal applications as well as applications that run on external networks (Principle 1: Plan for a virtual enterprise).
• It's constructed with the full awareness of the heterogeneity challenge (Principle 2: Be aware of the hidden integration foil: heterogeneity).
• Business processes are well-documented and understood (Principle 3: Describe business processes succinctly).
• Standards adherence is central to the architecture; not an afterthought (Principle 4: Leverage standards).
• Change is anticipated; the solution is designed to be adaptive. Interfaces are designed to be stable and withstand inevitable change (Principle 5: Anticipate change with adaptive infrastructure).
• The solution employs a platform, not point-to-point applications or proprietary architectures (Principle 6: Think integration platform, not integration applications).
• Applications have large-grain interfaces. They implement loosely coupled, highly cohesive interfaces, rather than low-level, tightly coupled conversations (Principle 6: Focus on granularity: think faxes, not phone calls).

Keep these principles in mind as you implement integration architectures, and you'll save time and money and derive more value from the integration of your enterprise.