What has surprised everyone in the past few years is how challenging it has been to actually do e-business. One of the reasons why this is so is that companies have found it difficult to manage their business processes, especially when they stretch across multiple systems, software applications, companies, and countries. That's about to change.

It must change, because shareholders still expect companies to fulfill the promise of e-business. As Doug Neal of CSC's Research Services reports, "Companies are under pressure to perform better, faster; to do more with less; and to be super pleasing to customers. This means changing the way they manage their business processes, allowing them to innovate around their own strategic processes while simultaneously collaborating with partners and customers." Why Is It So Difficult?
Many companies tried to make their business processes more manageable 10 years ago, by reengineering them. At the time, reengineering typically meant designing a new process, then implementing it through a one-time systems and organizational change program. These efforts were more about redesigning processes than about making those processes easy to change and combine with those of partners. Similar problems existed with ERP and other packaged solutions that emerged later. These packages implemented best-practice processes, but did so by ingraining business processes in the software applications that supported them. These solutions had all the flexibility of wet concrete before they were installed and all the flexibility of dry concrete after installation.

Achieving process collaboration inside the enterprise has been difficult enough. Getting processes to collaborate across the networked enterprise is much harder. B2B participants may have informal designs for the processes they need to implement, but as they refine those designs they also have to change the technical implementations that support them. This may be possible in simple cases, but in more complex cases, such as advanced supply chain design, the project may never be completed. According to Intalio, a vendor of standards-based Business Process Management Systems (BPMS), "upgrading applications or adding new suppliers or business units can cause the technical integration activities to escalate out of control." Adding process tools and best practices to existing middleware approaches helps, but it would be preferable if something similar were embodied in the platforms that support applications.

In the networked enterprise, collaboration isn't restricted to any one process domain. Collaboration is now 360 degrees, going on at all points on the compass. This creates a many-to-many integration task, and existing tools and techniques simply are not up to the job. IS departments often try to develop business processes by performing bottom-up technical integration, stitching together systems components that were never intended to work together at the business level. They soon find that these projects denude their budgets, and return on investment and delivery-time scales are often unacceptable to the project sponsors. Not only that, but changing processes thereafter, developing custom variants, or optimizing processes on particular business channels is either hard or impossible.

Not all integration problems are technical. Collaboration requires sharing representations for processes that once were proprietary, and this is not an easy step to take. But if companies are under pressure to be better, faster, and cheaper, they will have to do only what they do best; whatever a company doesn't do well has to be done by someone else - hence, the growth in process outsourcing and the sourcing of external services. If commerce is to be truly collaborative, the underlying business processes must collaborate too, both within and across firms. This must be achieved at the business level and from the top down, leveraging existing systems in the enterprise. That is, collaboration must start from the business purpose, not the technical constraints.

What's Needed
What business needs is not a one-time fix for individual processes but a connected systems environment that can flex and recombine as required by changes in the market. Most companies now want more control over their own processes; more interaction between their processes and those of their partners: and some control over, and monitoring of, processes performed on their behalf by partners. Firms are also seeking to expose discrete business competencies as processes they can sell to others or through channel partners. To do all this, firms need to understand the processes that underpin their core business competencies. In short, they need a BPMS capability, not a new suite of enterprise applications.

The situation is similar to the period before the invention of the relational database management system. Business data used to be embedded in applications. As the volume of data grew and the connections between data sets in different applications became important to business, it became obvious that data should be managed outside of the application architecture. For example, managers wanted to analyze the data for business performance indicators. To achieve this, the new methodology for data management was built on a formal model called the relational data model. Today, this is commonplace. By allowing a company to manage its data apart from the applications that use it, the database management system (DBMS) supports a variety of data models and data management tasks and tools. The IT industry as we perceive it today is largely founded on the DBMS. Today's enterprise applications are primarily concerned with reading, writing, and manipulating data tables - in other words, clerical tasks. This has a profound implication - such applications are stovepipes. Business logic, data model, time, and connectivity all exist within the individual application. Creating and managing end-to-end processes has, up to now, depended upon complex middleware solutions. Not only is this expensive, it's overly complex.

New process management systems offer a potentially simpler, more cost-effective, and manageable alternative. One immediate benefit will be the ability to align processes more directly with organizational objectives. Business processes literally define the firm and represent the source of all competitive advantages and market differentiation. Business processes are complex, long-lived, unique, numerous, and constantly evolving. As the drive toward automation, process outsourcing, and collaboration continues apace, processes have the potential to overwhelm the firm. Process management systems are tools for managing that complexity.

In this climate, standard processes delivered in the form of standard applications that are also available to competitors are less and less attractive. Businesses want to shape their processes themselves, perform continuous and incremental process improvement without impediment from technology, and simultaneously exploit low-cost best-of-breed application components. Powerful new process servers will support this approach, providing a hybrid environment combining the best of component-application engineering and the best of process engineering. The era of stovepipe applications will eventually give way to the era of process manufacturing.

A Process Language
The first step is to make processes explicit by abstracting them from application software. This is hardly new. Decades ago, operating systems were created by abstracting memory management, file access, and graphical user interfaces from applications. Database management systems removed both the management of data and the management of the schema. Today, business rules are held and managed in a separate environment. Process management is the next logical step.

One of the key enabling technologies is Business Process Modeling Language (BPML), published by the Business Process Management Initiative (BPMI.org). The new language - which, like SQL, has a strong mathematical foundation - is a methodological way to represent and interact with business process (see Figure 1). In effect, the process management system abstracts processes out of the application code and by doing so provides a powerful new capability to business: the ability to discover, design, deploy, execute, operate, optimize, and analyze processes independently of the applications built on them. The applications get an end-to-end view of the business, looking down from above. They see the whole process, not just the changes in data as the process executes.

The processes such systems will support will be reliable, transactional, concurrent, and distributed. They will allow collaboration between processes designed independently by different organizations. Like other standards, the value of BPML will be perceived only by demonstrating the power of a process management system to the business.

The Shift From the BPM Layer to BPMS Capability
Many Fortune 2000 companies are deploying BPM point solutions or individual layers in the BPM stack. They find it necessary to deploy EAI (enterprise application integration). Some use workflow extensively, a few are piloting departmental process managers, all use process engineering tools (BPR), and many have employed rules- or process-based development to speed application delivery. It's a complex picture. How this mosaic will migrate toward a standard process-management environment will be a fascinating journey. It won't happen overnight, and it won't happen at all unless a solution emerges that is both as compelling as the RDBMS and SQL and as well-founded as the relational data model.
Plugging all the pieces together is taxing even the most able enterprise IT architects. The hope is that process management will soon be a capability (the BPMS) leveraged by applications, not a layer in an already complex application stack (see Figure 2). It has to be, since the drivers for BPMS are not only technological but economic. The most complex, dynamic processes tend to be those of higher business value and strategic importance. Such processes are difficult to coordinate across multiple partners, are unique to the firm, embody sources of competitive differentiation, and rely upon continual process improvement and improved decision-making. BPMS offers the chance to provide a technological solution to an age-old problem - BPR and organizational change.