Editor's Note: This article contains benchmark data developed by Oracle. Because we feel this is timely information, we've decided to run the article ahead of confirmation by the JDJ laboratories. We'll confirm these figures in first quarter 2000 and will publish an update in our Enterprise Application Server issue, scheduled for April 2000.

As Java has evolved from the language of applets and JavaBeans to that of servlets, Enterprise JavaBeans and database stored procedures, a need has developed for a scalable Java platform. No longer are Java applications run only for a single user. Companies are now building enterprise-scale production systems using server-side Java technology, and these systems need to scale to serve tens ... often tens of thousands ... of concurrent users. Although many efforts have been made to enhance the JDK, Sun Microsystems' reference JVM implementation, Oracle pursued a different strategy as it set out in 1996 to develop an enterprise-scale, server-side Java Virtual Machine from the ground up. This effort manifested itself in the recent release of Oracle JServer, a 100% Java-compatible server environment supportive of Enterprise JavaBeans, CORBA servers and database stored procedures.

Challenges of Java Scalability
A platform or application is generally said to be scalable if it provides consistent performance whether serving a handful or an abundance of concurrent users. Ideally, the user of a system will experience delays that are independent of the number of users on the system. If a bank teller must wait 100 msec to perform a transaction during off-peak periods, he or she should wait just about 100 msec during peak periods. Naturally, systems don't scale ad infinitum, but begin to show degraded performance above some critical load.

An application may be said to scale to n concurrent users when response time begins to bloat with the n + 1th user (see Figure 1). Once usage exceeds this threshold, it's preferable for system performance to degrade gracefully and linearly, rather than in an erratic, unpredictable manner.

There are a number of central challenges in building a scalable Java platform. First, the server must provide an effective mechanism to handle multiple concurrent clients. The architecture of most JVMs forces application developers to implement their own scalability by writing multithreaded Java code. A JVM that professes to be a scalable Java server may claim to provide good support for Java language threads. Two fundamental problems impair this approach: the application developer is required to build in his or her own scalability (a difficult, error-prone task), and garbage collection on heavily threaded applications can be extremely inefficient. In a simple model in which each individual user maps to a Java language-level thread, a single garbage collector deals with all garbage from all users.

The technique by which a JVM performs storage management, as well as the JVM architecture, determines the efficiency of garbage collection. The second challenge facing the Java server provider is optimizing storage management. One of Java's primary benefits to developers is that they need not expressly allocate and free memory. Automated memory management is the responsibility of the VM, and a poor garbage collection algorithm will diminish the scalability of a Java server.

The third challenge in achieving platform scalability is minimizing the memory footprint consumed by each user of the system. The server will naturally be limited by the physical resources of the configuration, so the JVM must make the most efficient use of its available memory to maximize the number of clients it can serve.

Many vendor implementations attempt to achieve scalability by spawning multiple instances of a JDK or similar JVM. This approach incurs substantial redundancy, however, contributing to a burdensome client footprint and restricting scalability.

Finally, a Java server should execute code swiftly as a means to bolster scalability. An inefficient bytecode execution will uneconomically consume precious CPU, resulting in sluggish performance and degraded scalability potential. Common approaches to accelerating execution typically involve some sort of just-in-time (JIT) or dynamic native compilation of Java bytecodes.

Oracle JServer Architecture
The challenges of concurrent garbage collection, memory management and session footprint are directly addressed by the Oracle JServer architecture.

Oracle JServer is tightly integrated with the Oracle8i database, and both share the notion of a client session. When a client initiates a session either through the SQL database or directly with JServer, it receives a single session within Oracle8i that comprises both a SQL database session and a JServer session. Each JServer session maintains its own Java state, which is not shared between sessions. Every client to JServer thereby perceives a dedicated JVM. Although many JVM implementation resources are shared between sessions (read-only bytecode information, for example), the experience of a JServer client is that of a devoted JVM. The client session, appearing to the client as a dedicated VM, can thus be regarded as a virtual virtual machine, or VVM (a model similar to that of a UNIX operation system) (see Figure 2).

Oracle JServer was designed as a server-side Java platform, and it is specifically architected to address challenges of Java scalability. A fundamental advantage of the architecture is that it relieves the Java application developer of the need to write multithreaded server code.

When developing for the JServer platform, a developer is encouraged to write the application for a single user. Scalability is achieved once the application is loaded into JServer. The virtual machine uses the multithreaded server facilities of the RDBMS to concurrently schedule Java execution, enabling thousands of clients to simultaneously access the application through independent JServer sessions. In addition to freeing the application developer to focus on his or her application code, the JServer architecture provides the related benefit of escaping the garbage collection bottlenecks that can hamper performance in multithreaded Java applications. JServer sessions, or VVMs, maintain independent memory stores and can be garbage-collected independently.

JServer thereby enjoys a performance advantage because the burden and complexity of memory management doesn't increase as the number of users increases. The memory manager always deals with the allocation and collection of objects within a single session. The simplicity of this scenario welcomes the application of sophisticated allocation and collection schemes attuned to the types and lifetimes of objects. For example, new objects are allocated in fast and cheap call memory, designed for quick allocation and access. Objects held in Java static variables are migrated to more precious and expensive session memory. Different garbage collection algorithms are applied in the various memory areas, resulting in high efficiency and low overhead.

Oracle JServer shares common read-only code and other appropriate data between its concurrent sessions. Only state variables unique to a session need to be stored privately for each VVM. Through this model JServer minimizes the memory footprint required to service its clients. Since its architecture scales comfortably to the capacity of its host machine, it never needs to spawn new JVM instances to serve additional users.

Its shared memory model can therefore be leveraged over its full user population. Memory footprints for simple programs like hello, world require as little as 35K per concurrent user on JServer. This measurement compares favorably to analogous footprints exceeding 1M on servers that are forced to spawn additional JVMs.

Oracle JServer addresses garbage collection, memory management and footprint challenges by means of its architecture. It confronts the execution performance challenge with its JServer Accelerator native compilation technology. In contrast to the JIT approach employed by many JVM implementations, the JServer Accelerator uses a WAT (way ahead of time) technique. As server code tends to be long-lived on its host machine, it's generally worthwhile to compile it into well-optimized native code. A WAT will invest more time in compilation than a JIT to produce more comprehensive optimizations.

ScaleServer Benchmark
The ScaleServer benchmark was designed to provide a fundamental measurement of Java server scalability. It specifically targets a server's ability to run multiple concurrent users, perform basic computations, and allocate and free Java memory structures. These tasks encompass the four central challenges of Java scalability discussed previously.

Description
The benchmark employs a CORBA infrastructure in which all clients to the server are CORBA clients that connect to a CORBA server running either as a standalone HotSpot Java VM or in the Oracle JServer. Each client creates its own CORBA object on the server side. In the case of Oracle JServer, each client's server object operates in an independent session. Once connected to its server object, each client executes a loop that runs a "work" method on the CORBA server object and sleeps for a period of time. (With the HotSpot Java VM, a set of worker threads is formed to execute the workload on the server side.) The method tests processing power by computing Fibonacci numbers.

To test memory management and garbage collection efficiencies, the work method instantiates, holds and drops Java objects. The workload represents a typical conversational load with multiple concurrent, stateful sessions on the server. Clients are added until the load becomes unmanageable to the server. The benchmark clients measure the round-trip time between the issuance of a request and the receipt of a response from the server. (Note that no SQL database access is done in the benchmark.) Although swift SQL access is an inherent benefit of JServer's integration with the Oracle8i database, the ScaleServer benchmark is intended to produce a pure measure of a Java server's execution scalability.

Parameters
The Java servers were run on a Sun Workgroup Enterprise 450 machine running four 250MHz Ultrasparc II processors with a 1.6GB/sec UPA interconnect and 1GB/sec PCI I/O subsystem. The E-450 machine was configured with Solaris 2.5.1, 4GB main memory and 100GB of fast hot-swap Ultra SCSI internal storage. Client threads were run on separate machines. Every request by a client caused the server object to compute the twenty-third Fibonacci number, allocate and drop 200 binary object trees of depth 5, and recycle 10 object trees from an additional persistent store of 200 such trees. After instantiating the server object, a client repeatedly issued calls to the server object, sleeping for 3 minutes after receiving each response. The benchmark continually inserted additional clients and measured server perfomance as a function of the growing load. The benchmark was run for the Oracle JServer and the Sun HotSpot performance engine. The Visigenics 3.4 ORB was used in testing the HotSpot VM. Oracle JServer CORBA support is provided by an embedded portion of the Visigenics 3.2 ORB that performs IIOP marshaling functions. The Java application program was compiled in both VMs: in JServer, using the JServer Accelerator compiler; with the JDK, using HotSpot.

Results
To compare the relative scalability of the two servers, we compare the average round-trip time for a call to the "work" method as a function of the number of clients connected simultaneously to the server. On the Enterprise E-450 class machine, Oracle JServer was able to support over a thousand concurrent clients, providing flat and highly predictable response times until the full processing power of the machine was utilized. As additional clients were added, JServer performance degraded gracefully and predictably. In contrast, Sun's HotSpot Java VM provided highly unpredictable and nonlinear response times at about 65 to 70 clients. HotSpot was unable to leverage the full power of the machine, using less than 20% of the CPUs.

To scale the test even further and evaluate Oracle JServer's ability to exploit even more sophisticated hardware architectures, the same tests were run with Oracle JServer on a Sun Enterprise E-6500 class machine with 12 Ultrasparc CPUs, 4GB of memory and 100GB of fast hot-swap Ultra SCSI internal storage. Results continued as predicted - JServer scaled to support over 5,000 concurrent users while providing very consistent and predictable response times.

Conclusions
The ScaleServer benchmark clearly demonstrates the superior scalability of Oracle JServer over that of the Sun HotSpot Java VM. While HotSpot handles about 65 to 70 concurrent users before fundamentally failing (yielding poor and unpredictable response times), JServer provides fast and highly predictable response times as loads on the system increase to a thousand and even more than five thousand concurrent users.

A grave limitation of the HotSpot VM's performance was that it was able to leverage only about 15 to 20% of the E-450's CPUs. Thus, since CPU wasn't a constraining factor, HotSpot couldn't support additional clients by using additional CPUs. In contrast, JServer consistently scaled linearly up to the capacity of the hardware, exploiting the additional CPUs that were added as the test was scaled from a four-way to a 12-processor system.

Naturally, the more processing a system does, the more it will benefit from running efficient code. While the HotSpot dynamic compiler may be effective for single-user applications, much of its benefit is unrealized in a multiuser environment. The Oracle JServer Accelerator compilation technology on the other hand produces excellent results in highly concurrent scenarios. JServer's compilation technology works very efficiently with the server's shared memory architecture to provide excellent performance under multiuser configurations.

Oracle JServer Qualifications
Oracle JServer was designed and implemented to serve large numbers of concurrent users, and it naturally incurs overhead costs for its scalable infrastructure. Its advantages become more and more apparent as the number of clients increases. Consider an analogy to a delivery company that routinely transports lots of packages, using a truck. If it needs to transport only one or two packages, a car would probably be more appropriate. Although the truck accelerates less quickly than the car, its superior capacity soon becomes critical as loads increase. The car will have to make multiple trips once the third, fourth or fifth package doesn't fit in its trunk. For heavier loads the truck is the faster and more economic solution - especially a truck with a powerful engine!

Summary and Benefits
The component-oriented, memory-safe Java language is becoming the choice of many companies looking to develop and deploy large-scale enterprise applications. As these companies begin to leverage productive server-side Java technologies such as servlets and Enterprise JavaBeans, a new breed of JVM is necessary to provide sufficient scalability. Oracle JServer is currently the only Java platform in the industry that meets all the requirements of an enterprise-class Java server. Specifically, it:

• Supports thousands of concurrent, stateful, conversational clients
• Leverages state-of-the-art memory management and garbage collection algorithms
• Minimizes memory footprint required per session to between 35 and 50K as compared to 1 to 3MB for typical Java VMs such as the JDK
• Exploits advanced hardware architectures, such as SMP and MPP hardware clusters, scaling linearly as new processors are added to the cluster
• Provides superior performance via JServer Accelerator technology

In addition, its scalable architecture offers three fundamental benefits concerning the development, deployment and management of Java applications:

1. Ease of programming: The JServer architecture relieves the Java application developer of the responsibility of writing multithreaded Java code to achieve scalability. Using Java threads is error-prone, poses potential security issues in a server environment, and incurs fundamental scaling limitations. With JServer, application developers don't need to write multithreaded Java code. Since multithreading facilities and scheduling are provided by the JServer platform, the application developer can write the application as if it were to run for a single user. JServer will scale the application to automatically serve many multiples of concurrent users.
2. Ease of manageability: Oracle JServer provides the industry's most manageable Java server platform. Some non-JServer implementations spawn new VM instances to support large volumes of users, an approach that wastes memory by introducing redundancies and simultaneously creates a need to manage multiple JVM instances. In contrast, a single JServer instance scales to the capacity of its host hardware, making efficient use of memory and requiring administration of only a single server instance.
3. Smooth upgrade path: Since JServer is able to exploit the capacity of simple or sophisticated hardware systems, it provides an efficient upgrade path as user loads increase. It works efficiently with both 32- and 64-bit operating systems across all major hardware platforms. Oracle JServer was designed as an enterprise-caliber platform to run server-side Java applications. Its architecture and design facilitates application development, maintenance and upgrade, making it the industry's most scalable and most highly productive Java platform.