While all the above vendors offer (more or less) the features listed in Table 1, since we're especially zealous about standardization and vendor independence, we decided on JDO, a strategic decision. Left with choosing between entity beans and Java Data Objects (it was about a year ago that we made this decision), we went again for the second choice. Entity beans seemed rather too bulky and heavyweight for our purposes, obscuring the simple Java model while providing (unnecessarily in our case) factory-wrapped network interfaces and limited inheritance capabilities. We felt that JDO provided a simpler, cleaner API with pure OO syntax. Still another advantage of JDO lies in its relative lightness. (Note that by saying that we're certainly not trying to slight EJB entity beans. Indeed, the initial doubts about their true usefulness seem to have died out, especially after the improvements introduced in the EJB 2.0 spec.)

Short Overview of JDO
In the remainder of this article we'll share the experience gained while building an O/R persistency framework on top of JDO, designed and implemented in a large organization. Let's start by briefly summarizing the main features of JDO.

The JDO specification provides a flexible method for handling persistent objects. It originates from JSR-000012, approved in July 1999 - those not yet familiar with the spec should consult the "Resources" section of this article for the appropriate URL - and was designed to integrate with existing frameworks (most notably J2EE), offering more complete transactional facilities. Indeed, session beans can directly manipulate JDO persistent instances, while entity beans can delegate business logic and persistence management to JDO classes. In our project we took the first approach.

Working with JDO, classes that can be made persistent implement the PersistenceCapable interface, but the class author need not explicitly declare this in the code. All a developer needs to do is to provide a companion XML file, which contains metadata about the class (e.g., the names of the fields that are to be made persistent). This XML metadata is used by a "class enhancer" to generate a persistence-capable copy of the original class. (JDO ensures binary compatibility between persistence-capable classes, even if generated by different vendors' enhancers.)

As a result, all the details of the class's actual persistence management happen behind the scenes. Each JDO instance of a persistent class has its own unique identity in the datastore and hence can be shared between different applications concurrently. When a reference is followed from one persistent class to another, the JDO implementation transparently instantiates it as necessary into the JVM. When fields of a persistent object are accessed or modified, the JDO implementation transparently marks them for change in the datastore. Almost all user-defined classes can be made persistent. While this excludes most system classes (like those in java.io or java.net), some are handled specially by a JDO implementation. For instance, the java.util.Collection interface is supported, as are immutable classes like Integer, Float, String, and Date - in addition to arrays.

The PersistenceManager is the primary interface for JDO-aware applications. It enables users to store and fetch persistent objects from the datastore and allows users to perform synchronized transactions and queries on those objects. JDO uses a neutral query syntax similar to OQL (Object Query Language).

Pros and Cons of JDO
The JDO specification has honestly won its territory, being a compact, smart, and truly pure object-oriented persistency specification. However, being a newcomer also means being young, suffering from childhood fallacies, and, sadly enough, having older enemies.

JDO touts as an advantage its back-end and datastore neutrality. This is indeed a virtue of the specification. But on the other hand it entirely ignores various important O/R issues such as object locking paradigm and O/R mapping instructions (e.g., schema structure, referential integrity, constraints, and so on). That is, JDO can be categorized as a general object persistence framework, with no special interest in being "baptized" as a full-blown O/R persistence specification. Furnishing no O/R mapping guidelines (let alone standards) and leaving this important role to the different product vendors makes the application developer's life somewhat confusing. For instance, Kodo (www.solarmetric.com), ObjectFusion (www.prismtech.com), and IntelliBO (www.signsoft.com) don't use database-referential constraints. But other vendors may choose to do so. And while IntelliBO lets you define code mappings (translation of pre-defined values into more compact database representations), it requires you to use proprietary XML tags to do it.

Aside from its current "O/R uncertainties," there's still some uncertainty surrounding JDO. Sun, by not making a very good job of telling us when to use EJB entity beans and when to use Java Data Objects, has added to the overall confusion. Furthermore, the current JDO vendors tend to be "niche vendors" in the Java industry. A major step toward the acceptance of JDO would happen when the "big guns" like IBM, Oracle, and BEA begin to follow suit and join them - something that's not yet happened. Finally, JDO has also managed to win itself some sour enemies. The infamous duel on TheServerSide.com between JDO specification lead Craig Russel and Thought Inc. CTO Ward Mullins is just one example (see "Resources").

Balancing the pros and cons of JDO, we believe that in spite of its current shortcomings JDO will mature into a pervasive first-class object persistency solution. An up-to-date portrait of the JDO market can be found on the JDO official site (see "Resources").

A Simple Approach to Working with JDO
As can be seen in our UML class diagram (see Figure 1), the superclass of the entities in our example is LifeForm. Man, Dog, and Flea are all LifeForms, where Dog also references a collection of Fleas and holds a reference to his best friend, Man. Name and Age are aggregated types in a LifeForm where an Age is always positive and a Name consists entirely of English letters. Note that Name and Age are "primitive wrappers" (of a String and Integer, respectively) supplying us with better abstraction, type safety, and basic validation in the type itself (validated during construction).

The Java code pertaining to the UML specification is presented in Listing 1. Examining the syntax of class Age, a question arises: Is the use of an Integer necessary? Can't we simply use an int? Using the latter would, curiously enough, make it impossible to create an ageless life form, since there's no Java null value for int, thus no datastore NULL value for age. Of course, you can allocate a prespecified int value representing null (e.g., -1) but this would make writing your queries a tad peculiar.

Writing the Java code, our next step is to run the vendor's JDO enhancer tool on the classes. Using the XML metadata file supplied in Listing 2, the enhancer will make these classes PersistenceCapable by applying direct bytecode manipulation. Some enhancers will automatically generate DD SQL statements (or directly create the database schema given a JDBC connection) as part of the enhancing process, while other vendors will politely ask you to run their schema-building tool. Notwithstanding, some vendors let you capture a preexisting schema and declare the O/R mappings yourself.

Robust OO models should follow the real world. Thus, when our dog passes away, its fleas should gracefully pass along with it. The dog's best friend, however, won't (we hope). In JDO terms (see Listing 2) Dog and Man are First Class Objects (FCOs) - i.e., they possess an independent life cycle. On the other hand, Fleas are embedded in Dog as Second Class Objects (SCOs). SCOs have no JDO identity of their own and are stored/deleted from the datastore as part of their owning FCO. To complete the picture, note that Age and Name are embedded as SCOs in LifeForm.

Finally, some business logic code using our data objects can be found in Listing 3. Observe that in deleteDog(Dog) it's enough to explicitly delete the Dog object. Its fleas are implicitly deleted from the datastore along with its Name and Age objects.

Performance Analysis (Round 1)
Albeit simple, the naive implementation results in poor runtime performance. The database tables generated by a typical schema tool are depicted in Figure 2. The exact table and field names are, not surprisingly, vendor-proprietary as are the referential constraints imposed on the tables. Nevertheless, they would follow the general structure presented here. Looking at Figure 2 we can see that fetching a Dog object results in a minimum of four fetches and three joins (assuming no fleas, no friend, but proper identification). Fetching a LifeForm translates to three fetches and two joins. This is highly inefficient.

You're probably curious why Name and Age aren't represented as columns in LifeFormT, even though they've been declared as "embedded" SCOs. Well, in order to be consistent with the JDO specification, an O/R vendor must make the life cycle of Name and Age dependent on their owner, but there are no restrictions regarding the actual O/R mapping. All the vendors we've tested have failed to exploit the "embed" hint into a smart datastore embedding.

It should be clear that the issue that causes a problem isn't the creation of the NameT and AgeT tables. Since a vendor's knowledge of the application semantics is limited to that defined in the metadata XML file, it would rightfully choose to generate the NameT and AgeT tables. Indeed, Name and Age could be used as an FCO or as elements of Collection in another part of the application. It's the lack of O/R embedding capabilities in cases analogous to LifeFormT that causes the problem.

Introducing Persistency Framework Based on JDO
Looking back at the naive approach above, you can spot a three-layered persistency architecture consisting of the JDO enhanced classes, the JDO implementation, and the datastore

Our JDO-based persistency framework introduces two additional layers, namely the interface layer and the wrapper layer, thereby attaining the following added-value goals:

Figure 3 shows the architecture of our framework. A vertical view of the framework reveals two major components: data entities (generated from the repository) and persistency management (an adapter, delegating the application persistency instructions to the concrete JDO implementation - e.g., persistence by reachability, JDOQL query engine, integration with EJB container, and so on). The horizontal view unveils the framework's layers: interface layer, wrapper layer, JDO layer, JDO implementation, and datastore.

We won't go any further into the persistency management component, since it's mainly just an adapter. Readers eager to learn about persistency features are wholeheartedly directed to the JDO specification.

Figure 4 presents the life form example, refactored to fit our framework. Relations connecting different layers are, as you can see, framework driven, while relations within a layer are OO-model driven. In fact, each layer encapsulates a "replica" of the life form model, using its own abstraction level to do so. Note that whereas Dog, Flea, and Man are explicitly represented throughout the layers, Name and Age are missing from the JDO layer. The reason for this will be made clear later.

Interface Layer
It's hard to overstate the importance of using a metadata repository within any large J2EE project. Our project uses one ("home made") in which we define our data entities, their fields, properties, and much more. The metadata repository is the source for our active code-generation tools, which generate our entities' interfaces and implementations and also much of the framework's generic code.

The interface layer (see Listing 4) provides a façade toward the business logic, allowing it to work in terms of abstract interfaces (Goal A). The clear separation between interfaces and implementing classes is not just a good programming practice, but also allows for a more loosely coupled compilation model. Moreover, as our data entities can implement multiple interfaces, we're able to provide different business logic modules with personalized local views of the data and enable the creation of generic mechanisms/algorithms operating on interfaces. For instance, one could declare Dog and Man to implement a Trainable interface. Doing that, it's possible to write a generic algorithm working in terms of trainable life forms.

Wrapper Layer
The wrapper layer is the middleware between the abstract interfaces presented to the business logic and the persisted objects actually stored in the database. It provides the concrete implementation of the interfaces defined in the interface layer through a symbiotic relationship with the JDO-layer classes (the enhanced classes), which act as the actual value holders. The wrapper layer's chief responsibilities focus on attaining the remaining goals, namely:

Wrapper objects make use of OO mechanisms such as reflection, dynamic invocation, dynamic proxy, lazy wrappings, and more. Their concrete code, being somewhat intricate (yet generic and actively generated), is not listed.

Object Embedding
In Figure 5 (a subdiagram of Figure 4) you can see the symbiotic relationship between the wrapper instances and the JDO-layer instances. The business model views a Man instance as an entity composed of two wrapper fields, Name and Age, that respectively reference a String object and an Integer object. The JDO model, being less abstract - in order to be efficiently persisted (Goal B) - has no explicit Name or Age objects. Instead, a ManJdo instance shares the same String/Integer instances with its ManImpl wrapper. The following code fragment (line numbers correspond to tag numbers in Figure 6) illustrates this sharing. The code creates a man and assigns it a name. Figure 6 illustrates the procedure, marking each element with the code line that created it. Note that, when updated, the ManImpl wrapper internally synchronizes its ManJdo accordingly:

1 public static void main(String[] args)
{
2 Man man = new ManImpl();
// implicit creation of ManJdo
3 Name name = new NameImpl("John Smith");
4 man.setName(name);
// implicit update of ManJdo
5 }

Delta Support and Change Tracking
Our project's functional requirements turn delta support (Goal C) and change tracking (Goal D) into important technical mechanisms. However, given that they're not the principal focal point of this article, we offer the following concise review. (A thorough investigation of these mechanisms would require an article of its own.)

Delta Support
Supporting remote clients that work on local copies of data entities, we let our client-side code build and transmit an "entity-delta" (the entity subgraph defined only by those fields that have actually been made dirty) to the server. The server-side business logic can now pursue "delta-driven" (i.e., "change-driven") semantics. So, for example, a typical session bean method would fetch the matching persistent entity, apply the delta, or perform some additional logic depending on the delta values, and commit (see Listing 5 for an example of this).

As Listing 5 shows, each entity interface (implementation) has a matching - actively generated - delta interface (implementation) exposing only the getter (accessor) methods. This is both type-safe and logically firm.

Our entities (in fact, the wrapper objects) support the creation and application of deltas by continuously keeping track of the changes made on them during the transaction. The delta-build and delta-apply code is entity-specific - i.e., it's actively generated along with each data entity. We chose this method over a general mechanism that uses reflection due to complexity, maintainability, and speed issues.

Change Tracking
Our server-side framework often needs to be handed the entities changed so far (created, updated, or deleted) during the transaction. It needs these in order to perform certain generic functional mechanisms. The persistency framework helps out by supplying a ChangeTracker component, which cooperates with the wrapper layer in tracking these entities (JDO provides instance callbacks that help maintain this information).

JDO and Database Layers
The JDO layer is made up of the classes that are actually persisted (enhanced), i.e., the JDO-suffixed classes (see Figure 4). JDO regards dates, strings, and Java primitive wrappers like Integer and Float as fundamental SCOs. Consequently, the typical vendor will choose to embed such fields in their owning entity's table. In our case, since the enhanced LifeFormJDO contains fields referencing a String and an Integer, the correspondent LifeFormT table directly contains the Name and Age columns. We can thus see that, due to the embeddings performed by their wrappers, the JDO layer classes - and so too the conforming database tables - are better suited to efficient O/R mappings. The database schema depicted in Figure 7 conforms to the JDO layer classes of the life form example.

Performance Analysis (Round 2)
If you compare Figure 7 with Figure 2, you can see that the table structure has improved since Round 1. It has become more compact, resulting in faster, more efficient runtime behavior. Fetching a LifeForm requires only a single fetch as opposed to three fetches plus two joins, as required by the naive implementation. When looking at the toy example, the improvements might seem minor. But the framework's smart-embedding capabilities become much more important when you're confronted with large data sets and a variety of entity and field types, as is the case with e-businesses.

Conclusion
Summing up, the JDO specification is a promising newcomer in the object persistency arena, even in light of its current shortcomings. Its pure semantics and light model seamlessly integrate within the J2EE environment, making it a viable candidate for small as well as large J2EE projects. However, on projects that involve a rich set of interfaces, classes, and types, in addition to large runtime data sets, developers should take care when building their persistency layer around a JDO product to ensure abstraction and performance.

In this article we've tried to contribute from our experience of doing just that by presenting the JDO-based framework we've built for our own project. We demonstrated how our framework attains the important added-value goals of abstraction and performance, while providing additional capabilities such as code generation, delta-oriented business logics, and more.

Resources