Richard Davies wrote: The UK has a good crop of technology pioneers in cloud computing - for example ElasticHosts, FlexiScale, Flexiant, OnApp - and also some strong government initiatives such as G-Cloud.
We will have to see whether this kind of technical leadership converts into swift mass-market adoption or not.
Openadaptor is a software toolkit that may be classified as a lightweight Enterprise Application Integration (EAI) solution. It provides a configurable component framework for connecting various systems and middleware implementations. In less technical parlance, the components are akin to Lego building blocks that users can snap together to build adaptors, which themselves are the metaphorical glue or plumbing between applications. Originally written in 1997 to help a large financial organization integrate its applications with Message Oriented Middleware (MOM), while avoiding vendor lock-in in the process, Openadaptor has since joined the open source community and evolved to become one of the better recognized systems integration tools.
Through years of usage, however, several shortcomings have emerged. The initial architectural concepts of legacy Openadaptor predated the popularity of XML, and a canonical data representation called DataObjects became an integral part of the framework introducing certain performance overheads and overly strict rules as to what can, and what cannot, be exchanged between an adaptor's components. The experience of developing and supporting legacy Openadaptor has also highlighted several other shortcomings, such as the proliferation of custom code that led to the support hurdles, poorly defined and unclear customization and interface points, and the inevitable tendency toward bloat-ware. This was compounded by the fact that some of the inherent functionalities with time became superseded by industry standard equivalents (the clumsy, though effective, proprietary transaction manager would serve as an example).
Ismael,
Thanks for the feedback, it's always very useful to us. You should be able use stored procedures for updates - see the MapCallableStatementWriter (and corresponding map-stored-proc-writer.xml example, and their XML equivalents). In fact, if you can guarantee that the warehouse will always be based on Oracle it might be easiest to implement the 'upsert' funtionality in a stored procedure which the JDBC writer would invoke.
Having said that, with your comments in mind, we are looking at adding direct support for updates (and possibly 'upserts') with a view towards squeezing it into our upcoming 3.4.3 (November '08) release of openadaptor.
Also it's worth noting that the RawSQLWriter can achieve the same effect - it uses actual SQL strings which get directly executed against the db, albeit with a performance penalty (overhead of constructing the SQL manually, and not being able to compile it).
regards
Eddy Higgins, Kris Lachor (& the openadaptor dev team)
P.S. Note that www.openadaptor.org is the primary resource for openadaptor technical support. This affords the best visibility to the dev team, and other oa users.
#1
I Matos commented on 27 Oct 2008
Great article. Great tool. It seems a good fit to a new project I'm involved with.
The project will require extracting data from a ERP system to load into a data warehouse. The ERP system uses a DB2 UDB database, while the data warehouse will be based on an Oracle database. On a daily basis, it is necessary to extract all the new or updated records from selected tables on our ERP database to load on the data warehouse.
From the documentation and the samples, I found that it is very easy to INSERT data in a table, but could not find references to table UPDATES, or more specifically to the situation where an input record may be updating the corresponding record in the target table if it already exists, otherwise it will be inserted on the target table.
What would be the best approach to resolve this "UPSERT" type of problem using Openadaptor?
Eddy Higgins wrote: Ismael,
Thanks for the feedback, it's always very useful to us. You should be able use stored procedures for updates - see the MapCallableStatementWriter (and corresponding map-stored-proc-writer.xml example, and their XML equivalents). In fact, if you can guarantee that the warehouse will always be based on Oracle it might be easiest to implement the 'upsert' funtionality in a stored procedure which the JDBC writer would invoke.
Having said that, with your comments in mind, we are looking at adding direct support for updates (and possibly 'upserts') with a view towards squeezing it into our upcoming 3.4.3 (November '08) release of openadaptor.
Also it's worth noting that the RawSQLWriter can achieve the same effect - it uses actual SQL strings which get directly executed against the db, albeit with a performance penalty (overhead of constructing the SQL manually, and not being able...
I Matos wrote: Great article. Great tool. It seems a good fit to a new project I'm involved with.
The project will require extracting data from a ERP system to load into a data warehouse. The ERP system uses a DB2 UDB database, while the data warehouse will be based on an Oracle database. On a daily basis, it is necessary to extract all the new or updated records from selected tables on our ERP database to load on the data warehouse.
From the documentation and the samples, I found that it is very easy to INSERT data in a table, but could not find references to table UPDATES, or more specifically to the situation where an input record may be updating the corresponding record in the target table if it already exists, otherwise it will be inserted on the target table.
What would be the best approach to resolve this "UPSERT" type of problem using Openadaptor?
Many thanks for the help
I...
With Cloud Expo 2012 New York (10th Cloud Expo) just four months away, what better time to start introducing you in greater detail to the distinguished individuals in our incredible Speaker Faculty for the technical and strategy sessions at the conference...
We have technical and st...
AMD said late Tuesday that its chief sales officer Emilio Ghilardi had left the company and that CEO and president Rory Read is going to do his job while a replacement is sought.
AMD didn’t say why Ghilardi left but it’s assumed Read wants his own people. Read is relatively new to th...
During the lifespan of M3 (Monitis Monitor Manager) there has always been something lacking – timers.
M3 execution procedure was outlined in this previous article.
The execution mentioned in the latter was a one-time-execution, whereas server monitoring requires periodic invocati...
Red Hat is putting its bought-in Gluster scale-out NAS storage technology,
acquired in October, on the Amazon cloud.
It’s styled Red Hat Virtual Storage Appliance for Amazon Web Services and
other clouds are supposed to follow in short order.
A new episode of the screencast series is now available at the OpenNebula YouTube Channel.
This screencast demonstrates the new easily-customizable self-service portal for cloud consumers. Its aim is to offer a simplified access to shared infrastructure for non-IT end users. The scree...
C12G Labs has just announced an update release of OpenNebulaPro, the enterprise edition of the OpenNebula Toolkit. OpenNebula 3.2, released two weeks ago, brings important benefits to cloud providers with a new easily-customizable self-service portal for cloud consumers, and builders w...