Comments
yourfanat wrote: I am using another tool for Oracle developers - dbForge Studio for Oracle. This IDE has lots of usefull features, among them: oracle designer, code competion and formatter, query builder, debugger, profiler, erxport/import, reports and many others. The latest version supports Oracle 12C. More information here.
Cloud Expo on Google News

2008 West
DIAMOND SPONSOR:
Data Direct
SOA, WOA and Cloud Computing: The New Frontier for Data Services
PLATINUM SPONSORS:
Red Hat
The Opening of Virtualization
GOLD SPONSORS:
Appsense
User Environment Management – The Third Layer of the Desktop
Cordys
Cloud Computing for Business Agility
EMC
CMIS: A Multi-Vendor Proposal for a Service-Based Content Management Interoperability Standard
Freedom OSS
Practical SOA” Max Yankelevich
Intel
Architecting an Enterprise Service Router (ESR) – A Cost-Effective Way to Scale SOA Across the Enterprise
Sensedia
Return on Assests: Bringing Visibility to your SOA Strategy
Symantec
Managing Hybrid Endpoint Environments
VMWare
Game-Changing Technology for Enterprise Clouds and Applications
Click For 2008 West
Event Webcasts

2008 West
PLATINUM SPONSORS:
Appcelerator
Get ‘Rich’ Quick: Rapid Prototyping for RIA with ZERO Server Code
Keynote Systems
Designing for and Managing Performance in the New Frontier of Rich Internet Applications
GOLD SPONSORS:
ICEsoft
How Can AJAX Improve Homeland Security?
Isomorphic
Beyond Widgets: What a RIA Platform Should Offer
Oracle
REAs: Rich Enterprise Applications
Click For 2008 Event Webcasts
SYS-CON.TV
Top Links You Must Click On


New Massively Scalable Monitoring in OpenNebula 4.4
Monitoring tens of thousands of VMs in a few minutes

This is the fifth entry in a blog post series explaining how to make the most out of your OpenNebula 4.4 cloud. In previous posts we explained the new multiple datastore system with storage load balancing, the enhanced cloud bursting to Amazon features, the multiple groups functionality and the enhanced Amazon API implementation.

OpenNebula is aimed at extremely large production clouds and it's continuously evolving to become more and more scalable. We are excited to announce that the upcoming OpenNebula 4.4 release features a brand new monitoring system that delivers more performance and scalability.

The old monitoring system divided the hosts into chunks, and for each monitoring cycle it executed a set of static probes on that chunk of hosts via ssh, collecting the monitoring information (including Virtual Machines running in the hosts). The following monitoring cycle would do the same thing with the next chunk of hosts. As a consequence, when the number of hosts was very large the system took a large amount of time to monitor all the hosts. It was possible to increase the number of monitored hosts per cycle, but beyond a certain amount the system got overloaded with ssh connections.

The new monitoring system uses a radically different approach. Instead of relying on the pull model, where the frontend initiates ssh connections to receive monitoring data, the new system uses a push model, where the hosts actively send the monitoring data to the frontend.

The monitoring process starts the same way, OpenNebula initiates -and bootstraps- the hosts via ssh, but there is a special probe that starts an agent that sits in the background and sends the monitoring data via UDP to the frontend from then onwards. On the frontend side, a C++ multithreaded collector stores all the monitor messages from the hosts and flushes them periodically to oned.

The OpenNebula core realises that it's receiving data from those hosts and therefore doesn't try to monitor it again via ssh. If the agent is stopped, OpenNebula will monitor it actively via ssh, which will restart the agent in the host.

With the new system, the monitoring cycle, i.e. how often do the hosts send monitoring data, needs to be set considering the number of hosts and the number of VMs per hosts. So the total VMs/cycle processed by oned does not hit the the database I/O limit.

Let's see this monitoring system in action. In this test we are monitoring 25K Virtual Machines and 100 hosts in cycles of 2 minutes (monitoring_push_cycle = 120).

Yes, you are seeing it correctly, this amazing new system can deliver and process monitoring data for 25,000 Virtual Machines in every 2 minutes. The first part of the figure (cycles 0 to 5) is the bootstrapping of the monitor agents in the hosts. Once all the hosts have the agents up and running sending their UDP messages we get 25K VMs monitored every cicle, i.e. every 2 mins.

This test has been carried out with a Core i5 @ 2.40 GHz with 8GB of RAM running OpenNebula and the database. A dedicated frontend and a database with high-end storage would only improve this test's performance.

I hope we have conveyed the excitement we feel with this new monitoring system.

Read more in the new OpenNebula 4.4 guides about monitoring.

About Ignacio M. Llorente
Dr. Llorente is Director of the OpenNebula Project and CEO & co-founder at C12G Labs. He is an entrepreneur and researcher in the field of cloud and distributed computing, having managed several international projects and initiatives on Cloud Computing, and authored many articles in the leading journals and proceedings books. Dr. Llorente is one of the pioneers and world's leading authorities on Cloud Computing. He has held several appointments as independent expert and consultant for the European Commission and several companies and national governments. He has given many keynotes and invited talks in the main international events in cloud computing, has served on several Groups of Experts on Cloud Computing convened by international organizations, such as the European Commission and the World Economic Forum, and has contributed to several Cloud Computing panels and roadmaps. He founded and co-chaired the Open Grid Forum Working Group on Open Cloud Computing Interface, and has participated in the main European projects in Cloud Computing. Llorente holds a Ph.D in Computer Science (UCM) and an Executive MBA (IE Business School), and is a Full Professor (Catedratico) and the Head of the Distributed Systems Architecture Group at UCM.

Enterprise Open Source Magazine Latest Stories . . .
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one lo...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, will provide an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSy...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbrai...
Learn how to solve the problem of keeping files in sync between multiple Docker containers. In his session at 16th Cloud Expo, Aaron Brongersma, Senior Infrastructure Engineer at Modulus, discussed using rsync, GlusterFS, EBS and Bit Torrent Sync. He broke down the tools that are need...
It is ironic, but perhaps not unexpected, that many organizations who want the benefits of using an Agile approach to deliver software use a waterfall approach to adopting Agile practices: they form plans, they set milestones, and they measure progress by how many teams they have engag...
For better or worse, DevOps has gone mainstream. All doubt was removed when IBM and HP threw up their respective DevOps microsites. Where are we on the hype cycle? It's hard to say for sure but there's a feeling we're heading for the "Peak of Inflated Expectations." What does this mean...
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
Click to Add our RSS Feeds to the Service of Your Choice:
Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Subscribe in NewsGator Online
myFeedster Add to My AOL Subscribe in Rojo Add 'Hugg' to Newsburst from CNET News.com Kinja Digest View Additional SYS-CON Feeds
Publish Your Article! Please send it to editorial(at)sys-con.com!

Advertise on this site! Contact advertising(at)sys-con.com! 201 802-3021




SYS-CON Featured Whitepapers
Most Read This Week
ADS BY GOOGLE