Last week I had the opportunity to have a hallway conversation with a CTO of a large software company about how the interactions between development and operations teams are changing.  I asked his opinion on measuring the benefit of that collaboration.  He noted while they did not have direct metrics for their teams, they found that having a rating system for applications under development did effect how developers approached their projects.

Their rating system incorporated the expected operational characteristics of an application. For example, applications that must be as resilient as possible had a different set of ratings than applications with asynchronous or long running transactions.  He said that once their developers understood the rating system, operational requirements were considered more throughly during the design phase.  Additionally, they were more receptive to using existing operational knowledge as they were building the application.  The system helped operations teams because it clearly prioritized the business importance of specific operational characteristics, which enabled better proactive problem analysis and prevention.

I would also guess that his development teams work harder at resolving problems that were shown to impact an applications’ rated characteristics by his operations teams.  I would also guess that his most productive development teams were those that applied lessons learned from working on problem resolution to improve their next design effort.  (I wouldn’t have to guess if these improvements can be captured and reported in business outcome terms -- but that is an argument for another article.)  So it seems to me that technology executives in large organizations could try a similar approach to encourage, and possibly reward, collaborative dev/ops behavior.