SOA Enlightenment

This month, Gartner released the 2009 "hype cycle" curve, showing SOA has finally emerged from the "trough of disillusionment". As Gartner tells us, SOA is no different than other technologies, moving from initial inception, to extreme vendor hype, to disillusionment as users feel swindled when reality falls far short of expectations. See Joe McKendrick's article, which includes the hype cycle graph. SOA is now marching forward on the "slope of enlightenment". At this stage in its lifecycle, the over-hype of past years has been resoundingly squelched, and the fundamental benefits of SOA, without the vendor exaggerations we've come to expect, have become apparent to the masses of IT groups across many industries. These IT groups are now "rolling up their sleeves", in Joe's terms, implementing projects using service oriented principles, and actually getting real value from those efforts. SOA is not saving the world in and of itself, but rather has become an essential tool for those companies seeking greater productivity. SOA makes IT more efficient, and helps focus energy on the essential purpose of IT: making the business run better, both now and in the future when inevitable business changes demand rapid change from software and systems.

Our data services project at XAware continues to gain strength. As companies build business applications using service-oriented principles, accessing data in a service-oriented manner becomes critical. Software from the XAware project addresses this need elegantly, especially when a company has complex data structures and varied formats spread across different systems. XAware is open source, and can be found at www.xaware.org.

Cloud-based Integration

There has been a lot of press over the last year about cloud computing. Web-based applications like Salesforce.com have demonstrated the utility and cost-effectiveness of cloud-based resources. Integration vendors have also begun providing products designed for the cloud. Like the few other cloud-based integration products, the XAware engine can be installed and run in a cloud computing environment. This architecture lets you avoid local infrastructure investments, and is most beneficial when you need to integrate data sources and applications involving other cloud-based or SaaS resources (not all resources are premise-based). If you are using SaaS or other internet-based resources, placing an integration component like XAware on the internet makes sense. It makes less sense if all your application components are premise-based, depending on performance requirements. This latter scenario would have multiple premise-based components contributing data to a cloud-based integration application, where the data is combined and transformed, then sent back into premise-based resources. Only a few types of integration applications can afford the performance cost of such a round-trip to/from the cloud.

If you're interested in experimenting with XAware in the cloud, you might read the new Wiki article on installing the XAware engine in Amazon's Elastic Cloud Computing (EC2) environment . This environment lets you pay Amazon for hardware and software resources as you go, providing a very cost-effective and flexible environment for many integration applications.

A recent front-page article by Jeff Feinman in SD Times, "Bottom Line: Software had better pay", warns that economic conditions mandate any funded project better begin paying back immediately.  Supply chain management and automation improvements are cited as types of projects that continue to be funded during this economic down turn.  Automation in particular is an area ripe with opportunities aligned with service-oriented architecture (SOA).  Even more attractive is the fact that many automation opportunities are focused on improving a small number of business processes.  This means that a project has a manageable scope.  Project costs  are more predictable, thus return on investment (ROI) is more definite.  Projects with predictable and immediate ROI are much more attractive than those with fuzzy costs ROI, thus are more likely to get funded when budgets are tight.

Why is automation well-aligned with SOA?  Since the early days of SOA hype, the main goal of SOA has been to achieve business agility through the organization of computing functions as interchangeable parts, called services.  A business process is implemented by orchestrating services to accomplish the goals of the process.  Services are designed to be reusable, so a particular service invocation, like "Create new customer", can participate in many different business processes.  Most importantly, new business processes can largely be orchestrated from a comprehensive library of existing services.  Traditional development cycles of 12-18 months are replaced with the creation of a new orchestration, a process that may take just a few weeks.

While most companies are years away from having a comprehensive library of services, automation projects present an attractive "delivery vehicle" to begin or continue growing the service inventory.  Automation is the processes of exploiting computer resources to augment or replace manual processes traditionally performed by humans.  Architected in a service-oriented manner, a process or orchestration environment provides a visual tool to design and manage a business process.  Activities in the process generally are implemented by services.  Candidate tools for the process layer include ActiveEndpoints , Apache ODE , and ProcessMaker. Tools in the services layer would of course include XAware for information-oriented services, and Eclipse-based tools when custom-coded services are required.

So, while times are tough and budgets are tight, development work continues in key areas.  Companies can't afford to stop reacting to market demands or improving operations.  Service-oriented implementation strategies will conitinue to play a key role in the important work currently underway.  And, if architected properly, service-oriented projects help lay the groundwork for more strategic benefits in the future.

The Essense of SOA

Anne Thomas Manes’ recent article proclaiming the death of SOA has started a firestorm across many SOA and IT-related blogs and forums. I wrote about it here. In the post, Anne refers to the severe disillusionment some feel towards the over-hyped term, “SOA”, and proposes we drop the term altogether and simply refer to the core concept as “services”.

The term “SOA” is at the same time ubiquitous and ambiguous. So much energy has gone into molding it into a marketing story that it seems no two people share the same definition.. The truth is, SOA is really just a simple evolution in IT development strategy. I believe the essence of SOA is building software components as “interchangeable parts”, an idea preceding Eli Whitney himself. It is evolutionary (not revolutionary), because we’ve tried to do this for decades, culminating in object oriented and component-based software development. SOA is simply the next stepping stone, which loosens the chains of platform and vendor lock-in, catalyzed by internet and XML-based standards. It’s still about interchangeable parts, components that can be recombined and reused to either build new systems, or rapidly change existing ones. And it’s a superior strategy even if you don’t plan to reuse or recombine, because componentized systems are easier to build, manage, troubleshoot, and support.

In the software industry, I believe we are travelling a similar path as other industries, and finding that interchangeable parts are easy to design in the small, but exponentially more difficult to design in the large. Wiper blades and radios are easily replaced in your car. But I would just love to install a new hybrid-electric engine in my beloved ’96 Jeep Cherokee. Interchangeable parts on such a grand scale are much more problematic. I’m sure there’s marketing mechanics at work here, too. GM wraps their latest electric engine in the Chevy Volt, available next year for $42,000. They don’t want to sell just an engine.

So, I think the term we use to describe the concepts behind SOA is less important than agreeing on the core, underlying essence of SOA. I believe this to be extending the idea of “interchangeable parts”. Personally, I find myself avoiding the term “SOA” more and more, perhaps in a subconscious effort to avoid the pained or befuddled look on those faces in the room. Instead, I gravitate towards the term “service” or “service oriented”. Above all, we need to understand and accept that we are not revolutionaries. We are just carrying forward what others have already set in motion. By communicating this point, we gain credibility in our conversations with business people who control budgets. The alternative is to position this concept as “the next big thing”, something business people seem to immediately distrust.

Is SOA Dead?

Burton Group analyst and SOA guru Anne Thomas Manes recently blogged that “SOA is Dead” (http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html), referring to the disillusionment and even disgust some feel towards the over-hyped term. But she was referring just to the term SOA, not the concept itself. On the contrary, service orientation has seemed to find firm roots in diverse areas such as mashups, RIA, BPM, cloud computing, and others.

I completely agree that service orientation is here to stay. But I don’t agree that the term SOA is going away any time soon. We are in the typical “trough of disillusionment” Gartner speaks about, as a huge wave of over-hype sets high expectations for a technology. Industry buys into the vision, then slowly comes to realize it is not a silver bullet. Hard work is still to be done to extract the benefits of the new technology. When so many people express disappointment in a technology, negative momentum builds, and soon a consensus develops that the new technology is a failure at best, and evil at worst. Such is the case with SOA.

To be sure, some technologies never fully emerge from the trough. Artificial Intelligence and Object Databases are two examples that never achieved wide-spread adoption after huge early stage hype. Others fare much better, like EAI and even Java. I remember the early Java days working at MCI circa 1997. Despite huge investments including the best consultants Sun had to offer, projects were massively under-performing, or even failing altogether. But the gradual maturation of the platform and supporting tools pulled Java from the trough of disillusionment to eventually make it the most popular programming language ever.

Anne concludes by saying that we need to move away from the term SOA and simply use the term “services”, since that is the core foundation of the concept. I think that’s fine for the time being. In fact, I’ve recently found myself using the term “service orientation” instead of SOA anyway. But I believe this is a temporary diversion. Eventually, market noise will settle down, and we in the software industry will finally develop a consensus on what this concept really is. When that happens, I believe, it will mark the emergence from the trough of disillusionment, and the return to calling this concept “SOA” once again, without fear of scorn.

See my related post here.