Anne Thomas Manes’ recent article proclaiming the death of SOA has started a firestorm across many SOA and IT-related blogs and forums. I wrote about it here. In the post, Anne refers to the severe disillusionment some feel towards the over-hyped term, “SOA”, and proposes we drop the term altogether and simply refer to the core concept as “services”.
The term “SOA” is at the same time ubiquitous and ambiguous. So much energy has gone into molding it into a marketing story that it seems no two people share the same definition.. The truth is, SOA is really just a simple evolution in IT development strategy. I believe the essence of SOA is building software components as “interchangeable parts”, an idea preceding Eli Whitney himself. It is evolutionary (not revolutionary), because we’ve tried to do this for decades, culminating in object oriented and component-based software development. SOA is simply the next stepping stone, which loosens the chains of platform and vendor lock-in, catalyzed by internet and XML-based standards. It’s still about interchangeable parts, components that can be recombined and reused to either build new systems, or rapidly change existing ones. And it’s a superior strategy even if you don’t plan to reuse or recombine, because componentized systems are easier to build, manage, troubleshoot, and support.
In the software industry, I believe we are travelling a similar path as other industries, and finding that interchangeable parts are easy to design in the small, but exponentially more difficult to design in the large. Wiper blades and radios are easily replaced in your car. But I would just love to install a new hybrid-electric engine in my beloved ’96 Jeep Cherokee. Interchangeable parts on such a grand scale are much more problematic. I’m sure there’s marketing mechanics at work here, too. GM wraps their latest electric engine in the Chevy Volt, available next year for $42,000. They don’t want to sell just an engine.
So, I think the term we use to describe the concepts behind SOA is less important than agreeing on the core, underlying essence of SOA. I believe this to be extending the idea of “interchangeable parts”. Personally, I find myself avoiding the term “SOA” more and more, perhaps in a subconscious effort to avoid the pained or befuddled look on those faces in the room. Instead, I gravitate towards the term “service” or “service oriented”. Above all, we need to understand and accept that we are not revolutionaries. We are just carrying forward what others have already set in motion. By communicating this point, we gain credibility in our conversations with business people who control budgets. The alternative is to position this concept as “the next big thing”, something business people seem to immediately distrust.
Showing posts with label services. Show all posts
Showing posts with label services. Show all posts
Wednesday, January 7, 2009
Tuesday, January 6, 2009
Is SOA Dead?
Burton Group analyst and SOA guru Anne Thomas Manes recently blogged that “SOA is Dead” (http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html), referring to the disillusionment and even disgust some feel towards the over-hyped term. But she was referring just to the term SOA, not the concept itself. On the contrary, service orientation has seemed to find firm roots in diverse areas such as mashups, RIA, BPM, cloud computing, and others.
I completely agree that service orientation is here to stay. But I don’t agree that the term SOA is going away any time soon. We are in the typical “trough of disillusionment” Gartner speaks about, as a huge wave of over-hype sets high expectations for a technology. Industry buys into the vision, then slowly comes to realize it is not a silver bullet. Hard work is still to be done to extract the benefits of the new technology. When so many people express disappointment in a technology, negative momentum builds, and soon a consensus develops that the new technology is a failure at best, and evil at worst. Such is the case with SOA.
To be sure, some technologies never fully emerge from the trough. Artificial Intelligence and Object Databases are two examples that never achieved wide-spread adoption after huge early stage hype. Others fare much better, like EAI and even Java. I remember the early Java days working at MCI circa 1997. Despite huge investments including the best consultants Sun had to offer, projects were massively under-performing, or even failing altogether. But the gradual maturation of the platform and supporting tools pulled Java from the trough of disillusionment to eventually make it the most popular programming language ever.
Anne concludes by saying that we need to move away from the term SOA and simply use the term “services”, since that is the core foundation of the concept. I think that’s fine for the time being. In fact, I’ve recently found myself using the term “service orientation” instead of SOA anyway. But I believe this is a temporary diversion. Eventually, market noise will settle down, and we in the software industry will finally develop a consensus on what this concept really is. When that happens, I believe, it will mark the emergence from the trough of disillusionment, and the return to calling this concept “SOA” once again, without fear of scorn.
See my related post here.
I completely agree that service orientation is here to stay. But I don’t agree that the term SOA is going away any time soon. We are in the typical “trough of disillusionment” Gartner speaks about, as a huge wave of over-hype sets high expectations for a technology. Industry buys into the vision, then slowly comes to realize it is not a silver bullet. Hard work is still to be done to extract the benefits of the new technology. When so many people express disappointment in a technology, negative momentum builds, and soon a consensus develops that the new technology is a failure at best, and evil at worst. Such is the case with SOA.
To be sure, some technologies never fully emerge from the trough. Artificial Intelligence and Object Databases are two examples that never achieved wide-spread adoption after huge early stage hype. Others fare much better, like EAI and even Java. I remember the early Java days working at MCI circa 1997. Despite huge investments including the best consultants Sun had to offer, projects were massively under-performing, or even failing altogether. But the gradual maturation of the platform and supporting tools pulled Java from the trough of disillusionment to eventually make it the most popular programming language ever.
Anne concludes by saying that we need to move away from the term SOA and simply use the term “services”, since that is the core foundation of the concept. I think that’s fine for the time being. In fact, I’ve recently found myself using the term “service orientation” instead of SOA anyway. But I believe this is a temporary diversion. Eventually, market noise will settle down, and we in the software industry will finally develop a consensus on what this concept really is. When that happens, I believe, it will mark the emergence from the trough of disillusionment, and the return to calling this concept “SOA” once again, without fear of scorn.
See my related post here.
Monday, June 30, 2008
Categories of Services
In previous blogs, I have discussed contract-first service design and the benefits to constructing services as information-rich, smart endpoints. In his most recent book, SOA: Principles of Service Design, Thomas Erl formalizes and extends these ideas by defining 3 categories of services based on the type of logic they encapsulate, the extent of potential reuse, and the degree of focus on a singular domain. The service categories are:
Entity Services – modeled around business entities like customer, purchase order, insurance policy, and invoice. Such a service typically includes the CRUD-like operations to Create, Read, Update, and Delete instances of the business entity. As I’ve mentioned in the past, entity services are ideal candidates for contract-first service design, with entities modeled in XML Schema, and that schema becoming the basis of the interface to create, read, update, or delete a business entity.
Task Services – services that generally model a business process in the enterprise. This type of service very often combines business logic and calls to other services, and because of its focus on a particular business domain, often has a low degree of reusability. Orchestration and choreography environments, along with environments that facilitate flow logic (XAware, for example) are common ways in which to create task services, although many are still writing code to stitch together services with business logic to expose “task” services. Task services also typically involve operations on more than one entity. When processing a purchase order, for example, a process may require validating a customer (using the “customer” entity service), generation of a unique purchase order number (using a order number generator service – a utility service), and adding the order to the provisioning system (using the purchase order entity service). A requirement to access multiple entities is an indicator that your service is a task service. Task services are often referred to by other names, among them: business process services, process services, task-centric business services, or orchestration services (when using orchestration tools).
Utility Services – services that are technology-oriented rather than business focused. Utility services typically provide common and reusable functions that cut across functional domains, such as event logging, notification, and unique number generation. Utility services are often referred to as “technology services” or “infrastructure services”.
The service category definitions are just the beginning in Erl’s book. The heart of the book discusses a palette of principles that guide a designer through the complex activity of creating services with appropriate characteristics for a particular use case and environment within an SOA. This is a book that I highly recommend, and I plan on referencing Erl’s ideas frequently in future blogs.
Friday, March 7, 2008
Urban Sprawl in SOA Land
Service development is about to explode in the enterprise, but these are not the custom-crafted, high value, reusable services you might see cruising the pristine avenue of your company’s SOA center of excellence. No, this will be massive. Instead of nicely architected brownstones with clean lines and aesthetic appeal, you’ll brush with small, hastily constructed, one-off services as far as the eye can see. Its urban sprawl in SOA land, and things are about to spiral out of control.
Where will this “urban sprawl” of services come from? The new killer app for SOA is Rich Internet Applications (RIA). Vendors and pundits are vocal about the perfect fit of RIA and SOA. Environments like Flex , Appcelerator, and many of the AJAX-based development environments like Tibco GI seem to perfectly match RIA and services.
It seems like an excellent fit on the surface, using services to populate sections of the display and to perform major operations like "process order". But management of the services is a glaring risk area. SOA generally strives to create coarse-grained services aligned with steps in a business process. You can visualize each service as an activity node on a flow graph. From this basic idea you get many of SOA's benefits: coarse grained, loosely coupled, reusable services that can be re-orchestrated into new business processes.
In the RIA-SOA scenario, left unchecked, I envision a high number of very granular services. Most of these will be tightly coupled, not very reusable, and RPC style. Maybe there is nothing wrong with this, but it is a significant divergence from the core principles of SOA.
I personally like the idea of RIA-SOA, but the trick will be to encourage and enforce SOA principles in the development of this new style of application. Otherwise, expect rampant proliferation of low-value services.
I can envision certain RIA development styles that are more “service oriented” than others. A quick example can illustrate this. Suppose an auto insurance company needs a new policy administration system. From the user perspective, an agent may enter a policy number and expect to see the auto policy in a multiple-tab view. Lets say that one tab shows the policy owner (name, address, contact info, etc), another tab shows coverage information (dollar limits for damage, medical, liability, etc.), and a third tab shows premium information.
The designer of such an application has many alternatives to supply services to power the screens. The primary characteristic differentiating the approaches is how coarse-grained the services are. For illustrative purposes, lets talk about three degrees of coarse-grain-ness: coarse-grained, medium-grained, and fine-grained. A coarse-grained approach would provide services to “get policy” and “update policy”. The RIA application would call “get policy” when the agent enters the policy number. Each tab would read and update appropriate sections of the single, shared policy structure. The agent would click an update button to invoke the “update policy” service under the covers, which sends the policy structure to a service which performs the needed updates. This strategy not only limits round-trip calls to the server, but conforms to SOA norms by encouraging development of more coarse-grained services that are likely more reusable and more loosely coupled.
A medium-grained approach would provide us with a separate service invocation for each of the tabbed panels. Three tabs leads to 6 services to read and update appropriate portions of the policy. The application would likely exhibit better perceived performance, as the user could view the first screen as soon as the first service (get policy owner) completes. While we have potentially 6 round trips instead of 2 in the coarse-grained case, the update operation may deal with less data, since we only have to update individual portions that changed. These services seem reusable and loosely coupled as well, as they are grouped around business information that likely is used by many applications.
A fine-grained approach leads to a larger number of services. Imagine you design your GUI top-down. You go along building screens and stubbing out services you will later create to populate the fields and react to user events. Your services are precisely defined to fit the specific needs of each screen. It a bit harder to imagine this style with our insurance example, but perhaps we define a service to retrieve name, another for address information, and another for contact information. Each section of each screen gets its own service. We have many more services now, but we increase the opportunity for perceived visual refresh performance. Another benefit is that the service definitions fall out immediately from the information sections of each screen, making the design effort straightforward. But the sheer number of services will cost us in terms of management overhead. And these services are so closely aligned to screen layout as to make reuse outside the RIA environment very unlikely.
From this example, we see that as we move down the scale towards more fine-grained services, we increase the opportunity for perceived visual refresh performance while we increase the number of round-trip service invocations to the server. We also increase the management overhead by introducing dramatically more design artifacts.
While the policy administration example shows tradeoffs in the service design approaches, in the context of service orientation, the characteristics are not quite so arbitrary. To be service oriented, you must strive to provide coarse-grained services that are reusable and loosely coupled. To a large extent, if you think in terms of defining the information payloads for services at a higher level, from the business analyst perspective, you will achieve better reuse and loose coupling. On the other hand, if your service interface is defined by a specific set of screen fields, you have tightly coupled your service to its consumer. When the screen changes, your service interface must change. Such a service has a singular purpose, will not be reusable in any other context, and so by definition, is not service oriented.
The message to RIA designers and architects is simple. Yes, industry needs you to combine RIA and SOA to achieve a new level of productivity and richness of user experience. But you should take up of the cause of SOA while you build your RIA. Be service oriented so your services are loosely coupled. Be service oriented so your services can be reused elsewhere. Your efforts will be a productivity multiplier within your organization, bearing fruit from your work long after your current project. Be service oriented, and in the process, you can stop the sprawl.
Where will this “urban sprawl” of services come from? The new killer app for SOA is Rich Internet Applications (RIA). Vendors and pundits are vocal about the perfect fit of RIA and SOA. Environments like Flex , Appcelerator, and many of the AJAX-based development environments like Tibco GI seem to perfectly match RIA and services.
It seems like an excellent fit on the surface, using services to populate sections of the display and to perform major operations like "process order". But management of the services is a glaring risk area. SOA generally strives to create coarse-grained services aligned with steps in a business process. You can visualize each service as an activity node on a flow graph. From this basic idea you get many of SOA's benefits: coarse grained, loosely coupled, reusable services that can be re-orchestrated into new business processes.
In the RIA-SOA scenario, left unchecked, I envision a high number of very granular services. Most of these will be tightly coupled, not very reusable, and RPC style. Maybe there is nothing wrong with this, but it is a significant divergence from the core principles of SOA.
I personally like the idea of RIA-SOA, but the trick will be to encourage and enforce SOA principles in the development of this new style of application. Otherwise, expect rampant proliferation of low-value services.
I can envision certain RIA development styles that are more “service oriented” than others. A quick example can illustrate this. Suppose an auto insurance company needs a new policy administration system. From the user perspective, an agent may enter a policy number and expect to see the auto policy in a multiple-tab view. Lets say that one tab shows the policy owner (name, address, contact info, etc), another tab shows coverage information (dollar limits for damage, medical, liability, etc.), and a third tab shows premium information.
The designer of such an application has many alternatives to supply services to power the screens. The primary characteristic differentiating the approaches is how coarse-grained the services are. For illustrative purposes, lets talk about three degrees of coarse-grain-ness: coarse-grained, medium-grained, and fine-grained. A coarse-grained approach would provide services to “get policy” and “update policy”. The RIA application would call “get policy” when the agent enters the policy number. Each tab would read and update appropriate sections of the single, shared policy structure. The agent would click an update button to invoke the “update policy” service under the covers, which sends the policy structure to a service which performs the needed updates. This strategy not only limits round-trip calls to the server, but conforms to SOA norms by encouraging development of more coarse-grained services that are likely more reusable and more loosely coupled.
A medium-grained approach would provide us with a separate service invocation for each of the tabbed panels. Three tabs leads to 6 services to read and update appropriate portions of the policy. The application would likely exhibit better perceived performance, as the user could view the first screen as soon as the first service (get policy owner) completes. While we have potentially 6 round trips instead of 2 in the coarse-grained case, the update operation may deal with less data, since we only have to update individual portions that changed. These services seem reusable and loosely coupled as well, as they are grouped around business information that likely is used by many applications.
A fine-grained approach leads to a larger number of services. Imagine you design your GUI top-down. You go along building screens and stubbing out services you will later create to populate the fields and react to user events. Your services are precisely defined to fit the specific needs of each screen. It a bit harder to imagine this style with our insurance example, but perhaps we define a service to retrieve name, another for address information, and another for contact information. Each section of each screen gets its own service. We have many more services now, but we increase the opportunity for perceived visual refresh performance. Another benefit is that the service definitions fall out immediately from the information sections of each screen, making the design effort straightforward. But the sheer number of services will cost us in terms of management overhead. And these services are so closely aligned to screen layout as to make reuse outside the RIA environment very unlikely.
From this example, we see that as we move down the scale towards more fine-grained services, we increase the opportunity for perceived visual refresh performance while we increase the number of round-trip service invocations to the server. We also increase the management overhead by introducing dramatically more design artifacts.
While the policy administration example shows tradeoffs in the service design approaches, in the context of service orientation, the characteristics are not quite so arbitrary. To be service oriented, you must strive to provide coarse-grained services that are reusable and loosely coupled. To a large extent, if you think in terms of defining the information payloads for services at a higher level, from the business analyst perspective, you will achieve better reuse and loose coupling. On the other hand, if your service interface is defined by a specific set of screen fields, you have tightly coupled your service to its consumer. When the screen changes, your service interface must change. Such a service has a singular purpose, will not be reusable in any other context, and so by definition, is not service oriented.
The message to RIA designers and architects is simple. Yes, industry needs you to combine RIA and SOA to achieve a new level of productivity and richness of user experience. But you should take up of the cause of SOA while you build your RIA. Be service oriented so your services are loosely coupled. Be service oriented so your services can be reused elsewhere. Your efforts will be a productivity multiplier within your organization, bearing fruit from your work long after your current project. Be service oriented, and in the process, you can stop the sprawl.
Tuesday, February 26, 2008
4 Generations of Services
While working at MCI in 1995, I was exposed for the first time to the benefits of service orientation. I was responsible for the order entry application for 1-800-MUSIC-NOW (which turned out to be a marketing flop), in which call center agents needed to process credit card payments for music CDs. Fortunately for me, MCI was already accepting credit card payment for long distance service. Even more fortunate was that a forward-thinking architect working in the credit-card processing group in
The long term vision of SOA is to enable rapid assembly of applications by orchestrating services into new business processes. Unlike the modest cost savings I experienced at MCI, the cost reductions achieved in a mature SOA could be immense. The traditional 12 or 18 month development cycles involving armies of analysts and developers are replaced by a business analyst drawing a flow graph of the new process, then clicking a button for deployment to enterprise-ready infrastructure. The "service oriented" approach is measured in days or weeks using a fraction of the engineering resources.
The industry is a long way from realizing the SOA vision, of course, but service orientation has certainly evolved a good deal in the past decade. To better understand this evolution, it is helpful to categorize technologies into generations. For example, programming languages have been categorized as they matured, from machine code to assembly, then FORTRAN, C/C++ and Java, and finally 4GLs like PowerBuilder, SQL, ColdFusion. Each generation achieved higher levels of abstraction and more statement power, making it ever-easier to translate process descriptions and algorithms into machine-executable form.
As service orientation has matured, a number of generations of services have emerged. These generations are defined loosely by how well they support or implement the major characteristics of SOA: standards-based, loosely coupled, coarse-grained, and business-orientation enabling an analyst to understand and manipulate it. Early generations reflect less of these core SOA values, and later generations are more comprehensively based on these values. Looking at the landscape of services over the last 5 years, I see 4 generations of services:
1st Generation Services - Simple services coded in a 3GL language like C, C++, C#, or Java, which don't use modern service standards like WS-* or REST. These services tend to tightly couple the consumer with underlying resources. Older distributed computing technologies like CORBA and DCOM fall into this category as well.
2nd Generation Services - Services that are standards-based and fairly simple, like implementing an operation to retrieve, modify, create, or delete a data set on a database. These services can often be auto-generated from other sources, such as from a Java or C# class, an EJB, or a database query. These services tend to reflect a method on an object, or expose an underlying implementation strategy like a relational table. They are easy to create, but because they are technology-oriented rather than business-oriented, they are unwieldy to use directly in a business process. Instead, they require combining with other services and logic to provide the proper level of granularity for orchestration.
3rd Generation Services - Truly "service-oriented", these services are aligned with a step in the business process. Loose coupling is achieved by explicitly defining data formats that are the payload of the service request and response, and these formats are driven by analysts that know the business process at hand, not by technologists attempting to optimize execution times and storage requirements. These services are often created by stitching together and transforming 1st and 2nd generation services to achieve a coarse-grained service that is suitable for orchestration and at the same time achieves loose coupling.
4th Generation Services - These are 3rd generation services that are institutionalized as managed, secure, governed, and reusable services. 4th generation services involves an ecosystem of SOA-aware technologies and procedures, which allow construction and management of business processes and higher level services. Given time to achieve 4th generation services, a company will maximize the benefits of SOA, enabling them create and modify business processes quickly to meet the changing demands of the business.
The precise boundaries of the generations and the metrics used to categorize a service can certainly be debated. But the concept is valuable in that it helps clarify the goal, to ultimately maximize the benefits of an SOA. These concepts also help justify implementing services in projects that have more modest goals. Maybe you know a function will be re-used across departments. Or, you anticipate that your basic service will be used by a 3rd generation service, thereby growing the foundation that will become an SOA based on 4th generation services. We don't need it all today, but we can derive concrete benefits in the short-term while maneuvering towards an even more attractive set of benefits in the future.
Subscribe to:
Posts (Atom)
