The Long Tail of IT

In these days of recession and shrinking IT budgets, development groups are forced to do more with less. This appears to be an opportunity for growth for Open Source projects, as companies find it difficult to purchase products, or even expand use of products they currently own. Open source products are available to assist in implementations of a wide range of IT problems. And with a very low cost of entry, development groups can kick the tires, and even implement an entire project, without awaiting the decision of an enterprise architecture group or budget committee. In a recent meeting with AMR Research, analyst Dave Brown talked about “Long Tail” effects within IT, where large, mainstream projects are still getting funding, but the large number of smaller, tactical projects are left to fend for themselves. This is exactly where Open Source can make the biggest impact… the large number of tactical projects going on within a company, often “flying under the radar” of the corporate enterprise architects. And it is not just the stealth projects benefiting from Open Source. Many projects designated as “tactical” or short term solutions have the flexibility to select the most expedient solution, which often turns out to include Open Source. I discussed Long Tail effects in the creation of services here.

So, in addition to areas where Open Source has a solid beachhead, like Linux usage for corporate servers, it certainly appears that Open Source is making additional headway, filling many nooks and crannies in the IT development space.

The Long Tail of Services

The “Long Tail” moniker has been used (some would say overused) in marketing and mass media over the last couple years. Now, as different use cases for SOA evolve, the metaphor seems appropriate to explain the growing use of services outside of the core domain of SOA, business process implementation and integration. If you’re not familiar with the term “long tail”, it is used to describe how companies like Amazon, iTunes, and Netflix have monetized the huge number of low-volume products they carry in inventory. While traditional retailers make most of their money selling only the blockbusters, these companies have discovered how to make significant money from the products that sell in very small quantities. In “The Long Tail” diagram below, the green area (the head) represents the “blockbusters”, a small number of high volumes sales items. The yellow area is the tail, representing a large number of items selling at very low volume. The shaded area under the curve can be thought of as revenue. You can imagine that if the tail is long enough, its revenue is significant, and can even exceed that of the head. Chris Anderson coined the term “The Long Tail”, and wrote an excellent book on the subject, which you can read more about here.

The Long Tail

The Long Tail of Services

The long tale metaphor also aptly explains and helps justify the phenomenon of minimalist services created for singular purposes. While the SOA mantra is to create services for reuse, the rise of Web 2.0 technologies like AJAX and Flex has created a different niche for the development of services. I’ve ranted in the past that this style of service is not very service oriented.

The Long Tail of Services: Single-use and low reuse services may outnumber those designed for reuse.

In “The Long Tail of Services” diagram, the horizontal axis represents individual services, and the vertical axis shows “instances of use”. The head area represents services that are reused many times. The tail area shows services with ever decreasing reuse, culminating with services used only a single time (single-use services).

Services for Web 2.0 Applications

Building services for consumption in a user-facing application is quite different than building services to feed core business processes. If you download and use any of the AJAX-like environments, or Adobe’s Flex, and build a sample display fed from XML data, you’ll likely notice the XML structure tends to follow a simple, almost table-like structure. This is very simplistic compared to the information-rich XML structure you might expect in real-world business objects like a purchase order, invoice, or insurance policy. The latter are the domain of SOA proper, the subjects manipulated by services while moving through business processes.

At first glance, I would say the Web 2.0 style services are high-quantity, commoditized, point services fulfilling a specific, singular purpose. Don’t expect reuse from these, as they were not designed with reuse as a design criteria. But thinking further about this, I now feel that a small service like this may actually be a good candidate for reuse. The reason: its visualization is the main manifestation of its existence. In fact, a user who sees the visualization of the information is probably going to be the one to recognize that it can be used in other situations. Isn’t this a much more powerful reuse driver than a UDDI query, or worse, a WSDL file sitting on a web server or file system somewhere? Even with a nice WSDL viewer, it takes imagination to envision the data reused in another application. If this visualization aspect does turn out to be a big factor in reuse, perhaps that will drive a movement to publish services with default visualization components. It sure makes sense to be able to visualize a data structure in a display composition, rather than simply viewing an XML structure.

Other Aspects of The Long Tail of Services

The Long Tail of Services diagram is also a good context to help rationalize other aspects of service oriented design. I can envision where on the graph you would expect to apply the most governance energy (services whose intent is high reuse), and where you might expect to use light-weight REST or POX rather than SOAP as a delivery mechanism (in the long tail). Debates such as REST versus WS-* are still simmering, but The Long Tail of Services may at least provide a different dimension on the discussion, providing a context where one point of view makes more sense relative to one section of the curve.

Credits: The Long Tail picture was created by by Hay Kranen / PD, and annotated by me.