One of the major issues that CEOs face today is the responsiveness of businesses. Some 75 percent of CEOs place high priority on finding effective ways for businesses to respond more rapidly to competitive threats. According to Jim Wagner, Senior Vice President of Marketing and Licensing for Mattel, "New market ideas come in waves, so it is essential to move quickly to get first-mover advantage, even if preliminary information isn't 100 percent."

Only 10 percent of CEOs believe that their organizations have the ability to be responsive and to react to changing market conditions. A Business Performance Management survey revealed that only 11.2 percent could keep up with the business demand to change processes. Worse, 26.6 percent reported that their IT departments have significant difficulties keeping up with business changes and 9.2 percent acknowledged that they couldn't keep up at all. A report released recently from Forrester Research Inc. revealed that while 78 percent of the CEOs surveyed say that they're happy with the job performance of their CIOs, fewer than half have confidence in IT as a contributor to business success.

Management acknowledges that we must respond to continuing changing market conditions and risks. Agility has been made a high priority across organizations, yet few CEOs rated the responses to changing conditions to external forces to be very good. They recognize the need to create adaptable business processes that allow real-time responses to their customer needs. They are also aware of the power of IT and the weakness that comes from lagging behind. The idea is to have IT become more agile and to eliminate the backlog of applications so that businesses can get what they want whenever they demand it.

Despite this awareness, IT still continues to develop in isolation from other systems. This becomes time-consuming and costly. In order to deliver aggressive solutions, IT must sooner or later face integration and connectivity challenges.

Service-oriented architecture (SOA) has been touted as the key to business agility and the face of modern IT integration.

Evolution to Make IT Responsive

The desire to make IT more responsive has been going on for decades. Early initiatives involved making monolithic architectures more flexible and structured by breaking them into more-executable subroutines. Then came the idea of business objects—the pattern for three-tier architecture that recommends partitioning application functionality into business logic, access of data, and application behavior, which could change pending on the context or request. Object technologies were mostly tightly coupled, so messaging technologies were developed to loosely couple applications. Various Enterprise Application Integration (EAI) techniques were then developed to make applications even more modular. The good news is that today, with the culmination of all these architectures, we have SOA. SOA blends the best of all these concepts and promises to make the notion of applications even more flexible. It is built on the design of objects, message processing, and services, and it really paved the way for dynamically reconfigurable architectures of the future.

Are You MOM-ified?

Messaging Oriented Middleware (MOM) products are more flexible than Remote Object Invocation because they decouple services through an Intermediary Broker. But MOM lacks a service interface and is typically limited to a single transport. Functions must be written to call the messaging middleware and dictate specifically where and how messages should be delivered. Anytime function changes, the application code needs to be modified and redeployed. This method of dependency is hard to manage, change, or even keep track of. Functions linked through MOM also have implicit dependencies on the message protocol, the data format, the structures, security, and error handling.

The Role of ESB

As integration requirements rise, organizations will seek a technology platform that provides loose coupling and simple information exchange. An agile enterprise must be able to change the way it does business by adjusting processes without undertaking costly, lengthy, and risky modifications to its information system.

Can an Enterprise Service Bus (ESB) help bring agility and responsiveness? Articles about ESB often characterize it as the backbone upon which you build your SOA. ESB has become the SOA industry buzz as the new EAI. In the November 2006 issue of iTechnology Manager, I wrote an article called "FAQ on SOA" that discussed SOA's involvement in the strategic success of IT: "An ESB must eliminate the need to deal with upgrades and load balancing between instances of specific services. It is uncertain that today's ESB delivers all these capabilities." I had my doubts whether true mediation existed. Some vendors' ESBs are more enhanced EAI; some are MOM products that are proprietary in nature.

EAI Is Not ESB

Traditional EAI products were specifically designed for application integration. EAI was developed to make diverse applications in an enterprise, including your partner systems, communicate with each other. The business objectives connect seamlessly in a reliable fashion irrespective of platform and the locations of applications.

EAI performs application-to-application mapping and binding by means of "hub and spoke" architecture (see Figure 1).

Figure 1: EAI performs application-to-application mapping and binding by means of "hub and spoke" architecture. (Click figures to enlarge.)

Hub/spoke architecture uses a centralized broker (hub) and adapters (spokes) that connect applications to the hub. The spoke connects to an application and converts application data format to a format that the hub understands and vice versa. The hub, on the other hand, brokers all messages and takes care of transforming content and translating the incoming message into a format the destination system understands. Within the hub are activities for validation, routing, and asynchronous delivery of messages. In short, EAI architecture is based upon a central operation: the hub. Most EAI products come with an administration console to monitor and track the workings of the hub.

It is often too costly and impractical to staff projects with IT professionals deeply knowledgeable in an EAI product. EAI is often proprietary, and semantic details are required. During deployment, scaling EAI architecture to accept additional duties requires extensive resources within your enterprise.

As a result of tumultuous discussion in the integration market caused by the advent of ESB, EAI vendors started creating their own versions of "bus" technology.

Figure 2: EAI vendors created their own versions of "bus" technology.

The EAI version of bus technology uses a central messaging backbone (their bus) for message propagation. Applications publish messages to the bus using adapters. The key difference between hub/spoke and bus topology is that, for the bus architecture, the integration engine that performs message transformation and routing is distributed in the application adapters. The bus architecture requires an application adapter to run on the same platform as the original applications.

Will the True ESB Please Rise?

The true ESB provides the same functionality as the EAI version of the bus. It provides connectivity and routes messages based on rules and message transformation. The difference here is that true ESB capabilities are SOA-based.

Figure 3: The true ESB provides the same functionality as the EAI version of the bus.

The EAI version of the bus also considers itself as a messaging backbone; however, the technology still depends on adapters to publish application messages. True ESB does the protocol conversion, message transformation, routing, and message delivery to and from various services and applications without the help of an adapter.

A true sense of an ESB can be summed up in one word: agility. You can add a service at a drop of a dime without having to rebuild your application systems. Think of a merger scenario in which companies want to move consolidated divisions quickly and remove redundant systems. Services decoupled from one company should be able to merge into an existing bus with minimum impact to business. In these instances, any ESB can be agile. I guess it all depends on your topology of what an ESB must have.

Figure 4: Compare EAI hubs, EAI buses, and ESB.

Connectivity

Each of these bus approaches has set out to follow the principles of SOA. SOA advocates reusing existing assets to create new services. The EAI version of the bus uses connectivity and adapters to ensure that legacy applications can be connected to the ESB. This solves many of the integration issues, but does it make it SOA?

The most common way for the System i to process messages over a transport is by using MQSeries. One of the few things to consider when selecting an ESB is how easy is it to connect to such transport. Some ESBs can easily achieve this using their Java Messaging Service (JMS) integration. Service end-points can be easily exposed on the ESB to allow clients to contact systems using a variety of transports, MQ included.

The use of these standard transports ensures that integration issues are addressed, thus reducing the cost of reuse of these services. We need to enable existing systems to be reused; the quickest method is simply exposing the existing interfaces to these systems as Web services. The payloads on these services are usually text-based and typically consist of fixed-width fields. They are normally expressed in XML to achieve interoperability, an SOA standard. Transformation may occur at times on these payloads. Don't be fooled by the early ESB adopters who chose to develop their integration using servlets that process XML documents. While these provide the benefit of XML and HTTP transport, they still lack the features required by SOA. Exposing them as Web services provides the missing feature and standardizes the protocol envelope (SOAP). Your ESB acts as a buffer in front of your Web service to protect it from issues relating to exposure to new clients.

The trick early adopters used was using XML parsers that support only certain XML name spaces. They rely on particular prefix or proprietary name spaces, which may lead to interoperability. This transformation ensures that only XML messages that are understood are received.

Flavors of ESB

I recently attended the ESB-CON III Webinar. It was difficult to sit and listen to a lot of self-promotion and corporate chess moves during these four hours. In the past months, I have been evaluating three ESBs from leading vendors that are known to integrate with the System i. This seminar gave me more insight to other vendors. Here are the common key points I got from the vendors:

• Support numerous end-points—Customers can choose from a wide variety of end-points. The ESB must easily flow transaction requests and coordinate between end-points. The end-points must participate in the transaction.
• The ESB is more than just brokering a message—An ESB should include messaging, composition, management, and routing. Quality of service is the key.
• The ESB needs to support different types of protocols and standards—An ESB must be able to broker many protocols and support all open standards, not just Web services.
• The ESB must be easily configurable even without compatibility—An ESB must be able to change through metadata, allowing easy change without impacting business users.
• The ESB must increase agility and maintainability—An ESB makes it faster to integrate in an open and more operable fashion. Business rules are part of your architecture. You can encode rules in your application, but deployment will be hard. Rules engines must reside outside the application and must be easily modified without modifying your application.
• The ESB must be reliable—Many integration projects require a high level of uptime, and ESB must guarantee transaction delivery.

IBM, BEA, Progress Software, and Oracle provided samples of their best practices and actual experiences on ESB integration. I was not entirely sold by all the hoopla presented by the vendors, but I did get new views and answers as to where ESB products are today. I discovered there are three major types of players:

• Startup ESB vendors—Cape Clear and PolarLake are vendors that entered the integration market by providing ESB solutions.
• Integration vendors—These vendors evolved from EAI markets like TIBCO, iWay, Vitria, and WebMethods. There is no "standalone" ESB solution, but more provide ESB features as part of their suites. Other vendors, such as Progress Software and Fiorano, have a history in the JMS architecture, while IONA had CORBA middleware space.
• Platform vendors—These vendors come from the application or enterprise application arena and now have integration solutions in their product line. Vendors include IBM, BEA, Oracle, Microsoft, SAP, Sun, and Software AG.

The current offerings of most ESBs have features like connection (support of Web services and SOA protocols), mediation (transformation, mapping, repository, registry), and change management (policy, SDLC, security, monitor).

Not to confuse everyone, but "the new kids on the block" just emerged. Open-source enterprise vendors like LogicBlaze and Mule are added to the mix to provide their ESB solutions.

Proof of Concept

The System i is a remarkable server that plays very well in the SOA space. Many leading companies use it across highly competitive markets like finance, healthcare, manufacturing and distribution, and retail. Companies are seeking new ways to generate enterprise agility from their current infrastructure. Small to medium-size companies might see an ESB at the end of their horizon, but larger-scale companies with disparate systems, including the System i, may find the ESB to be the key solution in preventing disjointed information silos throughout the enterprise. If your company is in the process of undertaking SOA or ready to adopt SOA in a more gradual manner, it is a good idea to have an ESB as a long-term strategy.

Implement a proof-of-concept (POC) before running with the bus. It is important to choose the POC to expose your existing infrastructure. Do not choose a POC that is extremely hard to explain and takes six months to undertake. Something tangible can be done within six to eight weeks.

Here are some key POC points to consider:

• SOA standards—Make sure the ESB supports Web services created from RPG programs or prototypes created from service programs. Make sure that the Web service can be consumed by another disparate system or a user interface using a Web browser created from JSP or ASP. The underlying architecture uses the ESB as your middleware to communicate. Use your existing protocols. Do not test MQ if your enterprise does not currently support it.
• Provider and consumer of service—Your RPG program must also be able to consume a service created from your other servers.
• Variety of end-points—Connect to multiple service end-points (maybe one service to the System i and another to a System z or p). This allows you to process multiple business services within a single pipeline. This proves the ESB orchestration by taking part of business services based on rules and conditions as you exposed them. The mere fact that you can mix and match any of these business services proves the VETO (Validate, Enrich, Transform, Operate) pattern. The VETO pattern ensures that consistent data is routed throughout your ESB.

Having these proof points might be sufficient to initially jumpstart your POC. Evaluate the multiple vendors that I listed earlier. Make sure the vendor ESB has integrated with a System i and other systems in your enterprise. Your POC must be well-scripted. Compare apples to apples when scoring your POC. Some vendors will add functionality to juice up their showcase. Do not bite on the bait. I recommend developing a scorecard and listing all the proof points, showing different evaluators (business users, software implementers, and ITG). Include on the scorecard the POC lifecycle: implementation, design, development, testing, deployment, and execution. You may add monitoring and management if you have time. The last but probably most important piece of the POC is this: Do not let the vendor drive the POC. Don't let the vendor create this on its site and come back with a finished component. Have the vendors come in and illustrate their ESBs. The vendors are not going to create your WSDL for you, so come prepared with all the Web services already created in your systems.

If you are ready to take on the bus, follow the simple rules I have illustrated. Don't fall into the trap of a vendor's opinion on how the ESB process should work. Match your real-world needs and processes to the vendor's ESB. Always remember that the initial role of an ESB is to leverage your existing application.

Alex Nubla works as the Enterprise Architect for Health Net. He used to write technical articles for Powerbuilder magazine, Midrange Computing magazine, and iSeries News magazine (even way back in News 3/x days). Alex lives in Southern California now, after relocating from Charlotte, North Carolina, where he worked for Bank of America as the V.P. in charge of technical support for System i. 

Alex Nubla is a subject matter expert specializing in Service-Oriented Architecture (SOA). He has served in a variety of technical and business roles across a broad range of industries, including banking, finance, supply chain, and healthcare to name a few. Alex' current post is as a SOA Strategic Lead and a Director for Health Net Inc.  Email. This email address is being protected from spambots. You need JavaScript enabled to view it..

Also Read

The Power of Writing Code in a Low-Code Solution

TechTip: Let’s Have a Date, Part 2

Going off the Rails