With proper planning, you can ensure a smooth system integration project.

So, your next big project is to integrate one of your existing systems with another system of which you may or may not have in-depth knowledge. In today's fast-paced development world, new projects involve more than simply writing the next killer app. System integration projects run the gamut from exposing applications or data to the Web (often referred to as application modernization), to integrating new software packages with existing systems, or to blending applications because of a recent merger.

In any case, these projects have common characteristics not present in self-contained, build-from-scratch projects, and this article focuses on exploring some of the differences between these types of projects and the key factors of integration success to help you avoid common pitfalls.

Break Down the Barriers

One common aspect of system integration projects is that they often combine teams that have not previously worked together. Projects may be staffed from various departments within the same company or include third parties that bring in specific expertise or capacity for a defined project.

It is important to choose the right person to run the project, someone who will work well with other technical teams not under their control and who can facilitate the coordination of efforts between the team members. The key is to not place the most knowledgeable or most senior engineer in the role of coordinating with other disparate or geographically dispersed teams. Choose someone who can deal with the mundane administration and who can succinctly communicate with individuals who have different technical backgrounds.

One common aspect of system integration projects is that each team gets to control only what they build and then has to agree with the other team how the integration is to be accomplished. No single team gets to decide how the other team builds the entire solution. Each team should be looking at the other system they are integrating into as a black box that they send information into and/or from which they receive information. This coordination role does not necessarily need to be a senior technical role on the team but does need to be one that can align the technical chest-thumping to project needs while managing scope creep.

Additionally, clearly laying out project goals to all team members is critical. The goals should be explained from the perspective of the ultimate end-user of the systems. Everyone does not actually have to agree. You may hear "What's wrong with the green-screen?" or "But I loved the old accounting system!" Team members have to at least understand the business justification for the project. Doing this early in the project can eliminate many frustrations later.

What to Integrate

One of the largest and most successful integration projects I participated in was one in which, once the business objectives were defined, the next planning phase then simply defined what business information needed to be sent to support the business objectives. We agreed very early in the project which areas made the most sense for each team to be responsible for. While this may seem obvious, in comparison to other projects, this resulted in literally no discussion or disagreement about which team (my organization or our partner) had responsibility for whatever needed to be built as the project progressed.

It is also critical in these projects to require early involvement by experts in both systems to be integrated. If experts or documentation for legacy applications are unavailable, then developers who can dig into an application and come up with a reasonable understanding quickly are crucial. The goal here is not to completely document an existing application, but to find the best way for it to communicate with the other applications. Avoid trying to isolate every data element at this point in the project (that will be reviewed in the next section).

How to Integrate

In my opinion, there is no single technology solution for all system integrations. I have done system integration projects that involved some or all of the following: RPG, COBOL, PL/I, Java, commands, stored procedures, and product-specific APIs and user exits. For some projects, choosing a technology is a simple decision. For example, if I were integrating into a product like Eclipse or one based on Eclipse technology, I would use the built-in extensibility features of Eclipse and deliver this as an Eclipse plug-in. At the other end of the spectrum, two RPG applications that need to talk to each other and that might also need to expose their data to other applications in the future could be integrated in a variety of ways.

This is where projects can quickly become a technical slugfest where people come with a variety of preferred (aka "best") techniques. Ask yourselves the following questions and your choices should be whittled down quickly:

•  How complex is the system integration? Projects that involve exposing small sets of data from one application to another should consider different approaches than projects that expose virtually the entire user interface to the Web.

•  What integration mechanism already exists in the application? Most in-house legacy systems tend to be built assuming little or no outside connectivity with other applications.

•  What integration mechanisms exist in the application being targeted for integration? If integrating with a commercial application, development environment, or modernization tool, integration capabilities are often already built in. Those should be strongly considered as there is little point in reinventing the wheel without having a good reason to do so.

•  What are the real-time data requirements? While most system integrations these days need to be real-time, some do not. Invoicing once a night may mean integrations can be done differently than if invoices are sent on demand. A business application requiring online ordering over the Web has different real-time data requirements than an internal application with a smaller set of users without time-critical data. Don't over-engineer just because you can.

•  Does the integration involve just data or also business logic? Is the business logic and/or the data bi-directional? Certain techniques lend themselves better to bi-directional approaches. Some system integrations involve just exposing inventory data to a Web application for the customer and offering only viewing capabilities; others are fully transactional. These all have major impacts on which approach should or could be used.

•  Do you need to align with organizational initiatives such as SOA? Could the development be shared by other existing or future projects, or is it really single use? If strictly single use, it is probably outside the bounds of SOA because a key aspect of most SOA initiatives is that components or APIs being developed need to be consumable or usable by other business applications or processes.

•  What are the skills and interests of the team? Some developers struggle with the move to Java and object-oriented technologies. Do you have the right team in place to support the technology you choose?

Examine the above questions and make your choices, balancing the anticipated future needs with near-term project goals.

Project-Driven Process and Workflow

While process and workflow can conjure up images of SOX and audit requirements, having the right project management capabilities is important to system integration projects.

The disparate nature of most system integration projects means you will have two (or more) development teams building to the same end, and all need to be keenly aware of any changes to interfaces and the activities of others. One key here is to choose a risk-appropriate project-tracking and workflow approach and identify what is really important to manage, measure, and control. For smaller projects, a simpler workflow may be suitable. Whatever the project size, keep in mind that system integration projects often have higher visibility due to multiple development teams and outside resources being used and therefore may demand a management system that can provide visibility throughout the project and organization.

Ask yourself these questions:

•  What is the risk of failure? Rolling out a new payroll application or a new online Web site may prove critical to your customer's success, and any failures could have significant revenue or legal repercussions. However, an internal application with a small set of (patient) users without great time sensitivity could represent much less risk.

•  What data is important to be collected and reported? What is the cost to collect and present the data in a cohesive and/or customized fashion?

•  What level of management needs to be kept informed of the project? Given the interdepartmental/cross-organizational aspect of projects, all managers should be looking at the same, not rival, project metrics.

•  What level of involvement should be sought from the user community (and customer community, for externally visible applications)? For example, should they review alpha and/or beta releases of a new Web site? Would this be prudent, and how will they communicate their findings/experiences with development teams?

Based on the size and goals of the project, a commercial Application Lifecycle Management (ALM) solution can help deliver and manage much of the areas identified above. Choosing one that is flexible and robust enough to meet the needs of the variety of vested parties is critical to success.

Smooth Sailing

System integration projects can add significant complexity to project management. Key aspects of these system integration projects to understand include appropriate team structure, clear project objectives, separation of what needs to be integrated from how it is technically accomplished, and a method for choosing the best integration methodology based on the needs of the project. Finally, given the multi-disciplinary aspects of these projects, having the right mechanism in place to stay on top of the project is important to assure timely delivery of the agreed-upon requirements. If you include the aforementioned aspects of system integration projects into your plan, your next system integration project can go as smoothly as any other project.