How do programmers and managers use tools to maintain their programs?

The results of the IT Incendiary 2008 RPG Maintenance Survey are in, with 250 programmers and managers reporting how they use tools to help them maintain the thousands of programs in their libraries.

The survey was divided into two sections: participants who maintained RPG and participants who managed RPG maintenance programmers. The results were both intriguing and revealing.

Demographics

Of the 250 participants, the sectors represented in the sample should be a familiar set of demographics to most traditional midrange observers.

The size of the organizations surveyed also represented a good cross-section of midrange organizations.

Of the respondents, 78 percent said they actively maintained RPG programs, 15 percent said they supervised others who maintained RPG, and 7 percent said they did both.

Size of RPG Staff

When asked how many programmers were currently employed at these organizations, more than 50 percent responded that they had fewer than five RPG programmers on staff.

Use of Consultants

One of the more interesting demographic pieces of information that was derived from the management part of the survey was how these organizations use the services of RPG consultants. When managers were asked, "How many RPG consultants did your company employ last year?," the results were unusually large. Over 60 percent said they had hired at least one consultant, and 28 percent said they had hired more than two.

This is an interesting management statistic when compared with their perception of the difficulty these managers have in finding qualified RPG programmers.

Only 22 percent of managers felt it was easy or moderately easy to recruit qualified professionals, while 58 percent said it was difficult or very difficult to find talent.

This difficulty may explain why the use of consultants is unusually high. And when these figures are aligned with how RPG programming resources are utilized, the inferences are quite revealing.

•·                     Thirty-five percent of the respondents said that more than half of their RPG resources are spent maintaining currently existing code.

•·                     Thirty-six percent said that a third of their resources maintain RPG code.

•·                     Only 14 percent said that less than a third of their resources are devoted to RPG maintenance.

•·                     Less than 1 percent said that they devoted no time at all to RPG maintenance.

This, combined with the high use of consultants, demonstrates that the majority of IT personnel expenditures are being consumed by the day-to-day RPG maintenance functions. These are not new programs being created, but old programs that needed modification.

Libraries of RPG Code

So how much code are we talking about?

•·                     More than 47 percent of the respondents said they had more than 1000 RPG programs being actively maintained.

•·                     Almost 27 percent said that they worked on more than 500 programs.

•·                      Only 22 percent said they worked on fewer than 500 programs.

•·                     A surprising 4 percent said they had no idea how many programs were being maintained.

When one considers the doom and gloom purveyors that contend that RPG is dead, it would appear these statistics paint a very different picture. With so many active sites and programs in need of maintenance, it would seem that some RPG programmers have plenty of opportunity in the maintenance realm.

Versions of RPG

It's also interesting to see what versions of RPG are currently being supported. According to the participants:

•·                     67.2 percent are supporting RPG IV.

•·                     66.4 percent are supporting ILE RPG.

•·                     54 percent are supporting RPG/400.

•·                     28.8 percent are still supporting System/38 RPG III code.

•·                     A surprising 15.2 percent say they are still supporting legacy RPG II left from the days of the System/36 and before.

•·                     5.4 percent said they are supporting some other form of RPG, including RPGFree.

Analysis

What do these statistics mean?

We believe that the statistics reveal a different world of programming than the one that IBM and other industry observers see.

The statistics seem to show that RPG versions never really die; they continue to live on and require some form of maintenance long after active development within the language has ceased.

This may be disheartening to the language buffs who are always eager to try the latest version of the compiler, but it's also a powerful value statement to organizations that the code they have written can still retain productive significance, long after the particular RPG language version has been abandoned by the language labs in Toronto.

Implications

It also bears witness to a need to sustain the older skills once learned by programmers of RPG in days gone by. Without those skills within the organization, the only recourse for maintaining the business logic of older modules is to rewrite them in a newer version of the compiler.

But reconstructing code may not necessarily be the best use of IT programming resources, especially when so many other projects need attention.

What may make more sense is investing in tools that can reduce the difficulty of code comprehension so that the older modules can be more efficiently maintained.

It means more than the adage "If it ain't broke, don't fix it!" It could mean "Find the best tools to fix it before it breaks."

This raises a question: What tools are these organizations using to maintain the code?

Tools Currently in Use

We asked the survey participants to list those tools that they currently used to maintain code, and we found an interesting picture:

•·                     48 percent were using only the built-in tools of IBM OS.

•·                     26 percent were relying on WDSC, the tool that IBM is currently pushing.

•·                     Only 24 percent were using some third-party tool to help them maintain RPG.

This is what we found out about the third-party tools in use:

•·                     Hawkeye Pathfinder, created by Hawkeye Information Systems out of Fort Collins, Colorado, was used by 29 percent of the organizations surveyed.

•·                     IBM's Rational Software Delivery Platform for i (RDi) is being used at only 10 percent of these facilities.

•·                     Databorough's X-Analysis is also being used by 10 percent of these shops.

•·                      All other third-party tools combined were used in less than 7 percent of the sites surveyed.

This lack of penetration of a potential market for tools may represent the most telling story of how poorly IT management understands the problems of maintaining RPG. But what is more telling is what those shops that are using no third-party tools say they are using to help in their jobs.

•·                     40 percent are still using SEU.

•·                     32 percent are still using PDM.

•·                     Only 12 percent are using the DEBUG command.

•·                     9 percent said they are using nothing.

These survey results could represent, arguably, the lack of need for more tools for maintenance. But, considering that IBM has said that it is no longer enhancing SEU and PDM, it may more accurately reflect that management has yet to come to terms with a need that has remained invisible to them.

Managing Expectations

So we asked the managers how they rate their shops' ability to bring maintenance projects to completion, the quality and the stability of the code they received, and the overall satisfaction with the processes of RPG maintenance.

Unexpected Delays in Maintenance

First, we asked these shops to rate the frequency of delays they experienced in meeting maintenance schedules. Twenty-seven percent said they experienced delays weekly. Twenty-eight percent experienced delays at least once a month.

Of course, this is a subjective measure by managers, as maintenance projects run a gamut of difficulty and each organization has its own view of what an unexpected delay might comprise.

But the results are surprisingly consistent when the question of deadlines is asked.

Missed Deadlines for Maintenance Completion

Sixty-five percent of those surveyed said that maintenance deadlines were missed on a yearly basis, with 19 percent saying they missed deadlines two or three times a month. Only 13 percent said they rarely missed maintenance deadlines.

Application Crashes

Of course, a missed deadline is not so bad as long as those applications maintained are solid, right?

Yet 4 percent of those surveyed said that maintained code crashes predictably once a week. Eight percent said maintained code crashes at least 2 or 3 times a week. Thirteen percent said maintained code crashes at least once a month, and 11 percent said the code crashes two or three times a month.

Twenty-three percent said that their maintained code rarely crashes, but while this statistic is reassuring for those that have solid software, the larger context is very sobering: 55 percent of sites surveyed could identify that they were experiencing crashes after a programmer has "fixed" them at least once a year.

Management Satisfaction

So how does upper management feel about the state of RPG maintenance, as represented by this survey?

We asked about maintenance timeliness, appropriateness, quality, and cost. The results are intriguing:

•·                     Thirty-eight percent said that upper management rated maintenance timeliness as low.

•·                     Six percent said that management did not feel the maintenance was appropriate to the task.

•·                     Ten percent said that management did not feel that the quality of maintenance was adequate for the effort.

•·                     Thirty-eight percent said that management considered the cost of maintaining code to be too high.

The Programmers' Take

So what, exactly, is the problem? How is it that our shops spend so much time, effort, and money on maintaining code that should be known entities, code that has existed for years?

We decided to ask this question of the maintenance programmers themselves.

We asked the programmers to rate the "maintainability" aspects of their companies' code base. This is equivalent to asking a surgeon to rate the patients that he is operating on. The scale was from 1 to 5, with 1=poor, 3=adequate, and 5=excellent.

The results of the survey indicate that--as frustrated as management may feel--the maintenance programming staff is even more acutely frustrated.

Technical Challenges for RPG Maintenance

We asked the programmers to rate their ability to comprehend the impact of the code (knowledge of the systems upon which they work), the documentation that they encountered, the stability of the modules themselves, the quality of past modifications they came across, the quality of the interfaces between individual modules, and their ability to perform regression testing to track errors and fix problems.

What we discovered is that these programmers are faced with a combination of daunting challenges that create unexpected delays, missed deadlines, and cost overruns on a truly global scale.

•·                     14.4 percent said they had less than adequate knowledge of the systems they were maintaining.

•·                     52 percent said that documentation of their systems was poor or "inadequate."

•·                     Only 12 percent said documentation was, on average, "adequate."

•·                     Only 2 percent said that documentation was "excellent."

•·                     22.4 percent said the modules they were maintaining were unstable.

•·                     Only 22.8 percent said the modules were adequately stable.

•·                     Only 33 percent said that the stability of modules was better than adequate.

•·                     30 percent said that past modifications were poorly conceived or constructed.

•·                     16 percent said that interfaces to other modules and systems were poorly conceived or executed.

•·                     54 percent said they had limited or poor regression-testing capabilities.

Roundup

These statistics, if they are indeed representational of the larger RPG community, paint a sobering picture of our RPG application base and the ability of our programmers to readily maintain and bring change to the organization's code.

Lack of documentation is at one corner of these challenges. Poorly conceived interfaces are at another. Stability is the third corner of the challenge. But by far, the greatest challenge is the inability of programmers to see the impact of their modifications or perform regression testing on modules.

When one combines these views, voiced by managers and programmers, into a diagnosis of our RPG systems, one wonders why better diagnostic tools have not been developed--or, on the other hand, if these tools are available, why they haven't been more widely applied and accepted by the management of the programming shops.

Managing the code of our information systems currently appears to be consuming the majority of our time as programmers and managers. Yet, based upon this survey, it appears we may have neglected a basic management requirement: the responsibility to accumulate or develop the necessary tools to adequately diagnose and sustain the RPG applications in our libraries.