When I wrote the RPG xTools, I included a procedure that converts DB2/400 files to comma-separated values (CSV) format. Originally, I used this routine on a number of small files containing roughly several hundred to a few thousand records. It seemed to work quickly enough.

But then, one customer wanted to convert 46 million records. So I ran a small test over approximately 46,000 records to see how long it would take to convert to CSV. It seemed odd, but the tests showed correctly showed that it would take a few weeks to convert the file.

Of course, this situation was unacceptable. So I went back to the drawing board and did some performance studies and analysis. It turns out there were some issues with redundant logic and a few situations in which I was declaring rather large local variables that we also initialized each time a field in a record was converted.

Reengineering the CSV conversion routine wasn't all that challenging, but it was a lesson in optimization. Sadly, it seems that bad coding (coding for poor performance) is rather easy to do in RPG IV.

It's the Parameters, Stupid

It turns out that things we do in RPG IV to pass parameters to subprocedures can work against us. Certainly, there is some expected, yet subtle overhead in the call performance. But the increased overhead of adding CALLP (CALL with prototype) to a procedure should not make or break a routine.

In fact, back when IBM was trying to sell ILE (called "NPM" during development) to some industry analysts, one insider asked, "What's the benefit of reengineering everything just to try to make the CALL run faster? Why not just make the CALL faster?" IBM's answer was that languages other than RPG III and CL didn't run well in OS/400 and they had to create a single runtime environment for everything. The insider went on to ask, "Will the CALL be faster as a result?" The IBMer responded, "It better be, or somebody will get shot!"

Ironically, based on the V3R1 tests I ran, somebody in Rochester should have been "shot." But I digress....

Today, nearly everybody is using RPG IV, and many are trying to move to subprocedures. But there has been a somewhat quiet debate stirring regarding the performance of subroutines vs. subprocedures. As I mentioned in a recent article, EXSR is nothing more under the covers than a GOTO/branch instruction, whereas a procedure call requires several more instructions. Consequently, there is more overhead in a subprocedure call than in EXSR. No question about it.

But is it reasonable or excessive? It turns out it could be better, as it always can, but it is very acceptable. The problem involves with the extremely poor way IBM has implemented return values and parameter passing to subprocedures.

While infinitely faster than program-to-program calls, subprocedure calls can suffer if you pass parameters conveniently rather than effectively.

I recently ran a series of tests to timestamp study the overhead of calling a procedure with various types of parameter settings. Each variation accomplished the same task, but the results of passing parameters one way as opposed to another were drastic. Let's look at the various styles of parameter passing.

Before we begin, let me say that passing parameter using the standard or default method of "by reference" is very efficient. In fact, it is the second most efficient method, according to my test results.

Constant Parameters

Parameters passed by reference that also include the CONST keyword are becoming more and more popular. When it comes to performance, the CONST keyword is a good performer, but it can also cause problems if used blindly.

A fast-growing practice with character parameters is to include the VARYING keyword along with the CONST keyword. This allows the parameter to accept both fixed-length and variable-length character fields. The compiler converts fixed-length fields (or literals) to variable-length fields automatically when the procedure is called. It does this by copying the data to a compiler-generated temporary variable.

This extra step adds additional overhead to the procedure call. Why? Because copying the parameter value from a fixed-length field to a temporary variable-length field takes a little time.

Here's an example of the CVTCASE procedure prototype using CONST but not using VARYING:

Here is the same prototype with the addition of the VARYING keyword:

The benefit of having VARYING in addition to CONST is that the procedure can use the %LEN built-in function to determine the length of the incoming parameter value. Also, most RPG IV opcodes are optimized to work more efficiently with VARYING fields since they only process the data in the field as indicated by the field's current length.

So are CONST and VARYING bad? No, but it if VARYING isn't necessary to the success of a procedure, don't use it.

The CONST keyword on the other hand, while offering you greater flexibility for your parameters, can not only speed up a procedure call, but also slow it down. For example, if you define a parameter as 7P2 (7 packed with 2 decimals) and include the CONST keyword, the compiler allows you to pass not only a 7P2 field, but also any numeric value. So if you specify a literal or even a 4-byte binary value, the compiler will convert that value into 7P2 and send it to the subprocedure.

Normally, this is great, but it can slow down the procedure call because everything except 7P2 values are copied to temporary result fields, and then that copy is passed to the procedure. Again, additional overhead.

Granted, copying numbers isn't as severe an issue as character fields, particularly large character fields, but you get the point.

Regular "By Reference" Parameters

I've already mentioned that passing a parameter by reference with CONST can be the fast way to call a procedure. But this only applies when the value being passed already matches the parameter definition.

Traditional "by reference" parameters offer the second-best performance when calling a procedure because they limit the format of the data being specified for the procedure to the parameter definition. That is, a parameter defined as a 7P2 value is limited to accepting fields that are defined as 7P2. So if you attempt to pass a literal (constant) or a packed field with a different length or different decimal positions, the compiler will give you an error.

Character fields have a similar restriction. When a regular character parameter is defined by reference, you have to pass a value that is the same length as or longer than the parameter definition. The compiler does allow longer values because the procedure may ignore those extra characters, but it does not allow shorter character fields to be passed because the procedure could touch the bytes not passed to it.

Here's an example of a procedure prototype with a parameter passed by reference.

Since the compiler will only allow the above prototype to be passed a field that is at least as long as the parameter, we're sort of at a disadvantage if we specify 65535 for the parameter length.

The solution is to use the OPTIONS keyword. Specifying OPTIONS(*VARSIZE) for a parameter removes the size restriction from the parameter. You may pass shorter or longer values. Of course, by doing this, you're telling the compiler that you have a clever way to figure out if the caller sent you data that isn't the same length as the parameter.

One method used by OS/400 and i5/OS APIs and some RPG xTools subprocedures is to pass an addition "parameter length" parameter. This additional parameter tells the procedure the length of the original parameter. Of course, the caller has to provide accurate information in this case; otherwise, you could have a "learning experience." Here's an example:

Note that I used the CONST keyword on the length parameter to allow the caller to specify something useful, such as %SIZE(myField). CONST allows this type of value. Now, it's up to my procedure to interpret the length as well as the input string correctly.

What About Return Values?

As you know, a procedure may return a value to the caller. When a procedure is called, the first line of the prototype or procedure interface statement often includes a definition. This definition identifies the kind of data that is optionally sent back to the caller. It allows you to use a procedure just like a function. For example, the following prototype defines a return value for the CVTCASE procedure:

In this example, the return data is a 64K value. It doesn't have to match the parameter's attributes, but in this example, it does.

To call this procedure, you would use the EVAL or one of the conditional opcodes, such as IF or WHEN, as follows:

When you specify a return value, you may also specify the VARYING keyword. This allows the procedure to return only the number of bytes necessary and theoretically return them faster. However, in practice, character return values greater than a few dozen bytes seem to be poor performers.

Here's the bottom line on return values: Use them for an easy user interface, but don't go overboard. If you need to call a procedure 10 million times and it has a 64K return value, consider changing the design to allow the returned data to be sent back via a second parameter instead a return value.

Pointers Are Faster Than "By Reference"

Wouldn't it be nice if you could make things run faster? Well, you can. But you have to use pointers. It turns out, again from my own tests, that passing an address of a field to a procedure is faster than passing the field. Here's an example of a prototype that allows this:

The tricky part is calling the procedure; you have to pass the address of the parameter's value. You do this with the %ADDR built-in function. For example:

In the example above, the address of the two pieces of data are retrieved and stored in pointer variables. Then, those pointers are passed to the procedure.

You can avoid the tricky assignment (copying to pointer variables) if you specify that the parameters are also CONST or VALUE, as follows:

Then, when calling CVTCASE, you would specify %ADDR directly on the procedure call:

Again, my own tests showed that this method appears to run the fastest and to substantially reduce the overhead of procedure calls. In fact, the two fastest methods are by reference and pointer.

Make the Right Choice

The overall performance of any routine is going to be based on several factors, not the least of which is database I/O. But by understanding the options available to you and making the right choice of performance or function, you can eliminate some of the overhead introduced by using procedures over subroutines.

Bob Cozzi is a programmer/consultant, writer/author, and software developer of the RPG xTools, a popular add-on subprocedure library for RPG IV. His book The Modern RPG Language has been the most widely used RPG programming book for nearly two decades. He, along with others, speaks at and runs the highly-popular RPG World conference for RPG programmers.